2026-02-12T12:21:54.111 INFO:root:teuthology version: 0.0.1.dev29+gc433f1062 2026-02-12T12:21:54.126 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-12T12:21:54.202 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/moagrawa-2026-02-10_06:35:43-crimson-rados-main-distro-debug-trial/42514 branch: main description: crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} tasks/rbd_python_api_tests_old_format} email: null first_in_suite: false flavor: debug job_id: '42514' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: moagrawa-2026-02-10_06:35:43-crimson-rados-main-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug ms: 20 debug osd: 20 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore main device type: RANDOM_BLOCK_SSD seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - \(SLOW_OPS\) - slow request sha1: 5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: extra_system_packages: - python3-pytest flavor: debug sha1: 5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 workunit: branch: seastore_io_uring env: CRIMSON_COMPAT: '1' sha1: 5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 owner: scheduled_moagrawa@soko04.front.sepia.ceph.com priority: 1000 repo: https://github.com/ceph/ceph.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 2425 sha1: 5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 sleep_before_teardown: 0 suite: crimson-rados suite_branch: seastore_io_uring suite_path: /home/teuthworker/src/github.com_mohit84_ceph_5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/qa suite_relpath: qa suite_repo: https://github.com/mohit84/ceph.git suite_sha1: 5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 targets: trial050.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDG7oXs0XnfkuI46eO/CRk9qtDe7WEwaHkYWg3wGGv1tSLFo/eNvhNvgO1yJA0Qq3+T4CFdwTgl17clpTR+9Xds= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 - workunit: clients: client.0: - rbd/test_librbd_python.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: c433f1062990a0488dc29a553589bc609a460691 timestamp: 2026-02-10_06:35:43 tube: trial user: moagrawa verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.2161924 2026-02-12T12:21:54.203 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_mohit84_ceph_5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/qa; will attempt to use it 2026-02-12T12:21:54.203 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_mohit84_ceph_5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/qa/tasks 2026-02-12T12:21:54.203 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-12T12:21:54.205 INFO:teuthology.task.internal:Checking packages... 2026-02-12T12:21:54.205 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0' 2026-02-12T12:21:54.205 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-12T12:21:54.205 INFO:teuthology.packaging:ref: None 2026-02-12T12:21:54.205 INFO:teuthology.packaging:tag: None 2026-02-12T12:21:54.205 INFO:teuthology.packaging:branch: main 2026-02-12T12:21:54.205 INFO:teuthology.packaging:sha1: 5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 2026-02-12T12:21:54.205 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=main 2026-02-12T12:21:54.352 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5273.g2bf58db4 2026-02-12T12:21:54.353 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-12T12:21:54.364 INFO:teuthology.task.internal:no buildpackages task found 2026-02-12T12:21:54.364 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-12T12:21:54.378 INFO:teuthology.task.internal:Saving configuration 2026-02-12T12:21:54.391 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-12T12:21:54.404 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-12T12:21:54.489 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial050.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/moagrawa-2026-02-10_06:35:43-crimson-rados-main-distro-debug-trial/42514', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-12 12:18:38.036785', 'locked_by': 'scheduled_moagrawa@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDG7oXs0XnfkuI46eO/CRk9qtDe7WEwaHkYWg3wGGv1tSLFo/eNvhNvgO1yJA0Qq3+T4CFdwTgl17clpTR+9Xds='} 2026-02-12T12:21:54.489 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-12T12:21:54.500 INFO:teuthology.task.internal:roles: ubuntu@trial050.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-12T12:21:54.500 INFO:teuthology.run_tasks:Running task console_log... 2026-02-12T12:21:54.594 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7e08b6cfbec0>, signals=[15]) 2026-02-12T12:21:54.594 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-12T12:21:54.605 INFO:teuthology.task.internal:Opening connections... 2026-02-12T12:21:54.605 DEBUG:teuthology.task.internal:connecting to ubuntu@trial050.front.sepia.ceph.com 2026-02-12T12:21:54.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T12:21:54.667 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-12T12:21:54.677 DEBUG:teuthology.orchestra.run.trial050:> uname -m 2026-02-12T12:21:54.740 INFO:teuthology.orchestra.run.trial050.stdout:x86_64 2026-02-12T12:21:54.741 DEBUG:teuthology.orchestra.run.trial050:> cat /etc/os-release 2026-02-12T12:21:54.796 INFO:teuthology.orchestra.run.trial050.stdout:NAME="CentOS Stream" 2026-02-12T12:21:54.796 INFO:teuthology.orchestra.run.trial050.stdout:VERSION="9" 2026-02-12T12:21:54.796 INFO:teuthology.orchestra.run.trial050.stdout:ID="centos" 2026-02-12T12:21:54.796 INFO:teuthology.orchestra.run.trial050.stdout:ID_LIKE="rhel fedora" 2026-02-12T12:21:54.796 INFO:teuthology.orchestra.run.trial050.stdout:VERSION_ID="9" 2026-02-12T12:21:54.796 INFO:teuthology.orchestra.run.trial050.stdout:PLATFORM_ID="platform:el9" 2026-02-12T12:21:54.796 INFO:teuthology.orchestra.run.trial050.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-12T12:21:54.796 INFO:teuthology.orchestra.run.trial050.stdout:ANSI_COLOR="0;31" 2026-02-12T12:21:54.796 INFO:teuthology.orchestra.run.trial050.stdout:LOGO="fedora-logo-icon" 2026-02-12T12:21:54.797 INFO:teuthology.orchestra.run.trial050.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-12T12:21:54.797 INFO:teuthology.orchestra.run.trial050.stdout:HOME_URL="https://centos.org/" 2026-02-12T12:21:54.797 INFO:teuthology.orchestra.run.trial050.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-12T12:21:54.797 INFO:teuthology.orchestra.run.trial050.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-12T12:21:54.797 INFO:teuthology.orchestra.run.trial050.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-12T12:21:54.797 INFO:teuthology.lock.ops:Updating trial050.front.sepia.ceph.com on lock server 2026-02-12T12:21:54.883 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-12T12:21:54.896 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-12T12:21:54.909 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-12T12:21:54.909 DEBUG:teuthology.orchestra.run.trial050:> test '!' -e /home/ubuntu/cephtest 2026-02-12T12:21:54.924 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-12T12:21:54.936 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-12T12:21:54.936 DEBUG:teuthology.orchestra.run.trial050:> test -z $(ls -A /var/lib/ceph) 2026-02-12T12:21:54.981 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-12T12:21:55.089 INFO:teuthology.run_tasks:Running task kernel... 2026-02-12T12:21:55.112 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-12T12:21:55.112 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-12T12:21:55.113 DEBUG:teuthology.orchestra.run.trial050:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-12T12:21:55.127 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T12:21:55.127 DEBUG:teuthology.orchestra.run.trial050:> uname -r 2026-02-12T12:21:55.183 INFO:teuthology.orchestra.run.trial050.stdout:5.14.0-645.el9.x86_64 2026-02-12T12:21:55.183 INFO:teuthology.task.kernel:Running kernel on trial050: 5.14.0-645.el9.x86_64 2026-02-12T12:21:55.183 DEBUG:teuthology.orchestra.run.trial050:> sudo yum install -y kernel 2026-02-12T12:22:03.608 INFO:teuthology.orchestra.run.trial050.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 8.9 MB 00:08 2026-02-12T12:22:07.286 INFO:teuthology.orchestra.run.trial050.stdout:CentOS Stream 9 - AppStream 8.5 MB/s | 27 MB 00:03 2026-02-12T12:22:17.973 INFO:teuthology.orchestra.run.trial050.stdout:CentOS Stream 9 - CRB 957 kB/s | 7.8 MB 00:08 2026-02-12T12:22:18.798 INFO:teuthology.orchestra.run.trial050.stdout:CentOS Stream 9 - Extras packages 200 kB/s | 20 kB 00:00 2026-02-12T12:22:19.451 INFO:teuthology.orchestra.run.trial050.stdout:Extra Packages for Enterprise Linux 35 MB/s | 20 MB 00:00 2026-02-12T12:22:22.672 INFO:teuthology.orchestra.run.trial050.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-12T12:22:23.744 INFO:teuthology.orchestra.run.trial050.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-12T12:22:23.764 INFO:teuthology.orchestra.run.trial050.stdout:Dependencies resolved. 2026-02-12T12:22:23.765 INFO:teuthology.orchestra.run.trial050.stdout:================================================================================ 2026-02-12T12:22:23.765 INFO:teuthology.orchestra.run.trial050.stdout: Package Architecture Version Repository Size 2026-02-12T12:22:23.765 INFO:teuthology.orchestra.run.trial050.stdout:================================================================================ 2026-02-12T12:22:23.765 INFO:teuthology.orchestra.run.trial050.stdout:Installing: 2026-02-12T12:22:23.766 INFO:teuthology.orchestra.run.trial050.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-12T12:22:23.766 INFO:teuthology.orchestra.run.trial050.stdout:Installing dependencies: 2026-02-12T12:22:23.766 INFO:teuthology.orchestra.run.trial050.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-12T12:22:23.766 INFO:teuthology.orchestra.run.trial050.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-12T12:22:23.766 INFO:teuthology.orchestra.run.trial050.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-12T12:22:23.766 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:22:23.766 INFO:teuthology.orchestra.run.trial050.stdout:Transaction Summary 2026-02-12T12:22:23.766 INFO:teuthology.orchestra.run.trial050.stdout:================================================================================ 2026-02-12T12:22:23.766 INFO:teuthology.orchestra.run.trial050.stdout:Install 4 Packages 2026-02-12T12:22:23.766 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:22:23.766 INFO:teuthology.orchestra.run.trial050.stdout:Total download size: 88 M 2026-02-12T12:22:23.766 INFO:teuthology.orchestra.run.trial050.stdout:Installed size: 135 M 2026-02-12T12:22:23.767 INFO:teuthology.orchestra.run.trial050.stdout:Downloading Packages: 2026-02-12T12:22:23.852 INFO:teuthology.orchestra.run.trial050.stdout:(1/4): kernel-5.14.0-677.el9.x86_64.rpm 28 MB/s | 768 kB 00:00 2026-02-12T12:22:23.927 INFO:teuthology.orchestra.run.trial050.stdout:(2/4): kernel-core-5.14.0-677.el9.x86_64.rpm 168 MB/s | 17 MB 00:00 2026-02-12T12:22:24.005 INFO:teuthology.orchestra.run.trial050.stdout:(3/4): kernel-modules-core-5.14.0-677.el9.x86_6 201 MB/s | 31 MB 00:00 2026-02-12T12:22:24.104 INFO:teuthology.orchestra.run.trial050.stdout:(4/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 142 MB/s | 40 MB 00:00 2026-02-12T12:22:24.104 INFO:teuthology.orchestra.run.trial050.stdout:-------------------------------------------------------------------------------- 2026-02-12T12:22:24.105 INFO:teuthology.orchestra.run.trial050.stdout:Total 260 MB/s | 88 MB 00:00 2026-02-12T12:22:24.292 INFO:teuthology.orchestra.run.trial050.stdout:Running transaction check 2026-02-12T12:22:24.497 INFO:teuthology.orchestra.run.trial050.stdout:Transaction check succeeded. 2026-02-12T12:22:24.497 INFO:teuthology.orchestra.run.trial050.stdout:Running transaction test 2026-02-12T12:22:24.844 INFO:teuthology.orchestra.run.trial050.stdout:Transaction test succeeded. 2026-02-12T12:22:24.844 INFO:teuthology.orchestra.run.trial050.stdout:Running transaction 2026-02-12T12:22:25.935 INFO:teuthology.orchestra.run.trial050.stdout: Preparing : 1/1 2026-02-12T12:22:26.218 INFO:teuthology.orchestra.run.trial050.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-12T12:22:26.238 INFO:teuthology.orchestra.run.trial050.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T12:22:27.998 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T12:22:28.027 INFO:teuthology.orchestra.run.trial050.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T12:22:31.116 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T12:22:31.122 INFO:teuthology.orchestra.run.trial050.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T12:22:34.159 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T12:23:04.482 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T12:23:04.504 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-12T12:23:04.672 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T12:23:04.672 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-12T12:23:04.673 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T12:23:04.673 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T12:23:04.725 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T12:23:04.725 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:23:04.725 INFO:teuthology.orchestra.run.trial050.stdout:Installed: 2026-02-12T12:23:04.725 INFO:teuthology.orchestra.run.trial050.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-12T12:23:04.725 INFO:teuthology.orchestra.run.trial050.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-12T12:23:04.725 INFO:teuthology.orchestra.run.trial050.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-12T12:23:04.725 INFO:teuthology.orchestra.run.trial050.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-12T12:23:04.725 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:23:04.726 INFO:teuthology.orchestra.run.trial050.stdout:Complete! 2026-02-12T12:23:04.782 DEBUG:teuthology.orchestra.run.trial050:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T12:23:04.808 INFO:teuthology.orchestra.run.trial050.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T12:23:04.809 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T12:23:04.809 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-12T12:23:04.809 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-12T12:23:04.809 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-12T12:23:04.809 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-12T12:23:04.809 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial050.front.sepia.ceph.com, path=None, version=distro) 2026-02-12T12:23:04.810 DEBUG:teuthology.orchestra.run.trial050:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T12:23:04.873 INFO:teuthology.orchestra.run.trial050.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T12:23:04.874 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T12:23:04.874 DEBUG:teuthology.orchestra.run.trial050:> sudo rpm -qi grub2-tools 2026-02-12T12:23:04.941 INFO:teuthology.orchestra.run.trial050.stdout:Name : grub2-tools 2026-02-12T12:23:04.941 INFO:teuthology.orchestra.run.trial050.stdout:Epoch : 1 2026-02-12T12:23:04.941 INFO:teuthology.orchestra.run.trial050.stdout:Version : 2.06 2026-02-12T12:23:04.942 INFO:teuthology.orchestra.run.trial050.stdout:Release : 118.el9 2026-02-12T12:23:04.942 INFO:teuthology.orchestra.run.trial050.stdout:Architecture: x86_64 2026-02-12T12:23:04.942 INFO:teuthology.orchestra.run.trial050.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-12T12:23:04.942 INFO:teuthology.orchestra.run.trial050.stdout:Group : Unspecified 2026-02-12T12:23:04.942 INFO:teuthology.orchestra.run.trial050.stdout:Size : 8393909 2026-02-12T12:23:04.942 INFO:teuthology.orchestra.run.trial050.stdout:License : GPLv3+ 2026-02-12T12:23:04.942 INFO:teuthology.orchestra.run.trial050.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-12T12:23:04.942 INFO:teuthology.orchestra.run.trial050.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-12T12:23:04.942 INFO:teuthology.orchestra.run.trial050.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-12T12:23:04.942 INFO:teuthology.orchestra.run.trial050.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-12T12:23:04.942 INFO:teuthology.orchestra.run.trial050.stdout:Packager : builder@centos.org 2026-02-12T12:23:04.942 INFO:teuthology.orchestra.run.trial050.stdout:Vendor : CentOS 2026-02-12T12:23:04.942 INFO:teuthology.orchestra.run.trial050.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-12T12:23:04.943 INFO:teuthology.orchestra.run.trial050.stdout:Summary : Support tools for GRUB. 2026-02-12T12:23:04.943 INFO:teuthology.orchestra.run.trial050.stdout:Description : 2026-02-12T12:23:04.943 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:23:04.943 INFO:teuthology.orchestra.run.trial050.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-12T12:23:04.943 INFO:teuthology.orchestra.run.trial050.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-12T12:23:04.943 INFO:teuthology.orchestra.run.trial050.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-12T12:23:04.943 INFO:teuthology.orchestra.run.trial050.stdout:hardware devices. 2026-02-12T12:23:04.943 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:23:04.943 INFO:teuthology.orchestra.run.trial050.stdout:This subpackage provides tools for support of all platforms. 2026-02-12T12:23:04.944 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-12T12:23:04.944 INFO:teuthology.task.kernel:Updating grub on trial050 to boot 5.14.0-677.el9.x86_64 2026-02-12T12:23:04.945 DEBUG:teuthology.orchestra.run.trial050:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-12T12:23:05.010 INFO:teuthology.orchestra.run.trial050.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-12T12:23:05.011 DEBUG:teuthology.orchestra.run.trial050:> sudo command -v grubby && echo yes || echo no 2026-02-12T12:23:05.077 INFO:teuthology.orchestra.run.trial050.stdout:/sbin/grubby 2026-02-12T12:23:05.079 INFO:teuthology.orchestra.run.trial050.stdout:yes 2026-02-12T12:23:05.079 DEBUG:teuthology.orchestra.run.trial050:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-12T12:23:05.143 INFO:teuthology.orchestra.run.trial050.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-12T12:23:05.143 DEBUG:teuthology.orchestra.run.trial050:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-12T12:23:05.215 DEBUG:teuthology.orchestra.run.trial050:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-12T12:23:05.421 INFO:teuthology.orchestra.run.trial050.stderr:Generating grub configuration file ... 2026-02-12T12:23:05.484 INFO:teuthology.orchestra.run.trial050.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-12T12:23:05.491 INFO:teuthology.orchestra.run.trial050.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-12T12:23:05.739 INFO:teuthology.orchestra.run.trial050.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-12T12:23:05.745 INFO:teuthology.orchestra.run.trial050.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-12T12:23:05.777 INFO:teuthology.orchestra.run.trial050.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-12T12:23:05.784 INFO:teuthology.orchestra.run.trial050.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-12T12:23:05.807 INFO:teuthology.orchestra.run.trial050.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-12T12:23:05.814 INFO:teuthology.orchestra.run.trial050.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-12T12:23:05.895 INFO:teuthology.orchestra.run.trial050.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-12T12:23:05.904 INFO:teuthology.orchestra.run.trial050.stderr:done 2026-02-12T12:23:05.906 DEBUG:teuthology.orchestra.run.trial050:> test -d /sys/firmware/efi 2026-02-12T12:23:05.921 DEBUG:teuthology.orchestra.run.trial050:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-12T12:23:05.986 INFO:teuthology.orchestra.run.trial050.stdout:/boot/efi/EFI/BOOT 2026-02-12T12:23:05.986 INFO:teuthology.orchestra.run.trial050.stdout:/boot/efi/EFI/centos 2026-02-12T12:23:05.987 DEBUG:teuthology.orchestra.run.trial050:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-12T12:23:06.053 INFO:teuthology.orchestra.run.trial050.stdout:no 2026-02-12T12:23:06.053 DEBUG:teuthology.orchestra.run.trial050:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-12T12:23:06.118 INFO:teuthology.orchestra.run.trial050.stdout:yes 2026-02-12T12:23:06.119 DEBUG:teuthology.orchestra.run.trial050:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-12T12:23:06.182 DEBUG:teuthology.orchestra.run.trial050:> sudo shutdown -r now 2026-02-12T12:23:06.255 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2026-02-12T12:23:36.255 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-12T12:23:36.256 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial050.front.sepia.ceph.com' 2026-02-12T12:23:36.257 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T12:24:36.307 DEBUG:teuthology.orchestra.remote:timed out 2026-02-12T12:24:45.317 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial050.front.sepia.ceph.com' 2026-02-12T12:24:45.318 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T12:24:45.451 DEBUG:teuthology.orchestra.run.trial050:> true 2026-02-12T12:24:45.611 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial050.front.sepia.ceph.com' 2026-02-12T12:24:45.611 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-677.el9.x86_64"... 2026-02-12T12:24:45.612 DEBUG:teuthology.orchestra.run.trial050:> uname -r 2026-02-12T12:24:45.668 INFO:teuthology.orchestra.run.trial050.stdout:5.14.0-677.el9.x86_64 2026-02-12T12:24:45.669 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-677.el9.x86_64 vs 5.14.0-677.el9.x86_64 2026-02-12T12:24:45.669 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-12T12:24:45.669 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-12T12:24:46.670 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-12T12:24:46.670 DEBUG:teuthology.orchestra.run.trial050:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-12T12:24:46.699 INFO:teuthology.orchestra.run.trial050.stdout:ttyS1 2026-02-12T12:24:46.716 DEBUG:teuthology.parallel:result is None 2026-02-12T12:24:46.717 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-12T12:24:46.730 INFO:teuthology.task.internal:Creating test directory... 2026-02-12T12:24:46.730 DEBUG:teuthology.orchestra.run.trial050:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-12T12:24:46.772 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-12T12:24:46.782 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-12T12:24:46.797 INFO:teuthology.task.internal:Creating archive directory... 2026-02-12T12:24:46.797 DEBUG:teuthology.orchestra.run.trial050:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-12T12:24:46.835 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-12T12:24:46.845 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-12T12:24:46.846 DEBUG:teuthology.orchestra.run.trial050:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-12T12:24:46.895 INFO:teuthology.orchestra.run.trial050.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T12:24:46.904 INFO:teuthology.orchestra.run.trial050.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T12:24:46.906 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-12T12:24:46.918 INFO:teuthology.task.internal:Configuring sudo... 2026-02-12T12:24:46.918 DEBUG:teuthology.orchestra.run.trial050:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-12T12:24:46.971 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-12T12:24:46.987 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-12T12:24:46.988 DEBUG:teuthology.orchestra.run.trial050:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-12T12:24:47.029 DEBUG:teuthology.orchestra.run.trial050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T12:24:47.191 DEBUG:teuthology.orchestra.run.trial050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T12:24:47.216 DEBUG:teuthology.orchestra.run.trial050:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T12:24:47.365 DEBUG:teuthology.orchestra.run.trial050:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T12:24:47.389 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:24:47.389 DEBUG:teuthology.orchestra.run.trial050:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-12T12:24:47.454 DEBUG:teuthology.orchestra.run.trial050:> sudo service rsyslog restart 2026-02-12T12:24:47.522 INFO:teuthology.orchestra.run.trial050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-12T12:24:47.907 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-12T12:24:47.920 INFO:teuthology.task.internal:Starting timer... 2026-02-12T12:24:47.920 INFO:teuthology.run_tasks:Running task pcp... 2026-02-12T12:24:47.942 INFO:teuthology.run_tasks:Running task selinux... 2026-02-12T12:24:47.959 DEBUG:teuthology.orchestra.run.trial050:> sudo service auditd rotate 2026-02-12T12:24:47.989 INFO:teuthology.orchestra.run.trial050.stdout:Rotating logs: 2026-02-12T12:24:47.991 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-12T12:24:47.992 DEBUG:teuthology.orchestra.run.trial050:> /usr/sbin/getenforce 2026-02-12T12:24:48.049 INFO:teuthology.orchestra.run.trial050.stdout:Permissive 2026-02-12T12:24:48.050 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial050.front.sepia.ceph.com': 'permissive'} 2026-02-12T12:24:48.051 DEBUG:teuthology.orchestra.run.trial050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-12T12:24:48.117 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T12:24:48.118 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-12T12:24:48.118 DEBUG:teuthology.orchestra.run.trial050:> sudo /usr/sbin/setenforce permissive 2026-02-12T12:24:48.184 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-12T12:24:48.200 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-12T12:24:48.212 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-12T12:24:48.214 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial050.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-12T12:26:02.623 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial050.front.sepia.ceph.com')] 2026-02-12T12:26:02.624 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial050.front.sepia.ceph.com' 2026-02-12T12:26:02.625 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial050.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T12:26:02.776 DEBUG:teuthology.orchestra.run.trial050:> true 2026-02-12T12:26:02.849 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial050.front.sepia.ceph.com' 2026-02-12T12:26:02.849 INFO:teuthology.run_tasks:Running task clock... 2026-02-12T12:26:02.864 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-12T12:26:02.864 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-12T12:26:02.865 DEBUG:teuthology.orchestra.run.trial050:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-12T12:26:02.916 INFO:teuthology.orchestra.run.trial050.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-12T12:26:02.928 INFO:teuthology.orchestra.run.trial050.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-12T12:26:02.950 INFO:teuthology.orchestra.run.trial050.stderr:sudo: ntpd: command not found 2026-02-12T12:26:02.960 INFO:teuthology.orchestra.run.trial050.stdout:506 Cannot talk to daemon 2026-02-12T12:26:02.972 INFO:teuthology.orchestra.run.trial050.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-12T12:26:02.984 INFO:teuthology.orchestra.run.trial050.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-12T12:26:03.033 INFO:teuthology.orchestra.run.trial050.stderr:bash: line 1: ntpq: command not found 2026-02-12T12:26:03.045 INFO:teuthology.orchestra.run.trial050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-12T12:26:03.045 INFO:teuthology.orchestra.run.trial050.stdout:=============================================================================== 2026-02-12T12:26:03.045 INFO:teuthology.orchestra.run.trial050.stdout:^? time4.lshiy.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-12T12:26:03.045 INFO:teuthology.orchestra.run.trial050.stdout:^? nyc3.us.ntp.li 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-12T12:26:03.045 INFO:teuthology.orchestra.run.trial050.stdout:^? kc1cloud.147.72.155.23.m> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-12T12:26:03.045 INFO:teuthology.orchestra.run.trial050.stdout:^? 172-238-164-57.ip.linode> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-12T12:26:03.046 INFO:teuthology.run_tasks:Running task install... 2026-02-12T12:26:03.060 DEBUG:teuthology.task.install:project ceph 2026-02-12T12:26:03.060 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'}} 2026-02-12T12:26:03.060 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'} 2026-02-12T12:26:03.060 INFO:teuthology.task.install:Using flavor: debug 2026-02-12T12:26:03.069 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-12T12:26:03.069 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-12T12:26:03.070 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': ['python3-pytest'], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0', 'tag': None, 'wait_for_package': False} 2026-02-12T12:26:03.071 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 2026-02-12T12:26:03.227 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-moagrawa-seastore-io-uring-again-crimson-only/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/centos/9/flavors/debug/ 2026-02-12T12:26:03.227 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5145.g5fdb7d8f 2026-02-12T12:26:03.612 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-moagrawa-seastore-io-uring-again-crimson-only/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-moagrawa-seastore-io-uring-again-crimson-only/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-moagrawa-seastore-io-uring-again-crimson-only/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-12T12:26:03.613 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:26:03.613 DEBUG:teuthology.orchestra.run.trial050:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-12T12:26:03.640 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo, python3-pytest on remote rpm x86_64 2026-02-12T12:26:03.640 DEBUG:teuthology.orchestra.run.trial050:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-12T12:26:03.705 DEBUG:teuthology.orchestra.run.trial050:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-12T12:26:03.778 DEBUG:teuthology.orchestra.run.trial050:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-12T12:26:03.841 INFO:teuthology.orchestra.run.trial050.stdout:check_obsoletes = 1 2026-02-12T12:26:03.843 DEBUG:teuthology.orchestra.run.trial050:> sudo yum clean all 2026-02-12T12:26:03.994 INFO:teuthology.orchestra.run.trial050.stdout:41 files removed 2026-02-12T12:26:04.008 DEBUG:teuthology.orchestra.run.trial050:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd ceph-osd-crimson ceph-osd-crimson-debuginfo python3-pytest 2026-02-12T12:26:04.861 INFO:teuthology.orchestra.run.trial050.stdout:ceph packages for x86_64 133 kB/s | 94 kB 00:00 2026-02-12T12:26:05.525 INFO:teuthology.orchestra.run.trial050.stdout:ceph noarch packages 38 kB/s | 24 kB 00:00 2026-02-12T12:26:06.121 INFO:teuthology.orchestra.run.trial050.stdout:ceph source packages 3.6 kB/s | 2.1 kB 00:00 2026-02-12T12:26:14.091 INFO:teuthology.orchestra.run.trial050.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 8.9 MB 00:07 2026-02-12T12:26:17.103 INFO:teuthology.orchestra.run.trial050.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-12T12:26:27.681 INFO:teuthology.orchestra.run.trial050.stdout:CentOS Stream 9 - CRB 967 kB/s | 7.8 MB 00:08 2026-02-12T12:26:28.838 INFO:teuthology.orchestra.run.trial050.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 20 kB 00:00 2026-02-12T12:26:29.918 INFO:teuthology.orchestra.run.trial050.stdout:Extra Packages for Enterprise Linux 20 MB/s | 20 MB 00:01 2026-02-12T12:26:33.165 INFO:teuthology.orchestra.run.trial050.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-12T12:26:34.440 INFO:teuthology.orchestra.run.trial050.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-12T12:26:34.441 INFO:teuthology.orchestra.run.trial050.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-12T12:26:34.469 INFO:teuthology.orchestra.run.trial050.stdout:Dependencies resolved. 2026-02-12T12:26:34.473 INFO:teuthology.orchestra.run.trial050.stdout:======================================================================================= 2026-02-12T12:26:34.474 INFO:teuthology.orchestra.run.trial050.stdout: Package Arch Version Repository Size 2026-02-12T12:26:34.474 INFO:teuthology.orchestra.run.trial050.stdout:======================================================================================= 2026-02-12T12:26:34.474 INFO:teuthology.orchestra.run.trial050.stdout:Installing: 2026-02-12T12:26:34.474 INFO:teuthology.orchestra.run.trial050.stdout: ceph x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 6.5 k 2026-02-12T12:26:34.474 INFO:teuthology.orchestra.run.trial050.stdout: ceph-base x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 5.1 M 2026-02-12T12:26:34.474 INFO:teuthology.orchestra.run.trial050.stdout: ceph-fuse x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 914 k 2026-02-12T12:26:34.474 INFO:teuthology.orchestra.run.trial050.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 149 k 2026-02-12T12:26:34.474 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mgr x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 937 k 2026-02-12T12:26:34.474 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 183 k 2026-02-12T12:26:34.475 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 15 M 2026-02-12T12:26:34.475 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 7.4 M 2026-02-12T12:26:34.475 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mgr-rook noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 50 k 2026-02-12T12:26:34.475 INFO:teuthology.orchestra.run.trial050.stdout: ceph-osd x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 163 k 2026-02-12T12:26:34.475 INFO:teuthology.orchestra.run.trial050.stdout: ceph-osd-classic x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 15 M 2026-02-12T12:26:34.475 INFO:teuthology.orchestra.run.trial050.stdout: ceph-osd-crimson x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 87 M 2026-02-12T12:26:34.475 INFO:teuthology.orchestra.run.trial050.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 429 M 2026-02-12T12:26:34.475 INFO:teuthology.orchestra.run.trial050.stdout: ceph-radosgw x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 11 M 2026-02-12T12:26:34.475 INFO:teuthology.orchestra.run.trial050.stdout: ceph-test x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 126 M 2026-02-12T12:26:34.475 INFO:teuthology.orchestra.run.trial050.stdout: ceph-volume noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 299 k 2026-02-12T12:26:34.475 INFO:teuthology.orchestra.run.trial050.stdout: cephadm noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 1.0 M 2026-02-12T12:26:34.475 INFO:teuthology.orchestra.run.trial050.stdout: libcephfs-devel x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 37 k 2026-02-12T12:26:34.476 INFO:teuthology.orchestra.run.trial050.stdout: libcephfs2 x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 860 k 2026-02-12T12:26:34.476 INFO:teuthology.orchestra.run.trial050.stdout: librados-devel x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 124 k 2026-02-12T12:26:34.476 INFO:teuthology.orchestra.run.trial050.stdout: python3-cephfs x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 203 k 2026-02-12T12:26:34.476 INFO:teuthology.orchestra.run.trial050.stdout: python3-pytest noarch 6.2.2-7.el9 appstream 519 k 2026-02-12T12:26:34.476 INFO:teuthology.orchestra.run.trial050.stdout: python3-rados x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 324 k 2026-02-12T12:26:34.476 INFO:teuthology.orchestra.run.trial050.stdout: python3-rbd x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 305 k 2026-02-12T12:26:34.476 INFO:teuthology.orchestra.run.trial050.stdout: python3-rgw x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 100 k 2026-02-12T12:26:34.476 INFO:teuthology.orchestra.run.trial050.stdout: rbd-fuse x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 87 k 2026-02-12T12:26:34.476 INFO:teuthology.orchestra.run.trial050.stdout: rbd-mirror x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 2.6 M 2026-02-12T12:26:34.476 INFO:teuthology.orchestra.run.trial050.stdout: rbd-nbd x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 167 k 2026-02-12T12:26:34.476 INFO:teuthology.orchestra.run.trial050.stdout:Upgrading: 2026-02-12T12:26:34.476 INFO:teuthology.orchestra.run.trial050.stdout: librados2 x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 3.3 M 2026-02-12T12:26:34.477 INFO:teuthology.orchestra.run.trial050.stdout: librbd1 x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 2.7 M 2026-02-12T12:26:34.477 INFO:teuthology.orchestra.run.trial050.stdout:Installing dependencies: 2026-02-12T12:26:34.477 INFO:teuthology.orchestra.run.trial050.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-12T12:26:34.477 INFO:teuthology.orchestra.run.trial050.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-12T12:26:34.477 INFO:teuthology.orchestra.run.trial050.stdout: ceph-common x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 21 M 2026-02-12T12:26:34.477 INFO:teuthology.orchestra.run.trial050.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 57 k 2026-02-12T12:26:34.477 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mds x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 2.0 M 2026-02-12T12:26:34.477 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 297 k 2026-02-12T12:26:34.477 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mon x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 4.8 M 2026-02-12T12:26:34.477 INFO:teuthology.orchestra.run.trial050.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 17 k 2026-02-12T12:26:34.477 INFO:teuthology.orchestra.run.trial050.stdout: ceph-selinux x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 25 k 2026-02-12T12:26:34.477 INFO:teuthology.orchestra.run.trial050.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-12T12:26:34.478 INFO:teuthology.orchestra.run.trial050.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2026-02-12T12:26:34.478 INFO:teuthology.orchestra.run.trial050.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2026-02-12T12:26:34.478 INFO:teuthology.orchestra.run.trial050.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2026-02-12T12:26:34.478 INFO:teuthology.orchestra.run.trial050.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-12T12:26:34.478 INFO:teuthology.orchestra.run.trial050.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-12T12:26:34.478 INFO:teuthology.orchestra.run.trial050.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-12T12:26:34.478 INFO:teuthology.orchestra.run.trial050.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-12T12:26:34.478 INFO:teuthology.orchestra.run.trial050.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-12T12:26:34.478 INFO:teuthology.orchestra.run.trial050.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-12T12:26:34.478 INFO:teuthology.orchestra.run.trial050.stdout: libasan8 x86_64 15.1.1-2.5.el9 appstream 523 k 2026-02-12T12:26:34.478 INFO:teuthology.orchestra.run.trial050.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-12T12:26:34.478 INFO:teuthology.orchestra.run.trial050.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 24 k 2026-02-12T12:26:34.478 INFO:teuthology.orchestra.run.trial050.stdout: libcephsqlite x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 153 k 2026-02-12T12:26:34.479 INFO:teuthology.orchestra.run.trial050.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-12T12:26:34.479 INFO:teuthology.orchestra.run.trial050.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-12T12:26:34.479 INFO:teuthology.orchestra.run.trial050.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-12T12:26:34.479 INFO:teuthology.orchestra.run.trial050.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-12T12:26:34.479 INFO:teuthology.orchestra.run.trial050.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-12T12:26:34.479 INFO:teuthology.orchestra.run.trial050.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-12T12:26:34.479 INFO:teuthology.orchestra.run.trial050.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-12T12:26:34.479 INFO:teuthology.orchestra.run.trial050.stdout: libradosstriper1 x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 263 k 2026-02-12T12:26:34.479 INFO:teuthology.orchestra.run.trial050.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-12T12:26:34.479 INFO:teuthology.orchestra.run.trial050.stdout: librgw2 x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 5.6 M 2026-02-12T12:26:34.479 INFO:teuthology.orchestra.run.trial050.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-12T12:26:34.479 INFO:teuthology.orchestra.run.trial050.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-12T12:26:34.479 INFO:teuthology.orchestra.run.trial050.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-12T12:26:34.480 INFO:teuthology.orchestra.run.trial050.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-12T12:26:34.480 INFO:teuthology.orchestra.run.trial050.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-12T12:26:34.480 INFO:teuthology.orchestra.run.trial050.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-12T12:26:34.480 INFO:teuthology.orchestra.run.trial050.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-12T12:26:34.480 INFO:teuthology.orchestra.run.trial050.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-12T12:26:34.480 INFO:teuthology.orchestra.run.trial050.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-12T12:26:34.480 INFO:teuthology.orchestra.run.trial050.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-12T12:26:34.480 INFO:teuthology.orchestra.run.trial050.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-12T12:26:34.480 INFO:teuthology.orchestra.run.trial050.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-12T12:26:34.480 INFO:teuthology.orchestra.run.trial050.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-12T12:26:34.480 INFO:teuthology.orchestra.run.trial050.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-12T12:26:34.480 INFO:teuthology.orchestra.run.trial050.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-12T12:26:34.480 INFO:teuthology.orchestra.run.trial050.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-12T12:26:34.481 INFO:teuthology.orchestra.run.trial050.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-12T12:26:34.481 INFO:teuthology.orchestra.run.trial050.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-12T12:26:34.481 INFO:teuthology.orchestra.run.trial050.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-12T12:26:34.481 INFO:teuthology.orchestra.run.trial050.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-12T12:26:34.481 INFO:teuthology.orchestra.run.trial050.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-12T12:26:34.481 INFO:teuthology.orchestra.run.trial050.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-12T12:26:34.481 INFO:teuthology.orchestra.run.trial050.stdout: python3-ceph-argparse x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 45 k 2026-02-12T12:26:34.481 INFO:teuthology.orchestra.run.trial050.stdout: python3-ceph-common x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 184 k 2026-02-12T12:26:34.481 INFO:teuthology.orchestra.run.trial050.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-12T12:26:34.481 INFO:teuthology.orchestra.run.trial050.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-12T12:26:34.481 INFO:teuthology.orchestra.run.trial050.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-12T12:26:34.481 INFO:teuthology.orchestra.run.trial050.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-12T12:26:34.482 INFO:teuthology.orchestra.run.trial050.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-12T12:26:34.482 INFO:teuthology.orchestra.run.trial050.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-12T12:26:34.482 INFO:teuthology.orchestra.run.trial050.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-12T12:26:34.482 INFO:teuthology.orchestra.run.trial050.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-12T12:26:34.482 INFO:teuthology.orchestra.run.trial050.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-12T12:26:34.482 INFO:teuthology.orchestra.run.trial050.stdout: python3-iniconfig noarch 1.1.1-7.el9 appstream 17 k 2026-02-12T12:26:34.482 INFO:teuthology.orchestra.run.trial050.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-12T12:26:34.482 INFO:teuthology.orchestra.run.trial050.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-12T12:26:34.482 INFO:teuthology.orchestra.run.trial050.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-12T12:26:34.482 INFO:teuthology.orchestra.run.trial050.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-12T12:26:34.482 INFO:teuthology.orchestra.run.trial050.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-12T12:26:34.482 INFO:teuthology.orchestra.run.trial050.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-12T12:26:34.482 INFO:teuthology.orchestra.run.trial050.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-12T12:26:34.483 INFO:teuthology.orchestra.run.trial050.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-12T12:26:34.483 INFO:teuthology.orchestra.run.trial050.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-12T12:26:34.483 INFO:teuthology.orchestra.run.trial050.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-12T12:26:34.483 INFO:teuthology.orchestra.run.trial050.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-12T12:26:34.483 INFO:teuthology.orchestra.run.trial050.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-12T12:26:34.483 INFO:teuthology.orchestra.run.trial050.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-12T12:26:34.483 INFO:teuthology.orchestra.run.trial050.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-12T12:26:34.483 INFO:teuthology.orchestra.run.trial050.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-12T12:26:34.483 INFO:teuthology.orchestra.run.trial050.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-12T12:26:34.483 INFO:teuthology.orchestra.run.trial050.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-12T12:26:34.483 INFO:teuthology.orchestra.run.trial050.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-12T12:26:34.483 INFO:teuthology.orchestra.run.trial050.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-12T12:26:34.483 INFO:teuthology.orchestra.run.trial050.stdout: python3-pluggy noarch 0.13.1-7.el9 appstream 41 k 2026-02-12T12:26:34.484 INFO:teuthology.orchestra.run.trial050.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-12T12:26:34.484 INFO:teuthology.orchestra.run.trial050.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-12T12:26:34.484 INFO:teuthology.orchestra.run.trial050.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-12T12:26:34.484 INFO:teuthology.orchestra.run.trial050.stdout: python3-py noarch 1.10.0-6.el9 appstream 477 k 2026-02-12T12:26:34.484 INFO:teuthology.orchestra.run.trial050.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-12T12:26:34.484 INFO:teuthology.orchestra.run.trial050.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-12T12:26:34.484 INFO:teuthology.orchestra.run.trial050.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-12T12:26:34.484 INFO:teuthology.orchestra.run.trial050.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-12T12:26:34.484 INFO:teuthology.orchestra.run.trial050.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-12T12:26:34.484 INFO:teuthology.orchestra.run.trial050.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-12T12:26:34.484 INFO:teuthology.orchestra.run.trial050.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-12T12:26:34.484 INFO:teuthology.orchestra.run.trial050.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-12T12:26:34.484 INFO:teuthology.orchestra.run.trial050.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-12T12:26:34.485 INFO:teuthology.orchestra.run.trial050.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-12T12:26:34.485 INFO:teuthology.orchestra.run.trial050.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-12T12:26:34.485 INFO:teuthology.orchestra.run.trial050.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-12T12:26:34.485 INFO:teuthology.orchestra.run.trial050.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-12T12:26:34.485 INFO:teuthology.orchestra.run.trial050.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-12T12:26:34.485 INFO:teuthology.orchestra.run.trial050.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-12T12:26:34.485 INFO:teuthology.orchestra.run.trial050.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-12T12:26:34.485 INFO:teuthology.orchestra.run.trial050.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-12T12:26:34.485 INFO:teuthology.orchestra.run.trial050.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-12T12:26:34.485 INFO:teuthology.orchestra.run.trial050.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-12T12:26:34.485 INFO:teuthology.orchestra.run.trial050.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-12T12:26:34.485 INFO:teuthology.orchestra.run.trial050.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-12T12:26:34.485 INFO:teuthology.orchestra.run.trial050.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-12T12:26:34.486 INFO:teuthology.orchestra.run.trial050.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-12T12:26:34.486 INFO:teuthology.orchestra.run.trial050.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-12T12:26:34.486 INFO:teuthology.orchestra.run.trial050.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-12T12:26:34.486 INFO:teuthology.orchestra.run.trial050.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-12T12:26:34.486 INFO:teuthology.orchestra.run.trial050.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-12T12:26:34.486 INFO:teuthology.orchestra.run.trial050.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-12T12:26:34.486 INFO:teuthology.orchestra.run.trial050.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-12T12:26:34.486 INFO:teuthology.orchestra.run.trial050.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-12T12:26:34.486 INFO:teuthology.orchestra.run.trial050.stdout:Installing weak dependencies: 2026-02-12T12:26:34.486 INFO:teuthology.orchestra.run.trial050.stdout: ceph-debugsource x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 12 M 2026-02-12T12:26:34.486 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 22 k 2026-02-12T12:26:34.486 INFO:teuthology.orchestra.run.trial050.stdout: libcephfs-daemon x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 35 k 2026-02-12T12:26:34.487 INFO:teuthology.orchestra.run.trial050.stdout: nvme-cli x86_64 2.13-1.el9 baseos 1.1 M 2026-02-12T12:26:34.487 INFO:teuthology.orchestra.run.trial050.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-12T12:26:34.487 INFO:teuthology.orchestra.run.trial050.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-12T12:26:34.487 INFO:teuthology.orchestra.run.trial050.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-12T12:26:34.487 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:26:34.487 INFO:teuthology.orchestra.run.trial050.stdout:Transaction Summary 2026-02-12T12:26:34.487 INFO:teuthology.orchestra.run.trial050.stdout:======================================================================================= 2026-02-12T12:26:34.487 INFO:teuthology.orchestra.run.trial050.stdout:Install 156 Packages 2026-02-12T12:26:34.487 INFO:teuthology.orchestra.run.trial050.stdout:Upgrade 2 Packages 2026-02-12T12:26:34.487 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:26:34.488 INFO:teuthology.orchestra.run.trial050.stdout:Total download size: 827 M 2026-02-12T12:26:34.488 INFO:teuthology.orchestra.run.trial050.stdout:Downloading Packages: 2026-02-12T12:26:35.306 INFO:teuthology.orchestra.run.trial050.stdout:(1/158): ceph-20.3.0-5145.g5fdb7d8f.el9.x86_64. 23 kB/s | 6.5 kB 00:00 2026-02-12T12:26:39.622 INFO:teuthology.orchestra.run.trial050.stdout:(2/158): ceph-debugsource-20.3.0-5145.g5fdb7d8f 2.8 MB/s | 12 MB 00:04 2026-02-12T12:26:40.359 INFO:teuthology.orchestra.run.trial050.stdout:(3/158): ceph-fuse-20.3.0-5145.g5fdb7d8f.el9.x8 1.2 MB/s | 914 kB 00:00 2026-02-12T12:26:40.500 INFO:teuthology.orchestra.run.trial050.stdout:(4/158): ceph-immutable-object-cache-20.3.0-514 1.0 MB/s | 149 kB 00:00 2026-02-12T12:26:40.672 INFO:teuthology.orchestra.run.trial050.stdout:(5/158): ceph-base-20.3.0-5145.g5fdb7d8f.el9.x8 926 kB/s | 5.1 MB 00:05 2026-02-12T12:26:41.980 INFO:teuthology.orchestra.run.trial050.stdout:(6/158): ceph-mds-20.3.0-5145.g5fdb7d8f.el9.x86 1.4 MB/s | 2.0 MB 00:01 2026-02-12T12:26:42.316 INFO:teuthology.orchestra.run.trial050.stdout:(7/158): ceph-mgr-20.3.0-5145.g5fdb7d8f.el9.x86 570 kB/s | 937 kB 00:01 2026-02-12T12:26:42.590 INFO:teuthology.orchestra.run.trial050.stdout:(8/158): ceph-osd-20.3.0-5145.g5fdb7d8f.el9.x86 595 kB/s | 163 kB 00:00 2026-02-12T12:26:45.809 INFO:teuthology.orchestra.run.trial050.stdout:(9/158): ceph-mon-20.3.0-5145.g5fdb7d8f.el9.x86 1.3 MB/s | 4.8 MB 00:03 2026-02-12T12:27:02.273 INFO:teuthology.orchestra.run.trial050.stdout:(10/158): ceph-osd-classic-20.3.0-5145.g5fdb7d8 767 kB/s | 15 MB 00:19 2026-02-12T12:27:04.132 INFO:teuthology.orchestra.run.trial050.stdout:(11/158): ceph-common-20.3.0-5145.g5fdb7d8f.el9 732 kB/s | 21 MB 00:29 2026-02-12T12:27:21.031 INFO:teuthology.orchestra.run.trial050.stdout:(12/158): ceph-radosgw-20.3.0-5145.g5fdb7d8f.el 677 kB/s | 11 MB 00:16 2026-02-12T12:27:21.100 INFO:teuthology.orchestra.run.trial050.stdout:(13/158): ceph-selinux-20.3.0-5145.g5fdb7d8f.el 363 kB/s | 25 kB 00:00 2026-02-12T12:28:15.450 INFO:teuthology.orchestra.run.trial050.stdout:(14/158): ceph-osd-crimson-20.3.0-5145.g5fdb7d8 990 kB/s | 87 MB 01:29 2026-02-12T12:28:15.518 INFO:teuthology.orchestra.run.trial050.stdout:(15/158): libcephfs-daemon-20.3.0-5145.g5fdb7d8 516 kB/s | 35 kB 00:00 2026-02-12T12:28:15.587 INFO:teuthology.orchestra.run.trial050.stdout:(16/158): libcephfs-devel-20.3.0-5145.g5fdb7d8f 543 kB/s | 37 kB 00:00 2026-02-12T12:28:15.654 INFO:teuthology.orchestra.run.trial050.stdout:(17/158): libcephfs-proxy2-20.3.0-5145.g5fdb7d8 362 kB/s | 24 kB 00:00 2026-02-12T12:28:16.563 INFO:teuthology.orchestra.run.trial050.stdout:(18/158): libcephfs2-20.3.0-5145.g5fdb7d8f.el9. 946 kB/s | 860 kB 00:00 2026-02-12T12:28:16.761 INFO:teuthology.orchestra.run.trial050.stdout:(19/158): libcephsqlite-20.3.0-5145.g5fdb7d8f.e 773 kB/s | 153 kB 00:00 2026-02-12T12:28:16.897 INFO:teuthology.orchestra.run.trial050.stdout:(20/158): librados-devel-20.3.0-5145.g5fdb7d8f. 912 kB/s | 124 kB 00:00 2026-02-12T12:28:17.173 INFO:teuthology.orchestra.run.trial050.stdout:(21/158): libradosstriper1-20.3.0-5145.g5fdb7d8 954 kB/s | 263 kB 00:00 2026-02-12T12:28:21.444 INFO:teuthology.orchestra.run.trial050.stdout:(22/158): librgw2-20.3.0-5145.g5fdb7d8f.el9.x86 1.3 MB/s | 5.6 MB 00:04 2026-02-12T12:28:21.515 INFO:teuthology.orchestra.run.trial050.stdout:(23/158): python3-ceph-argparse-20.3.0-5145.g5f 639 kB/s | 45 kB 00:00 2026-02-12T12:28:21.716 INFO:teuthology.orchestra.run.trial050.stdout:(24/158): python3-ceph-common-20.3.0-5145.g5fdb 915 kB/s | 184 kB 00:00 2026-02-12T12:28:21.922 INFO:teuthology.orchestra.run.trial050.stdout:(25/158): python3-cephfs-20.3.0-5145.g5fdb7d8f. 990 kB/s | 203 kB 00:00 2026-02-12T12:28:22.197 INFO:teuthology.orchestra.run.trial050.stdout:(26/158): python3-rados-20.3.0-5145.g5fdb7d8f.e 1.1 MB/s | 324 kB 00:00 2026-02-12T12:28:22.453 INFO:teuthology.orchestra.run.trial050.stdout:(27/158): python3-rbd-20.3.0-5145.g5fdb7d8f.el9 1.2 MB/s | 305 kB 00:00 2026-02-12T12:28:22.517 INFO:teuthology.orchestra.run.trial050.stdout:(28/158): python3-rgw-20.3.0-5145.g5fdb7d8f.el9 1.5 MB/s | 100 kB 00:00 2026-02-12T12:28:22.582 INFO:teuthology.orchestra.run.trial050.stdout:(29/158): rbd-fuse-20.3.0-5145.g5fdb7d8f.el9.x8 1.3 MB/s | 87 kB 00:00 2026-02-12T12:28:24.472 INFO:teuthology.orchestra.run.trial050.stdout:(30/158): rbd-mirror-20.3.0-5145.g5fdb7d8f.el9. 1.4 MB/s | 2.6 MB 00:01 2026-02-12T12:28:24.608 INFO:teuthology.orchestra.run.trial050.stdout:(31/158): rbd-nbd-20.3.0-5145.g5fdb7d8f.el9.x86 1.2 MB/s | 167 kB 00:00 2026-02-12T12:28:24.677 INFO:teuthology.orchestra.run.trial050.stdout:(32/158): ceph-grafana-dashboards-20.3.0-5145.g 831 kB/s | 57 kB 00:00 2026-02-12T12:28:24.881 INFO:teuthology.orchestra.run.trial050.stdout:(33/158): ceph-mgr-cephadm-20.3.0-5145.g5fdb7d8 897 kB/s | 183 kB 00:00 2026-02-12T12:28:34.093 INFO:teuthology.orchestra.run.trial050.stdout:(34/158): ceph-mgr-dashboard-20.3.0-5145.g5fdb7 1.6 MB/s | 15 MB 00:09 2026-02-12T12:28:38.655 INFO:teuthology.orchestra.run.trial050.stdout:(35/158): ceph-mgr-diskprediction-local-20.3.0- 1.6 MB/s | 7.4 MB 00:04 2026-02-12T12:28:38.716 INFO:teuthology.orchestra.run.trial050.stdout:(36/158): ceph-mgr-k8sevents-20.3.0-5145.g5fdb7 363 kB/s | 22 kB 00:00 2026-02-12T12:28:38.907 INFO:teuthology.orchestra.run.trial050.stdout:(37/158): ceph-mgr-modules-core-20.3.0-5145.g5f 1.5 MB/s | 297 kB 00:00 2026-02-12T12:28:38.971 INFO:teuthology.orchestra.run.trial050.stdout:(38/158): ceph-mgr-rook-20.3.0-5145.g5fdb7d8f.e 783 kB/s | 50 kB 00:00 2026-02-12T12:28:39.031 INFO:teuthology.orchestra.run.trial050.stdout:(39/158): ceph-prometheus-alerts-20.3.0-5145.g5 288 kB/s | 17 kB 00:00 2026-02-12T12:28:39.208 INFO:teuthology.orchestra.run.trial050.stdout:(40/158): ceph-volume-20.3.0-5145.g5fdb7d8f.el9 1.7 MB/s | 299 kB 00:00 2026-02-12T12:28:39.821 INFO:teuthology.orchestra.run.trial050.stdout:(41/158): cephadm-20.3.0-5145.g5fdb7d8f.el9.noa 1.7 MB/s | 1.0 MB 00:00 2026-02-12T12:28:39.938 INFO:teuthology.orchestra.run.trial050.stdout:(42/158): cryptsetup-2.8.1-2.el9.x86_64.rpm 2.9 MB/s | 350 kB 00:00 2026-02-12T12:28:40.006 INFO:teuthology.orchestra.run.trial050.stdout:(43/158): hwloc-libs-2.4.1-6.el9.x86_64.rpm 31 MB/s | 2.1 MB 00:00 2026-02-12T12:28:40.028 INFO:teuthology.orchestra.run.trial050.stdout:(44/158): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.8 MB/s | 40 kB 00:00 2026-02-12T12:28:40.050 INFO:teuthology.orchestra.run.trial050.stdout:(45/158): libatomic-11.5.0-14.el9.x86_64.rpm 1.0 MB/s | 23 kB 00:00 2026-02-12T12:28:40.072 INFO:teuthology.orchestra.run.trial050.stdout:(46/158): libconfig-1.7.2-9.el9.x86_64.rpm 3.2 MB/s | 72 kB 00:00 2026-02-12T12:28:40.098 INFO:teuthology.orchestra.run.trial050.stdout:(47/158): libgfortran-11.5.0-14.el9.x86_64.rpm 31 MB/s | 794 kB 00:00 2026-02-12T12:28:40.121 INFO:teuthology.orchestra.run.trial050.stdout:(48/158): libquadmath-11.5.0-14.el9.x86_64.rpm 7.7 MB/s | 184 kB 00:00 2026-02-12T12:28:40.144 INFO:teuthology.orchestra.run.trial050.stdout:(49/158): lksctp-tools-1.0.19-2.el9.x86_64.rpm 4.1 MB/s | 94 kB 00:00 2026-02-12T12:28:40.165 INFO:teuthology.orchestra.run.trial050.stdout:(50/158): mailcap-2.1.49-5.el9.noarch.rpm 1.5 MB/s | 33 kB 00:00 2026-02-12T12:28:40.190 INFO:teuthology.orchestra.run.trial050.stdout:(51/158): nvme-cli-2.13-1.el9.x86_64.rpm 44 MB/s | 1.1 MB 00:00 2026-02-12T12:28:40.212 INFO:teuthology.orchestra.run.trial050.stdout:(52/158): pciutils-3.7.0-7.el9.x86_64.rpm 4.1 MB/s | 93 kB 00:00 2026-02-12T12:28:40.236 INFO:teuthology.orchestra.run.trial050.stdout:(53/158): python3-cffi-1.14.5-5.el9.x86_64.rpm 11 MB/s | 253 kB 00:00 2026-02-12T12:28:40.263 INFO:teuthology.orchestra.run.trial050.stdout:(54/158): python3-cryptography-36.0.1-5.el9.x86 46 MB/s | 1.2 MB 00:00 2026-02-12T12:28:40.286 INFO:teuthology.orchestra.run.trial050.stdout:(55/158): python3-ply-3.11-14.el9.noarch.rpm 4.5 MB/s | 106 kB 00:00 2026-02-12T12:28:40.310 INFO:teuthology.orchestra.run.trial050.stdout:(56/158): python3-pycparser-2.20-6.el9.noarch.r 5.6 MB/s | 135 kB 00:00 2026-02-12T12:28:40.333 INFO:teuthology.orchestra.run.trial050.stdout:(57/158): python3-pyparsing-2.4.7-9.el9.noarch. 6.2 MB/s | 150 kB 00:00 2026-02-12T12:28:40.357 INFO:teuthology.orchestra.run.trial050.stdout:(58/158): python3-requests-2.25.1-10.el9.noarch 5.2 MB/s | 126 kB 00:00 2026-02-12T12:28:40.381 INFO:teuthology.orchestra.run.trial050.stdout:(59/158): python3-urllib3-1.26.5-7.el9.noarch.r 8.9 MB/s | 218 kB 00:00 2026-02-12T12:28:40.406 INFO:teuthology.orchestra.run.trial050.stdout:(60/158): unzip-6.0-59.el9.x86_64.rpm 7.3 MB/s | 182 kB 00:00 2026-02-12T12:28:40.429 INFO:teuthology.orchestra.run.trial050.stdout:(61/158): zip-3.0-35.el9.x86_64.rpm 11 MB/s | 266 kB 00:00 2026-02-12T12:28:40.457 INFO:teuthology.orchestra.run.trial050.stdout:(62/158): boost-program-options-1.75.0-13.el9.x 3.7 MB/s | 104 kB 00:00 2026-02-12T12:28:40.461 INFO:teuthology.orchestra.run.trial050.stdout:(63/158): flexiblas-3.0.4-8.el9.x86_64.rpm 7.5 MB/s | 33 kB 00:00 2026-02-12T12:28:40.485 INFO:teuthology.orchestra.run.trial050.stdout:(64/158): flexiblas-netlib-3.0.4-8.el9.x86_64.r 130 MB/s | 3.0 MB 00:00 2026-02-12T12:28:40.489 INFO:teuthology.orchestra.run.trial050.stdout:(65/158): flexiblas-openblas-openmp-3.0.4-8.el9 4.3 MB/s | 18 kB 00:00 2026-02-12T12:28:40.495 INFO:teuthology.orchestra.run.trial050.stdout:(66/158): libasan8-15.1.1-2.5.el9.x86_64.rpm 79 MB/s | 523 kB 00:00 2026-02-12T12:28:40.501 INFO:teuthology.orchestra.run.trial050.stdout:(67/158): libnbd-1.20.3-4.el9.x86_64.rpm 27 MB/s | 164 kB 00:00 2026-02-12T12:28:40.507 INFO:teuthology.orchestra.run.trial050.stdout:(68/158): libpmemobj-1.12.1-1.el9.x86_64.rpm 27 MB/s | 160 kB 00:00 2026-02-12T12:28:40.512 INFO:teuthology.orchestra.run.trial050.stdout:(69/158): librabbitmq-0.11.0-7.el9.x86_64.rpm 10 MB/s | 45 kB 00:00 2026-02-12T12:28:40.518 INFO:teuthology.orchestra.run.trial050.stdout:(70/158): librdkafka-1.6.1-102.el9.x86_64.rpm 98 MB/s | 662 kB 00:00 2026-02-12T12:28:40.524 INFO:teuthology.orchestra.run.trial050.stdout:(71/158): libstoragemgmt-1.10.1-1.el9.x86_64.rp 39 MB/s | 246 kB 00:00 2026-02-12T12:28:40.530 INFO:teuthology.orchestra.run.trial050.stdout:(72/158): libubsan-11.5.0-14.el9.x86_64.rpm 29 MB/s | 176 kB 00:00 2026-02-12T12:28:40.537 INFO:teuthology.orchestra.run.trial050.stdout:(73/158): libxslt-1.1.34-12.el9.x86_64.rpm 38 MB/s | 233 kB 00:00 2026-02-12T12:28:40.543 INFO:teuthology.orchestra.run.trial050.stdout:(74/158): lttng-ust-2.12.0-6.el9.x86_64.rpm 45 MB/s | 292 kB 00:00 2026-02-12T12:28:40.549 INFO:teuthology.orchestra.run.trial050.stdout:(75/158): lua-5.4.4-4.el9.x86_64.rpm 32 MB/s | 188 kB 00:00 2026-02-12T12:28:40.553 INFO:teuthology.orchestra.run.trial050.stdout:(76/158): openblas-0.3.29-1.el9.x86_64.rpm 9.5 MB/s | 42 kB 00:00 2026-02-12T12:28:40.573 INFO:teuthology.orchestra.run.trial050.stdout:(77/158): openblas-openmp-0.3.29-1.el9.x86_64.r 267 MB/s | 5.3 MB 00:00 2026-02-12T12:28:40.582 INFO:teuthology.orchestra.run.trial050.stdout:(78/158): protobuf-3.14.0-17.el9.x86_64.rpm 122 MB/s | 1.0 MB 00:00 2026-02-12T12:28:40.606 INFO:teuthology.orchestra.run.trial050.stdout:(79/158): python3-babel-2.9.1-2.el9.noarch.rpm 250 MB/s | 6.0 MB 00:00 2026-02-12T12:28:40.613 INFO:teuthology.orchestra.run.trial050.stdout:(80/158): python3-devel-3.9.25-2.el9.x86_64.rpm 34 MB/s | 244 kB 00:00 2026-02-12T12:28:40.630 INFO:teuthology.orchestra.run.trial050.stdout:(81/158): python3-iniconfig-1.1.1-7.el9.noarch. 1.0 MB/s | 17 kB 00:00 2026-02-12T12:28:40.637 INFO:teuthology.orchestra.run.trial050.stdout:(82/158): python3-jinja2-2.11.3-8.el9.noarch.rp 38 MB/s | 249 kB 00:00 2026-02-12T12:28:40.642 INFO:teuthology.orchestra.run.trial050.stdout:(83/158): python3-jmespath-1.0.1-1.el9.noarch.r 9.5 MB/s | 48 kB 00:00 2026-02-12T12:28:40.647 INFO:teuthology.orchestra.run.trial050.stdout:(84/158): python3-libstoragemgmt-1.10.1-1.el9.x 31 MB/s | 177 kB 00:00 2026-02-12T12:28:40.657 INFO:teuthology.orchestra.run.trial050.stdout:(85/158): python3-lxml-4.6.5-3.el9.x86_64.rpm 129 MB/s | 1.2 MB 00:00 2026-02-12T12:28:40.662 INFO:teuthology.orchestra.run.trial050.stdout:(86/158): python3-markupsafe-1.1.1-12.el9.x86_6 6.8 MB/s | 35 kB 00:00 2026-02-12T12:28:40.685 INFO:teuthology.orchestra.run.trial050.stdout:(87/158): python3-numpy-1.23.5-2.el9.x86_64.rpm 271 MB/s | 6.1 MB 00:00 2026-02-12T12:28:40.693 INFO:teuthology.orchestra.run.trial050.stdout:(88/158): python3-numpy-f2py-1.23.5-2.el9.x86_6 56 MB/s | 442 kB 00:00 2026-02-12T12:28:40.698 INFO:teuthology.orchestra.run.trial050.stdout:(89/158): python3-packaging-20.9-5.el9.noarch.r 14 MB/s | 77 kB 00:00 2026-02-12T12:28:40.717 INFO:teuthology.orchestra.run.trial050.stdout:(90/158): python3-pluggy-0.13.1-7.el9.noarch.rp 2.1 MB/s | 41 kB 00:00 2026-02-12T12:28:40.722 INFO:teuthology.orchestra.run.trial050.stdout:(91/158): python3-protobuf-3.14.0-17.el9.noarch 52 MB/s | 267 kB 00:00 2026-02-12T12:28:40.757 INFO:teuthology.orchestra.run.trial050.stdout:(92/158): python3-py-1.10.0-6.el9.noarch.rpm 14 MB/s | 477 kB 00:00 2026-02-12T12:28:40.764 INFO:teuthology.orchestra.run.trial050.stdout:(93/158): python3-pyasn1-0.4.8-6.el9.noarch.rpm 21 MB/s | 159 kB 00:00 2026-02-12T12:28:40.771 INFO:teuthology.orchestra.run.trial050.stdout:(94/158): python3-pyasn1-modules-0.4.8-6.el9.no 39 MB/s | 279 kB 00:00 2026-02-12T12:28:40.807 INFO:teuthology.orchestra.run.trial050.stdout:(95/158): python3-pytest-6.2.2-7.el9.noarch.rpm 14 MB/s | 519 kB 00:00 2026-02-12T12:28:40.812 INFO:teuthology.orchestra.run.trial050.stdout:(96/158): python3-requests-oauthlib-1.3.0-12.el 10 MB/s | 54 kB 00:00 2026-02-12T12:28:40.874 INFO:teuthology.orchestra.run.trial050.stdout:(97/158): python3-scipy-1.9.3-2.el9.x86_64.rpm 311 MB/s | 19 MB 00:00 2026-02-12T12:28:40.880 INFO:teuthology.orchestra.run.trial050.stdout:(98/158): python3-toml-0.10.2-6.el9.noarch.rpm 7.5 MB/s | 42 kB 00:00 2026-02-12T12:28:40.885 INFO:teuthology.orchestra.run.trial050.stdout:(99/158): qatlib-25.08.0-2.el9.x86_64.rpm 42 MB/s | 240 kB 00:00 2026-02-12T12:28:40.890 INFO:teuthology.orchestra.run.trial050.stdout:(100/158): qatlib-service-25.08.0-2.el9.x86_64. 8.4 MB/s | 37 kB 00:00 2026-02-12T12:28:40.894 INFO:teuthology.orchestra.run.trial050.stdout:(101/158): qatzip-libs-1.3.1-1.el9.x86_64.rpm 14 MB/s | 66 kB 00:00 2026-02-12T12:28:40.899 INFO:teuthology.orchestra.run.trial050.stdout:(102/158): socat-1.7.4.1-8.el9.x86_64.rpm 60 MB/s | 303 kB 00:00 2026-02-12T12:28:40.906 INFO:teuthology.orchestra.run.trial050.stdout:(103/158): xmlsec1-1.2.29-13.el9.x86_64.rpm 31 MB/s | 189 kB 00:00 2026-02-12T12:28:40.911 INFO:teuthology.orchestra.run.trial050.stdout:(104/158): xmlsec1-openssl-1.2.29-13.el9.x86_64 17 MB/s | 90 kB 00:00 2026-02-12T12:28:40.916 INFO:teuthology.orchestra.run.trial050.stdout:(105/158): xmlstarlet-1.6.1-20.el9.x86_64.rpm 14 MB/s | 64 kB 00:00 2026-02-12T12:28:41.342 INFO:teuthology.orchestra.run.trial050.stdout:(106/158): lua-devel-5.4.4-4.el9.x86_64.rpm 52 kB/s | 22 kB 00:00 2026-02-12T12:28:42.349 INFO:teuthology.orchestra.run.trial050.stdout:(107/158): protobuf-compiler-3.14.0-17.el9.x86_ 857 kB/s | 862 kB 00:01 2026-02-12T12:28:42.388 INFO:teuthology.orchestra.run.trial050.stdout:(108/158): abseil-cpp-20211102.0-4.el9.x86_64.r 14 MB/s | 551 kB 00:00 2026-02-12T12:28:42.395 INFO:teuthology.orchestra.run.trial050.stdout:(109/158): gperftools-libs-2.9.1-3.el9.x86_64.r 43 MB/s | 308 kB 00:00 2026-02-12T12:28:42.400 INFO:teuthology.orchestra.run.trial050.stdout:(110/158): grpc-data-1.46.7-10.el9.noarch.rpm 4.2 MB/s | 19 kB 00:00 2026-02-12T12:28:42.423 INFO:teuthology.orchestra.run.trial050.stdout:(111/158): libarrow-9.0.0-15.el9.x86_64.rpm 191 MB/s | 4.4 MB 00:00 2026-02-12T12:28:42.428 INFO:teuthology.orchestra.run.trial050.stdout:(112/158): libarrow-doc-9.0.0-15.el9.noarch.rpm 5.0 MB/s | 25 kB 00:00 2026-02-12T12:28:42.433 INFO:teuthology.orchestra.run.trial050.stdout:(113/158): liboath-2.6.12-1.el9.x86_64.rpm 9.8 MB/s | 49 kB 00:00 2026-02-12T12:28:42.487 INFO:teuthology.orchestra.run.trial050.stdout:(114/158): libunwind-1.6.2-1.el9.x86_64.rpm 1.2 MB/s | 67 kB 00:00 2026-02-12T12:28:42.494 INFO:teuthology.orchestra.run.trial050.stdout:(115/158): luarocks-3.9.2-5.el9.noarch.rpm 22 MB/s | 151 kB 00:00 2026-02-12T12:28:42.515 INFO:teuthology.orchestra.run.trial050.stdout:(116/158): parquet-libs-9.0.0-15.el9.x86_64.rpm 39 MB/s | 838 kB 00:00 2026-02-12T12:28:42.524 INFO:teuthology.orchestra.run.trial050.stdout:(117/158): python3-asyncssh-2.13.2-5.el9.noarch 57 MB/s | 548 kB 00:00 2026-02-12T12:28:42.530 INFO:teuthology.orchestra.run.trial050.stdout:(118/158): python3-autocommand-2.2.2-8.el9.noar 5.3 MB/s | 29 kB 00:00 2026-02-12T12:28:42.539 INFO:teuthology.orchestra.run.trial050.stdout:(119/158): python3-backports-tarfile-1.2.0-1.el 6.4 MB/s | 60 kB 00:00 2026-02-12T12:28:42.544 INFO:teuthology.orchestra.run.trial050.stdout:(120/158): python3-bcrypt-3.2.2-1.el9.x86_64.rp 8.2 MB/s | 43 kB 00:00 2026-02-12T12:28:42.550 INFO:teuthology.orchestra.run.trial050.stdout:(121/158): python3-cachetools-4.2.4-1.el9.noarc 6.2 MB/s | 32 kB 00:00 2026-02-12T12:28:42.557 INFO:teuthology.orchestra.run.trial050.stdout:(122/158): python3-certifi-2023.05.07-4.el9.noa 1.9 MB/s | 14 kB 00:00 2026-02-12T12:28:42.564 INFO:teuthology.orchestra.run.trial050.stdout:(123/158): python3-cheroot-10.0.1-4.el9.noarch. 25 MB/s | 173 kB 00:00 2026-02-12T12:28:42.576 INFO:teuthology.orchestra.run.trial050.stdout:(124/158): python3-cherrypy-18.6.1-2.el9.noarch 31 MB/s | 358 kB 00:00 2026-02-12T12:28:42.610 INFO:teuthology.orchestra.run.trial050.stdout:(125/158): python3-google-auth-2.45.0-1.el9.noa 7.3 MB/s | 254 kB 00:00 2026-02-12T12:28:42.720 INFO:teuthology.orchestra.run.trial050.stdout:(126/158): python3-grpcio-1.46.7-10.el9.x86_64. 18 MB/s | 2.0 MB 00:00 2026-02-12T12:28:42.729 INFO:teuthology.orchestra.run.trial050.stdout:(127/158): python3-grpcio-tools-1.46.7-10.el9.x 17 MB/s | 144 kB 00:00 2026-02-12T12:28:42.737 INFO:teuthology.orchestra.run.trial050.stdout:(128/158): python3-influxdb-5.3.1-1.el9.noarch. 17 MB/s | 139 kB 00:00 2026-02-12T12:28:42.743 INFO:teuthology.orchestra.run.trial050.stdout:(129/158): python3-isodate-0.6.1-3.el9.noarch.r 8.8 MB/s | 56 kB 00:00 2026-02-12T12:28:42.748 INFO:teuthology.orchestra.run.trial050.stdout:(130/158): python3-jaraco-8.2.1-3.el9.noarch.rp 2.1 MB/s | 11 kB 00:00 2026-02-12T12:28:42.753 INFO:teuthology.orchestra.run.trial050.stdout:(131/158): python3-jaraco-classes-3.2.1-5.el9.n 3.3 MB/s | 18 kB 00:00 2026-02-12T12:28:42.759 INFO:teuthology.orchestra.run.trial050.stdout:(132/158): python3-jaraco-collections-3.0.0-8.e 4.1 MB/s | 23 kB 00:00 2026-02-12T12:28:42.781 INFO:teuthology.orchestra.run.trial050.stdout:(133/158): python3-jaraco-context-6.0.1-3.el9.n 893 kB/s | 20 kB 00:00 2026-02-12T12:28:42.787 INFO:teuthology.orchestra.run.trial050.stdout:(134/158): python3-jaraco-functools-3.5.0-2.el9 3.5 MB/s | 19 kB 00:00 2026-02-12T12:28:42.792 INFO:teuthology.orchestra.run.trial050.stdout:(135/158): python3-jaraco-text-4.0.0-2.el9.noar 4.8 MB/s | 26 kB 00:00 2026-02-12T12:28:42.802 INFO:teuthology.orchestra.run.trial050.stdout:(136/158): python3-kubernetes-26.1.0-3.el9.noar 107 MB/s | 1.0 MB 00:00 2026-02-12T12:28:42.844 INFO:teuthology.orchestra.run.trial050.stdout:(137/158): python3-more-itertools-8.12.0-2.el9. 1.8 MB/s | 79 kB 00:00 2026-02-12T12:28:42.850 INFO:teuthology.orchestra.run.trial050.stdout:(138/158): python3-msgpack-1.0.3-2.el9.x86_64.r 13 MB/s | 86 kB 00:00 2026-02-12T12:28:42.856 INFO:teuthology.orchestra.run.trial050.stdout:(139/158): python3-natsort-7.1.1-5.el9.noarch.r 9.8 MB/s | 58 kB 00:00 2026-02-12T12:28:42.861 INFO:teuthology.orchestra.run.trial050.stdout:(140/158): python3-portend-3.1.0-2.el9.noarch.r 3.3 MB/s | 16 kB 00:00 2026-02-12T12:28:42.867 INFO:teuthology.orchestra.run.trial050.stdout:(141/158): python3-pyOpenSSL-21.0.0-1.el9.noarc 15 MB/s | 90 kB 00:00 2026-02-12T12:28:42.873 INFO:teuthology.orchestra.run.trial050.stdout:(142/158): python3-repoze-lru-0.7-16.el9.noarch 5.6 MB/s | 31 kB 00:00 2026-02-12T12:28:42.880 INFO:teuthology.orchestra.run.trial050.stdout:(143/158): python3-routes-2.5.1-5.el9.noarch.rp 27 MB/s | 188 kB 00:00 2026-02-12T12:28:42.885 INFO:teuthology.orchestra.run.trial050.stdout:(144/158): python3-rsa-4.9-2.el9.noarch.rpm 10 MB/s | 59 kB 00:00 2026-02-12T12:28:42.892 INFO:teuthology.orchestra.run.trial050.stdout:(145/158): python3-saml-1.16.0-1.el9.noarch.rpm 18 MB/s | 125 kB 00:00 2026-02-12T12:28:42.898 INFO:teuthology.orchestra.run.trial050.stdout:(146/158): python3-tempora-5.0.0-2.el9.noarch.r 6.4 MB/s | 36 kB 00:00 2026-02-12T12:28:42.904 INFO:teuthology.orchestra.run.trial050.stdout:(147/158): python3-typing-extensions-4.15.0-1.e 14 MB/s | 86 kB 00:00 2026-02-12T12:28:42.910 INFO:teuthology.orchestra.run.trial050.stdout:(148/158): python3-websocket-client-1.2.3-2.el9 16 MB/s | 90 kB 00:00 2026-02-12T12:28:42.916 INFO:teuthology.orchestra.run.trial050.stdout:(149/158): python3-xmlsec-1.3.13-1.el9.x86_64.r 8.0 MB/s | 48 kB 00:00 2026-02-12T12:28:42.921 INFO:teuthology.orchestra.run.trial050.stdout:(150/158): python3-xmltodict-0.12.0-15.el9.noar 4.2 MB/s | 22 kB 00:00 2026-02-12T12:28:42.926 INFO:teuthology.orchestra.run.trial050.stdout:(151/158): python3-zc-lockfile-2.0-10.el9.noarc 4.0 MB/s | 20 kB 00:00 2026-02-12T12:28:42.935 INFO:teuthology.orchestra.run.trial050.stdout:(152/158): re2-20211101-20.el9.x86_64.rpm 21 MB/s | 191 kB 00:00 2026-02-12T12:28:42.977 INFO:teuthology.orchestra.run.trial050.stdout:(153/158): thrift-0.15.0-4.el9.x86_64.rpm 38 MB/s | 1.6 MB 00:00 2026-02-12T12:28:43.007 INFO:teuthology.orchestra.run.trial050.stdout:(154/158): yaml-cpp-0.6.3-5.el9.x86_64.rpm 4.0 MB/s | 122 kB 00:00 2026-02-12T12:28:44.980 INFO:teuthology.orchestra.run.trial050.stdout:(155/158): librados2-20.3.0-5145.g5fdb7d8f.el9. 1.7 MB/s | 3.3 MB 00:01 2026-02-12T12:28:46.354 INFO:teuthology.orchestra.run.trial050.stdout:(156/158): librbd1-20.3.0-5145.g5fdb7d8f.el9.x8 2.0 MB/s | 2.7 MB 00:01 2026-02-12T12:29:02.086 INFO:teuthology.orchestra.run.trial050.stdout:(157/158): ceph-test-20.3.0-5145.g5fdb7d8f.el9. 1.2 MB/s | 126 MB 01:40 2026-02-12T12:29:07.802 INFO:teuthology.orchestra.run.trial050.stdout:(158/158): ceph-osd-crimson-debuginfo-20.3.0-51 3.4 MB/s | 429 MB 02:05 2026-02-12T12:29:07.804 INFO:teuthology.orchestra.run.trial050.stdout:-------------------------------------------------------------------------------- 2026-02-12T12:29:07.804 INFO:teuthology.orchestra.run.trial050.stdout:Total 5.4 MB/s | 827 MB 02:33 2026-02-12T12:29:08.358 INFO:teuthology.orchestra.run.trial050.stdout:Running transaction check 2026-02-12T12:29:08.412 INFO:teuthology.orchestra.run.trial050.stdout:Transaction check succeeded. 2026-02-12T12:29:08.412 INFO:teuthology.orchestra.run.trial050.stdout:Running transaction test 2026-02-12T12:29:11.054 INFO:teuthology.orchestra.run.trial050.stdout:Transaction test succeeded. 2026-02-12T12:29:11.054 INFO:teuthology.orchestra.run.trial050.stdout:Running transaction 2026-02-12T12:29:13.760 INFO:teuthology.orchestra.run.trial050.stdout: Preparing : 1/1 2026-02-12T12:29:13.763 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/160 2026-02-12T12:29:13.772 INFO:teuthology.orchestra.run.trial050.stdout: Installing : liboath-2.6.12-1.el9.x86_64 2/160 2026-02-12T12:29:13.783 INFO:teuthology.orchestra.run.trial050.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/160 2026-02-12T12:29:13.929 INFO:teuthology.orchestra.run.trial050.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/160 2026-02-12T12:29:13.931 INFO:teuthology.orchestra.run.trial050.stdout: Upgrading : librados2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 5/160 2026-02-12T12:29:13.978 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: librados2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 5/160 2026-02-12T12:29:13.979 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libcephfs2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 6/160 2026-02-12T12:29:13.991 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: libcephfs2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 6/160 2026-02-12T12:29:13.993 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libcephfs-daemon-2:20.3.0-5145.g5fdb7d8f.el9.x86 7/160 2026-02-12T12:29:13.993 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libcephfs-proxy2-2:20.3.0-5145.g5fdb7d8f.el9.x86 8/160 2026-02-12T12:29:14.020 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5145.g5fdb7d8f.el9.x86 8/160 2026-02-12T12:29:14.025 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-rados-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 9/160 2026-02-12T12:29:14.034 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 10/160 2026-02-12T12:29:14.036 INFO:teuthology.orchestra.run.trial050.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 11/160 2026-02-12T12:29:14.039 INFO:teuthology.orchestra.run.trial050.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 12/160 2026-02-12T12:29:14.041 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 13/160 2026-02-12T12:29:14.045 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 14/160 2026-02-12T12:29:14.161 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 15/160 2026-02-12T12:29:14.162 INFO:teuthology.orchestra.run.trial050.stdout: Upgrading : librbd1-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 16/160 2026-02-12T12:29:14.203 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: librbd1-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 16/160 2026-02-12T12:29:14.209 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 17/160 2026-02-12T12:29:14.217 INFO:teuthology.orchestra.run.trial050.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 18/160 2026-02-12T12:29:14.218 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libcephsqlite-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 19/160 2026-02-12T12:29:14.240 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 19/160 2026-02-12T12:29:14.241 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libradosstriper1-2:20.3.0-5145.g5fdb7d8f.el9.x86 20/160 2026-02-12T12:29:14.253 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5145.g5fdb7d8f.el9.x86 20/160 2026-02-12T12:29:14.257 INFO:teuthology.orchestra.run.trial050.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 21/160 2026-02-12T12:29:14.288 INFO:teuthology.orchestra.run.trial050.stdout: Installing : re2-1:20211101-20.el9.x86_64 22/160 2026-02-12T12:29:14.294 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 23/160 2026-02-12T12:29:14.297 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 24/160 2026-02-12T12:29:14.318 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 25/160 2026-02-12T12:29:14.324 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 26/160 2026-02-12T12:29:14.328 INFO:teuthology.orchestra.run.trial050.stdout: Installing : lua-5.4.4-4.el9.x86_64 27/160 2026-02-12T12:29:14.333 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 28/160 2026-02-12T12:29:14.333 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libasan8-15.1.1-2.5.el9.x86_64 29/160 2026-02-12T12:29:14.344 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: libasan8-15.1.1-2.5.el9.x86_64 29/160 2026-02-12T12:29:14.349 INFO:teuthology.orchestra.run.trial050.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 30/160 2026-02-12T12:29:14.372 INFO:teuthology.orchestra.run.trial050.stdout: Installing : unzip-6.0-59.el9.x86_64 31/160 2026-02-12T12:29:14.385 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 32/160 2026-02-12T12:29:14.391 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 33/160 2026-02-12T12:29:14.396 INFO:teuthology.orchestra.run.trial050.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 34/160 2026-02-12T12:29:14.402 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 35/160 2026-02-12T12:29:14.404 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 36/160 2026-02-12T12:29:14.407 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 37/160 2026-02-12T12:29:14.416 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 38/160 2026-02-12T12:29:14.450 INFO:teuthology.orchestra.run.trial050.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 39/160 2026-02-12T12:29:14.455 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-ceph-common-2:20.3.0-5145.g5fdb7d8f.el9. 40/160 2026-02-12T12:29:14.466 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-ceph-argparse-2:20.3.0-5145.g5fdb7d8f.el 41/160 2026-02-12T12:29:14.477 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-cephfs-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 42/160 2026-02-12T12:29:14.483 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 43/160 2026-02-12T12:29:14.507 INFO:teuthology.orchestra.run.trial050.stdout: Installing : zip-3.0-35.el9.x86_64 44/160 2026-02-12T12:29:14.510 INFO:teuthology.orchestra.run.trial050.stdout: Installing : luarocks-3.9.2-5.el9.noarch 45/160 2026-02-12T12:29:14.560 INFO:teuthology.orchestra.run.trial050.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 46/160 2026-02-12T12:29:14.573 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 47/160 2026-02-12T12:29:14.580 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-rsa-4.9-2.el9.noarch 48/160 2026-02-12T12:29:14.584 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/160 2026-02-12T12:29:14.586 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/160 2026-02-12T12:29:14.590 INFO:teuthology.orchestra.run.trial050.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 51/160 2026-02-12T12:29:14.607 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 52/160 2026-02-12T12:29:14.612 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-rbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 53/160 2026-02-12T12:29:14.615 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 54/160 2026-02-12T12:29:14.623 INFO:teuthology.orchestra.run.trial050.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 55/160 2026-02-12T12:29:14.630 INFO:teuthology.orchestra.run.trial050.stdout: Installing : librados-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 56/160 2026-02-12T12:29:14.641 INFO:teuthology.orchestra.run.trial050.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 57/160 2026-02-12T12:29:14.646 INFO:teuthology.orchestra.run.trial050.stdout: Installing : thrift-0.15.0-4.el9.x86_64 58/160 2026-02-12T12:29:14.649 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 59/160 2026-02-12T12:29:14.664 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 60/160 2026-02-12T12:29:14.668 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 61/160 2026-02-12T12:29:14.673 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 62/160 2026-02-12T12:29:14.684 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 63/160 2026-02-12T12:29:14.693 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 64/160 2026-02-12T12:29:14.700 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 65/160 2026-02-12T12:29:14.719 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 66/160 2026-02-12T12:29:14.730 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 67/160 2026-02-12T12:29:14.745 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 68/160 2026-02-12T12:29:14.750 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 69/160 2026-02-12T12:29:14.758 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 70/160 2026-02-12T12:29:14.798 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 71/160 2026-02-12T12:29:15.121 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 72/160 2026-02-12T12:29:15.131 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 73/160 2026-02-12T12:29:15.135 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 74/160 2026-02-12T12:29:15.140 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 75/160 2026-02-12T12:29:15.146 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 76/160 2026-02-12T12:29:15.150 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 77/160 2026-02-12T12:29:15.153 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 78/160 2026-02-12T12:29:15.159 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 79/160 2026-02-12T12:29:15.161 INFO:teuthology.orchestra.run.trial050.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 80/160 2026-02-12T12:29:15.163 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 81/160 2026-02-12T12:29:15.192 INFO:teuthology.orchestra.run.trial050.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 82/160 2026-02-12T12:29:15.235 INFO:teuthology.orchestra.run.trial050.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 83/160 2026-02-12T12:29:15.244 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 84/160 2026-02-12T12:29:15.290 INFO:teuthology.orchestra.run.trial050.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 85/160 2026-02-12T12:29:15.319 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-py-1.10.0-6.el9.noarch 86/160 2026-02-12T12:29:15.329 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 87/160 2026-02-12T12:29:15.335 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 88/160 2026-02-12T12:29:15.341 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-pluggy-0.13.1-7.el9.noarch 89/160 2026-02-12T12:29:15.348 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 90/160 2026-02-12T12:29:15.352 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 91/160 2026-02-12T12:29:15.386 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-iniconfig-1.1.1-7.el9.noarch 92/160 2026-02-12T12:29:15.609 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 93/160 2026-02-12T12:29:15.632 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 94/160 2026-02-12T12:29:15.636 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 95/160 2026-02-12T12:29:15.695 INFO:teuthology.orchestra.run.trial050.stdout: Installing : openblas-0.3.29-1.el9.x86_64 96/160 2026-02-12T12:29:15.697 INFO:teuthology.orchestra.run.trial050.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 97/160 2026-02-12T12:29:15.718 INFO:teuthology.orchestra.run.trial050.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 98/160 2026-02-12T12:29:16.043 INFO:teuthology.orchestra.run.trial050.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 99/160 2026-02-12T12:29:16.116 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 100/160 2026-02-12T12:29:16.787 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 101/160 2026-02-12T12:29:16.801 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 102/160 2026-02-12T12:29:16.807 INFO:teuthology.orchestra.run.trial050.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 103/160 2026-02-12T12:29:17.057 INFO:teuthology.orchestra.run.trial050.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 104/160 2026-02-12T12:29:17.059 INFO:teuthology.orchestra.run.trial050.stdout: Installing : librgw2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 105/160 2026-02-12T12:29:17.076 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: librgw2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 105/160 2026-02-12T12:29:17.077 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-rgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 106/160 2026-02-12T12:29:18.036 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-common-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 107/160 2026-02-12T12:29:18.038 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-common-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 107/160 2026-02-12T12:29:18.055 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-common-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 107/160 2026-02-12T12:29:18.066 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 108/160 2026-02-12T12:29:18.073 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-packaging-20.9-5.el9.noarch 109/160 2026-02-12T12:29:18.086 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-ply-3.11-14.el9.noarch 110/160 2026-02-12T12:29:18.102 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 111/160 2026-02-12T12:29:18.177 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 112/160 2026-02-12T12:29:18.186 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 113/160 2026-02-12T12:29:18.209 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 114/160 2026-02-12T12:29:18.238 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 115/160 2026-02-12T12:29:18.289 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 116/160 2026-02-12T12:29:18.295 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 117/160 2026-02-12T12:29:18.298 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 118/160 2026-02-12T12:29:18.304 INFO:teuthology.orchestra.run.trial050.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 119/160 2026-02-12T12:29:18.306 INFO:teuthology.orchestra.run.trial050.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 120/160 2026-02-12T12:29:18.307 INFO:teuthology.orchestra.run.trial050.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 121/160 2026-02-12T12:29:18.325 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 121/160 2026-02-12T12:29:18.440 INFO:teuthology.orchestra.run.trial050.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 122/160 2026-02-12T12:29:18.443 INFO:teuthology.orchestra.run.trial050.stdout: Installing : nvme-cli-2.13-1.el9.x86_64 123/160 2026-02-12T12:29:18.715 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: nvme-cli-2.13-1.el9.x86_64 123/160 2026-02-12T12:29:18.715 INFO:teuthology.orchestra.run.trial050.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-12T12:29:18.715 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:29:18.975 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-base-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 124/160 2026-02-12T12:29:19.010 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-base-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 124/160 2026-02-12T12:29:19.010 INFO:teuthology.orchestra.run.trial050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-12T12:29:19.010 INFO:teuthology.orchestra.run.trial050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-12T12:29:19.011 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:29:19.013 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-selinux-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 125/160 2026-02-12T12:29:23.716 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 125/160 2026-02-12T12:29:23.716 INFO:teuthology.orchestra.run.trial050.stdout:skipping the directory /sys 2026-02-12T12:29:23.716 INFO:teuthology.orchestra.run.trial050.stdout:skipping the directory /proc 2026-02-12T12:29:23.716 INFO:teuthology.orchestra.run.trial050.stdout:skipping the directory /mnt 2026-02-12T12:29:23.717 INFO:teuthology.orchestra.run.trial050.stdout:skipping the directory /var/tmp 2026-02-12T12:29:23.717 INFO:teuthology.orchestra.run.trial050.stdout:skipping the directory /home 2026-02-12T12:29:23.717 INFO:teuthology.orchestra.run.trial050.stdout:skipping the directory /root 2026-02-12T12:29:23.717 INFO:teuthology.orchestra.run.trial050.stdout:skipping the directory /tmp 2026-02-12T12:29:23.717 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:29:23.806 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-mds-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 126/160 2026-02-12T12:29:23.828 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-mds-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 126/160 2026-02-12T12:29:23.828 INFO:teuthology.orchestra.run.trial050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T12:29:23.828 INFO:teuthology.orchestra.run.trial050.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-12T12:29:23.828 INFO:teuthology.orchestra.run.trial050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-12T12:29:23.828 INFO:teuthology.orchestra.run.trial050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-12T12:29:23.828 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:29:24.040 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-mon-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 127/160 2026-02-12T12:29:24.060 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-mon-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 127/160 2026-02-12T12:29:24.060 INFO:teuthology.orchestra.run.trial050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T12:29:24.060 INFO:teuthology.orchestra.run.trial050.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-12T12:29:24.060 INFO:teuthology.orchestra.run.trial050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-12T12:29:24.060 INFO:teuthology.orchestra.run.trial050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-12T12:29:24.060 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:29:24.066 INFO:teuthology.orchestra.run.trial050.stdout: Installing : mailcap-2.1.49-5.el9.noarch 128/160 2026-02-12T12:29:24.067 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 129/160 2026-02-12T12:29:24.084 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 130/160 2026-02-12T12:29:24.084 INFO:teuthology.orchestra.run.trial050.stdout:Creating group 'qat' with GID 992. 2026-02-12T12:29:24.084 INFO:teuthology.orchestra.run.trial050.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-12T12:29:24.084 INFO:teuthology.orchestra.run.trial050.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-12T12:29:24.084 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:29:24.091 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 130/160 2026-02-12T12:29:24.113 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 130/160 2026-02-12T12:29:24.114 INFO:teuthology.orchestra.run.trial050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-12T12:29:24.114 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:29:24.148 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 131/160 2026-02-12T12:29:24.746 INFO:teuthology.orchestra.run.trial050.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 132/160 2026-02-12T12:29:30.157 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-osd-classic-2:20.3.0-5145.g5fdb7d8f.el9.x86 133/160 2026-02-12T12:29:30.219 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-osd-crimson-2:20.3.0-5145.g5fdb7d8f.el9.x86 134/160 2026-02-12T12:29:30.222 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-volume-2:20.3.0-5145.g5fdb7d8f.el9.noarch 135/160 2026-02-12T12:29:30.235 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-volume-2:20.3.0-5145.g5fdb7d8f.el9.noarch 135/160 2026-02-12T12:29:30.235 INFO:teuthology.orchestra.run.trial050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T12:29:30.235 INFO:teuthology.orchestra.run.trial050.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-12T12:29:30.235 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:29:30.247 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-osd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 136/160 2026-02-12T12:29:30.269 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-osd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 136/160 2026-02-12T12:29:30.270 INFO:teuthology.orchestra.run.trial050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T12:29:30.270 INFO:teuthology.orchestra.run.trial050.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-12T12:29:30.270 INFO:teuthology.orchestra.run.trial050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-12T12:29:30.270 INFO:teuthology.orchestra.run.trial050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-12T12:29:30.270 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:29:30.330 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarch 137/160 2026-02-12T12:29:30.332 INFO:teuthology.orchestra.run.trial050.stdout: Installing : cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarch 137/160 2026-02-12T12:29:30.341 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5145.g5fdb7d8f.e 138/160 2026-02-12T12:29:30.364 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5145.g5fdb7d8f. 139/160 2026-02-12T12:29:30.366 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noa 140/160 2026-02-12T12:29:31.937 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noa 140/160 2026-02-12T12:29:31.944 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5145.g5fdb7d8f.el9.n 141/160 2026-02-12T12:29:32.400 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5145.g5fdb7d8f.el9.n 141/160 2026-02-12T12:29:32.401 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5145.g5fd 142/160 2026-02-12T12:29:32.413 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5145.g5fd 142/160 2026-02-12T12:29:32.414 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5145.g5fdb7d8f.el9.n 143/160 2026-02-12T12:29:32.473 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5145.g5fdb7d8f.el9.n 143/160 2026-02-12T12:29:32.513 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5145.g5fdb7d8f.el 144/160 2026-02-12T12:29:32.514 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-mgr-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 145/160 2026-02-12T12:29:32.534 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 145/160 2026-02-12T12:29:32.534 INFO:teuthology.orchestra.run.trial050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T12:29:32.534 INFO:teuthology.orchestra.run.trial050.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-12T12:29:32.534 INFO:teuthology.orchestra.run.trial050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-12T12:29:32.535 INFO:teuthology.orchestra.run.trial050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-12T12:29:32.535 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:29:32.546 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-mgr-rook-2:20.3.0-5145.g5fdb7d8f.el9.noarch 146/160 2026-02-12T12:29:34.763 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5145.g5fdb7d8f.el9.noarch 146/160 2026-02-12T12:29:57.222 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-debugsource-2:20.3.0-5145.g5fdb7d8f.el9.x86 147/160 2026-02-12T12:29:57.223 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5145.g5fdb7d 148/160 2026-02-12T12:29:57.712 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 149/160 2026-02-12T12:29:57.714 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-radosgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 150/160 2026-02-12T12:29:57.736 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 150/160 2026-02-12T12:29:57.736 INFO:teuthology.orchestra.run.trial050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T12:29:57.736 INFO:teuthology.orchestra.run.trial050.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-12T12:29:57.736 INFO:teuthology.orchestra.run.trial050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-12T12:29:57.736 INFO:teuthology.orchestra.run.trial050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-12T12:29:57.736 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:29:57.745 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5145.g5fdb7 151/160 2026-02-12T12:29:57.764 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5145.g5fdb7 151/160 2026-02-12T12:29:57.764 INFO:teuthology.orchestra.run.trial050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T12:29:57.765 INFO:teuthology.orchestra.run.trial050.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-12T12:29:57.765 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:29:57.875 INFO:teuthology.orchestra.run.trial050.stdout: Installing : rbd-mirror-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 152/160 2026-02-12T12:29:57.896 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 152/160 2026-02-12T12:29:57.896 INFO:teuthology.orchestra.run.trial050.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T12:29:57.896 INFO:teuthology.orchestra.run.trial050.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-12T12:29:57.896 INFO:teuthology.orchestra.run.trial050.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-12T12:29:57.896 INFO:teuthology.orchestra.run.trial050.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-12T12:29:57.896 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:04.926 INFO:teuthology.orchestra.run.trial050.stdout: Installing : python3-pytest-6.2.2-7.el9.noarch 153/160 2026-02-12T12:30:04.933 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-test-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 154/160 2026-02-12T12:30:04.938 INFO:teuthology.orchestra.run.trial050.stdout: Installing : libcephfs-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_ 155/160 2026-02-12T12:30:04.947 INFO:teuthology.orchestra.run.trial050.stdout: Installing : rbd-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 156/160 2026-02-12T12:30:04.988 INFO:teuthology.orchestra.run.trial050.stdout: Installing : rbd-nbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 157/160 2026-02-12T12:30:04.991 INFO:teuthology.orchestra.run.trial050.stdout: Installing : ceph-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 158/160 2026-02-12T12:30:04.991 INFO:teuthology.orchestra.run.trial050.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 159/160 2026-02-12T12:30:05.003 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 159/160 2026-02-12T12:30:05.003 INFO:teuthology.orchestra.run.trial050.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 160/160 2026-02-12T12:30:05.014 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2026-02-12T12:30:05.020 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5145.g5fdb7d8f.el9.x86 160/160 2026-02-12T12:30:05.044 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5145.g5fdb7d8f.el9.x86 160/160 2026-02-12T12:30:08.244 INFO:teuthology.orchestra.run.trial050.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 160/160 2026-02-12T12:30:08.245 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/160 2026-02-12T12:30:08.245 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-base-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2/160 2026-02-12T12:30:08.245 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-common-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 3/160 2026-02-12T12:30:08.245 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-debugsource-2:20.3.0-5145.g5fdb7d8f.el9.x86 4/160 2026-02-12T12:30:08.245 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 5/160 2026-02-12T12:30:08.245 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5145.g5fdb7 6/160 2026-02-12T12:30:08.245 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-mds-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 7/160 2026-02-12T12:30:08.245 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-mgr-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 8/160 2026-02-12T12:30:08.245 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-mon-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 9/160 2026-02-12T12:30:08.245 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-osd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 10/160 2026-02-12T12:30:08.246 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-osd-classic-2:20.3.0-5145.g5fdb7d8f.el9.x86 11/160 2026-02-12T12:30:08.246 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5145.g5fdb7d8f.el9.x86 12/160 2026-02-12T12:30:08.246 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5145.g5fdb7d 13/160 2026-02-12T12:30:08.246 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-radosgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 14/160 2026-02-12T12:30:08.246 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-selinux-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 15/160 2026-02-12T12:30:08.246 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-test-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 16/160 2026-02-12T12:30:08.246 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libcephfs-daemon-2:20.3.0-5145.g5fdb7d8f.el9.x86 17/160 2026-02-12T12:30:08.246 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libcephfs-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_ 18/160 2026-02-12T12:30:08.246 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5145.g5fdb7d8f.el9.x86 19/160 2026-02-12T12:30:08.246 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libcephfs2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 20/160 2026-02-12T12:30:08.246 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libcephsqlite-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 21/160 2026-02-12T12:30:08.246 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : librados-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 22/160 2026-02-12T12:30:08.247 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libradosstriper1-2:20.3.0-5145.g5fdb7d8f.el9.x86 23/160 2026-02-12T12:30:08.247 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : librgw2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 24/160 2026-02-12T12:30:08.247 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5145.g5fdb7d8f.el 25/160 2026-02-12T12:30:08.247 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-ceph-common-2:20.3.0-5145.g5fdb7d8f.el9. 26/160 2026-02-12T12:30:08.247 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-cephfs-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 27/160 2026-02-12T12:30:08.247 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-rados-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 28/160 2026-02-12T12:30:08.247 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-rbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 29/160 2026-02-12T12:30:08.247 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-rgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 30/160 2026-02-12T12:30:08.247 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : rbd-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 31/160 2026-02-12T12:30:08.247 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : rbd-mirror-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 32/160 2026-02-12T12:30:08.247 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : rbd-nbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 33/160 2026-02-12T12:30:08.247 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5145.g5fdb7d8f. 34/160 2026-02-12T12:30:08.247 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noa 35/160 2026-02-12T12:30:08.248 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5145.g5fdb7d8f.el9.n 36/160 2026-02-12T12:30:08.248 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5145.g5fd 37/160 2026-02-12T12:30:08.248 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5145.g5fdb7d8f.el9.n 38/160 2026-02-12T12:30:08.248 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5145.g5fdb7d8f.el 39/160 2026-02-12T12:30:08.248 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5145.g5fdb7d8f.el9.noarch 40/160 2026-02-12T12:30:08.248 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5145.g5fdb7d8f.e 41/160 2026-02-12T12:30:08.248 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ceph-volume-2:20.3.0-5145.g5fdb7d8f.el9.noarch 42/160 2026-02-12T12:30:08.248 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarch 43/160 2026-02-12T12:30:08.248 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 44/160 2026-02-12T12:30:08.248 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 45/160 2026-02-12T12:30:08.248 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 46/160 2026-02-12T12:30:08.248 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 47/160 2026-02-12T12:30:08.249 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 48/160 2026-02-12T12:30:08.249 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 49/160 2026-02-12T12:30:08.249 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 50/160 2026-02-12T12:30:08.249 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 51/160 2026-02-12T12:30:08.249 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 52/160 2026-02-12T12:30:08.249 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : nvme-cli-2.13-1.el9.x86_64 53/160 2026-02-12T12:30:08.249 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 54/160 2026-02-12T12:30:08.249 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 55/160 2026-02-12T12:30:08.249 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 56/160 2026-02-12T12:30:08.249 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-ply-3.11-14.el9.noarch 57/160 2026-02-12T12:30:08.249 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 58/160 2026-02-12T12:30:08.249 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 59/160 2026-02-12T12:30:08.249 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 60/160 2026-02-12T12:30:08.250 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 61/160 2026-02-12T12:30:08.250 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : unzip-6.0-59.el9.x86_64 62/160 2026-02-12T12:30:08.250 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : zip-3.0-35.el9.x86_64 63/160 2026-02-12T12:30:08.250 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 64/160 2026-02-12T12:30:08.250 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 65/160 2026-02-12T12:30:08.250 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 66/160 2026-02-12T12:30:08.250 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 67/160 2026-02-12T12:30:08.250 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libasan8-15.1.1-2.5.el9.x86_64 68/160 2026-02-12T12:30:08.250 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 69/160 2026-02-12T12:30:08.250 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 70/160 2026-02-12T12:30:08.250 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 71/160 2026-02-12T12:30:08.250 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 72/160 2026-02-12T12:30:08.250 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 73/160 2026-02-12T12:30:08.251 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 74/160 2026-02-12T12:30:08.251 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 75/160 2026-02-12T12:30:08.251 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 76/160 2026-02-12T12:30:08.251 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : lua-5.4.4-4.el9.x86_64 77/160 2026-02-12T12:30:08.251 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 78/160 2026-02-12T12:30:08.251 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 79/160 2026-02-12T12:30:08.251 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 80/160 2026-02-12T12:30:08.251 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 81/160 2026-02-12T12:30:08.251 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 82/160 2026-02-12T12:30:08.251 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-iniconfig-1.1.1-7.el9.noarch 83/160 2026-02-12T12:30:08.251 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 84/160 2026-02-12T12:30:08.251 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 85/160 2026-02-12T12:30:08.251 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 86/160 2026-02-12T12:30:08.252 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 87/160 2026-02-12T12:30:08.252 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 88/160 2026-02-12T12:30:08.252 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 89/160 2026-02-12T12:30:08.252 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 90/160 2026-02-12T12:30:08.252 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 91/160 2026-02-12T12:30:08.252 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-pluggy-0.13.1-7.el9.noarch 92/160 2026-02-12T12:30:08.252 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 93/160 2026-02-12T12:30:08.252 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-py-1.10.0-6.el9.noarch 94/160 2026-02-12T12:30:08.252 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 95/160 2026-02-12T12:30:08.252 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 96/160 2026-02-12T12:30:08.252 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-pytest-6.2.2-7.el9.noarch 97/160 2026-02-12T12:30:08.252 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 98/160 2026-02-12T12:30:08.252 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 99/160 2026-02-12T12:30:08.253 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 100/160 2026-02-12T12:30:08.253 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 101/160 2026-02-12T12:30:08.253 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 102/160 2026-02-12T12:30:08.253 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 103/160 2026-02-12T12:30:08.253 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 104/160 2026-02-12T12:30:08.253 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 105/160 2026-02-12T12:30:08.253 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 106/160 2026-02-12T12:30:08.253 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 107/160 2026-02-12T12:30:08.253 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 108/160 2026-02-12T12:30:08.253 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 109/160 2026-02-12T12:30:08.253 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 110/160 2026-02-12T12:30:08.253 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 111/160 2026-02-12T12:30:08.254 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 112/160 2026-02-12T12:30:08.254 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 113/160 2026-02-12T12:30:08.254 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 114/160 2026-02-12T12:30:08.254 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 115/160 2026-02-12T12:30:08.254 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 116/160 2026-02-12T12:30:08.254 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 117/160 2026-02-12T12:30:08.254 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 118/160 2026-02-12T12:30:08.254 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 119/160 2026-02-12T12:30:08.254 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 120/160 2026-02-12T12:30:08.254 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 121/160 2026-02-12T12:30:08.254 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 122/160 2026-02-12T12:30:08.254 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 123/160 2026-02-12T12:30:08.255 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 124/160 2026-02-12T12:30:08.255 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 125/160 2026-02-12T12:30:08.255 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 126/160 2026-02-12T12:30:08.255 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 127/160 2026-02-12T12:30:08.255 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 128/160 2026-02-12T12:30:08.255 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 129/160 2026-02-12T12:30:08.255 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 130/160 2026-02-12T12:30:08.255 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 131/160 2026-02-12T12:30:08.255 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 132/160 2026-02-12T12:30:08.255 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 133/160 2026-02-12T12:30:08.255 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 134/160 2026-02-12T12:30:08.255 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 135/160 2026-02-12T12:30:08.255 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 136/160 2026-02-12T12:30:08.256 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 137/160 2026-02-12T12:30:08.256 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 138/160 2026-02-12T12:30:08.256 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 139/160 2026-02-12T12:30:08.256 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 140/160 2026-02-12T12:30:08.256 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 141/160 2026-02-12T12:30:08.256 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 142/160 2026-02-12T12:30:08.256 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 143/160 2026-02-12T12:30:08.256 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 144/160 2026-02-12T12:30:08.256 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 145/160 2026-02-12T12:30:08.256 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 146/160 2026-02-12T12:30:08.256 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 147/160 2026-02-12T12:30:08.256 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 148/160 2026-02-12T12:30:08.256 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 149/160 2026-02-12T12:30:08.257 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 150/160 2026-02-12T12:30:08.257 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 151/160 2026-02-12T12:30:08.257 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 152/160 2026-02-12T12:30:08.257 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 153/160 2026-02-12T12:30:08.257 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : re2-1:20211101-20.el9.x86_64 154/160 2026-02-12T12:30:08.257 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 155/160 2026-02-12T12:30:08.257 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 156/160 2026-02-12T12:30:08.257 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : librados2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 157/160 2026-02-12T12:30:08.257 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 158/160 2026-02-12T12:30:08.257 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : librbd1-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 159/160 2026-02-12T12:30:08.312 INFO:teuthology.orchestra.run.trial050.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 160/160 2026-02-12T12:30:08.313 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:08.313 INFO:teuthology.orchestra.run.trial050.stdout:Upgraded: 2026-02-12T12:30:08.313 INFO:teuthology.orchestra.run.trial050.stdout: librados2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.313 INFO:teuthology.orchestra.run.trial050.stdout: librbd1-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.313 INFO:teuthology.orchestra.run.trial050.stdout:Installed: 2026-02-12T12:30:08.313 INFO:teuthology.orchestra.run.trial050.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-12T12:30:08.313 INFO:teuthology.orchestra.run.trial050.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-12T12:30:08.313 INFO:teuthology.orchestra.run.trial050.stdout: ceph-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.313 INFO:teuthology.orchestra.run.trial050.stdout: ceph-base-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.313 INFO:teuthology.orchestra.run.trial050.stdout: ceph-common-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.314 INFO:teuthology.orchestra.run.trial050.stdout: ceph-debugsource-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.314 INFO:teuthology.orchestra.run.trial050.stdout: ceph-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.314 INFO:teuthology.orchestra.run.trial050.stdout: ceph-grafana-dashboards-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T12:30:08.314 INFO:teuthology.orchestra.run.trial050.stdout: ceph-immutable-object-cache-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.314 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mds-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.314 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mgr-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.314 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mgr-cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T12:30:08.314 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mgr-dashboard-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T12:30:08.314 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T12:30:08.314 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mgr-k8sevents-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T12:30:08.314 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mgr-modules-core-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T12:30:08.314 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mgr-rook-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T12:30:08.314 INFO:teuthology.orchestra.run.trial050.stdout: ceph-mon-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.315 INFO:teuthology.orchestra.run.trial050.stdout: ceph-osd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.315 INFO:teuthology.orchestra.run.trial050.stdout: ceph-osd-classic-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.315 INFO:teuthology.orchestra.run.trial050.stdout: ceph-osd-crimson-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.315 INFO:teuthology.orchestra.run.trial050.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.315 INFO:teuthology.orchestra.run.trial050.stdout: ceph-prometheus-alerts-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T12:30:08.315 INFO:teuthology.orchestra.run.trial050.stdout: ceph-radosgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.315 INFO:teuthology.orchestra.run.trial050.stdout: ceph-selinux-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.315 INFO:teuthology.orchestra.run.trial050.stdout: ceph-test-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.315 INFO:teuthology.orchestra.run.trial050.stdout: ceph-volume-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T12:30:08.315 INFO:teuthology.orchestra.run.trial050.stdout: cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T12:30:08.315 INFO:teuthology.orchestra.run.trial050.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-12T12:30:08.315 INFO:teuthology.orchestra.run.trial050.stdout: flexiblas-3.0.4-8.el9.x86_64 2026-02-12T12:30:08.315 INFO:teuthology.orchestra.run.trial050.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2026-02-12T12:30:08.316 INFO:teuthology.orchestra.run.trial050.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2026-02-12T12:30:08.316 INFO:teuthology.orchestra.run.trial050.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-12T12:30:08.316 INFO:teuthology.orchestra.run.trial050.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-12T12:30:08.316 INFO:teuthology.orchestra.run.trial050.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-12T12:30:08.316 INFO:teuthology.orchestra.run.trial050.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-12T12:30:08.316 INFO:teuthology.orchestra.run.trial050.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-12T12:30:08.316 INFO:teuthology.orchestra.run.trial050.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-12T12:30:08.316 INFO:teuthology.orchestra.run.trial050.stdout: libasan8-15.1.1-2.5.el9.x86_64 2026-02-12T12:30:08.316 INFO:teuthology.orchestra.run.trial050.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-12T12:30:08.316 INFO:teuthology.orchestra.run.trial050.stdout: libcephfs-daemon-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.316 INFO:teuthology.orchestra.run.trial050.stdout: libcephfs-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.316 INFO:teuthology.orchestra.run.trial050.stdout: libcephfs-proxy2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.317 INFO:teuthology.orchestra.run.trial050.stdout: libcephfs2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.317 INFO:teuthology.orchestra.run.trial050.stdout: libcephsqlite-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.317 INFO:teuthology.orchestra.run.trial050.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-12T12:30:08.317 INFO:teuthology.orchestra.run.trial050.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-12T12:30:08.317 INFO:teuthology.orchestra.run.trial050.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-12T12:30:08.317 INFO:teuthology.orchestra.run.trial050.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-12T12:30:08.317 INFO:teuthology.orchestra.run.trial050.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-12T12:30:08.317 INFO:teuthology.orchestra.run.trial050.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-12T12:30:08.317 INFO:teuthology.orchestra.run.trial050.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-12T12:30:08.317 INFO:teuthology.orchestra.run.trial050.stdout: librados-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.317 INFO:teuthology.orchestra.run.trial050.stdout: libradosstriper1-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.317 INFO:teuthology.orchestra.run.trial050.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-12T12:30:08.318 INFO:teuthology.orchestra.run.trial050.stdout: librgw2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.318 INFO:teuthology.orchestra.run.trial050.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-12T12:30:08.318 INFO:teuthology.orchestra.run.trial050.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-12T12:30:08.318 INFO:teuthology.orchestra.run.trial050.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-12T12:30:08.318 INFO:teuthology.orchestra.run.trial050.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-12T12:30:08.318 INFO:teuthology.orchestra.run.trial050.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-12T12:30:08.318 INFO:teuthology.orchestra.run.trial050.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-12T12:30:08.318 INFO:teuthology.orchestra.run.trial050.stdout: lua-5.4.4-4.el9.x86_64 2026-02-12T12:30:08.318 INFO:teuthology.orchestra.run.trial050.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-12T12:30:08.318 INFO:teuthology.orchestra.run.trial050.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-12T12:30:08.318 INFO:teuthology.orchestra.run.trial050.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-12T12:30:08.318 INFO:teuthology.orchestra.run.trial050.stdout: nvme-cli-2.13-1.el9.x86_64 2026-02-12T12:30:08.318 INFO:teuthology.orchestra.run.trial050.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-12T12:30:08.319 INFO:teuthology.orchestra.run.trial050.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-12T12:30:08.319 INFO:teuthology.orchestra.run.trial050.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-12T12:30:08.319 INFO:teuthology.orchestra.run.trial050.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-12T12:30:08.319 INFO:teuthology.orchestra.run.trial050.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-12T12:30:08.319 INFO:teuthology.orchestra.run.trial050.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-12T12:30:08.319 INFO:teuthology.orchestra.run.trial050.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-12T12:30:08.319 INFO:teuthology.orchestra.run.trial050.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-12T12:30:08.319 INFO:teuthology.orchestra.run.trial050.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-12T12:30:08.319 INFO:teuthology.orchestra.run.trial050.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-12T12:30:08.319 INFO:teuthology.orchestra.run.trial050.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-12T12:30:08.319 INFO:teuthology.orchestra.run.trial050.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-12T12:30:08.319 INFO:teuthology.orchestra.run.trial050.stdout: python3-ceph-argparse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.319 INFO:teuthology.orchestra.run.trial050.stdout: python3-ceph-common-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.320 INFO:teuthology.orchestra.run.trial050.stdout: python3-cephfs-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.320 INFO:teuthology.orchestra.run.trial050.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-12T12:30:08.320 INFO:teuthology.orchestra.run.trial050.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-12T12:30:08.320 INFO:teuthology.orchestra.run.trial050.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-12T12:30:08.320 INFO:teuthology.orchestra.run.trial050.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-12T12:30:08.320 INFO:teuthology.orchestra.run.trial050.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-12T12:30:08.320 INFO:teuthology.orchestra.run.trial050.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-12T12:30:08.320 INFO:teuthology.orchestra.run.trial050.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-12T12:30:08.320 INFO:teuthology.orchestra.run.trial050.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-12T12:30:08.320 INFO:teuthology.orchestra.run.trial050.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-12T12:30:08.320 INFO:teuthology.orchestra.run.trial050.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-12T12:30:08.320 INFO:teuthology.orchestra.run.trial050.stdout: python3-iniconfig-1.1.1-7.el9.noarch 2026-02-12T12:30:08.320 INFO:teuthology.orchestra.run.trial050.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-12T12:30:08.321 INFO:teuthology.orchestra.run.trial050.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-12T12:30:08.321 INFO:teuthology.orchestra.run.trial050.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-12T12:30:08.321 INFO:teuthology.orchestra.run.trial050.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-12T12:30:08.321 INFO:teuthology.orchestra.run.trial050.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-12T12:30:08.321 INFO:teuthology.orchestra.run.trial050.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-12T12:30:08.321 INFO:teuthology.orchestra.run.trial050.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-12T12:30:08.321 INFO:teuthology.orchestra.run.trial050.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-12T12:30:08.321 INFO:teuthology.orchestra.run.trial050.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-12T12:30:08.321 INFO:teuthology.orchestra.run.trial050.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-12T12:30:08.321 INFO:teuthology.orchestra.run.trial050.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-12T12:30:08.321 INFO:teuthology.orchestra.run.trial050.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-12T12:30:08.321 INFO:teuthology.orchestra.run.trial050.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-12T12:30:08.322 INFO:teuthology.orchestra.run.trial050.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-12T12:30:08.322 INFO:teuthology.orchestra.run.trial050.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-12T12:30:08.322 INFO:teuthology.orchestra.run.trial050.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-12T12:30:08.322 INFO:teuthology.orchestra.run.trial050.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-12T12:30:08.322 INFO:teuthology.orchestra.run.trial050.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-12T12:30:08.322 INFO:teuthology.orchestra.run.trial050.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-12T12:30:08.322 INFO:teuthology.orchestra.run.trial050.stdout: python3-pluggy-0.13.1-7.el9.noarch 2026-02-12T12:30:08.322 INFO:teuthology.orchestra.run.trial050.stdout: python3-ply-3.11-14.el9.noarch 2026-02-12T12:30:08.322 INFO:teuthology.orchestra.run.trial050.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-12T12:30:08.322 INFO:teuthology.orchestra.run.trial050.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-12T12:30:08.322 INFO:teuthology.orchestra.run.trial050.stdout: python3-py-1.10.0-6.el9.noarch 2026-02-12T12:30:08.322 INFO:teuthology.orchestra.run.trial050.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-12T12:30:08.322 INFO:teuthology.orchestra.run.trial050.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-12T12:30:08.323 INFO:teuthology.orchestra.run.trial050.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-12T12:30:08.323 INFO:teuthology.orchestra.run.trial050.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-12T12:30:08.323 INFO:teuthology.orchestra.run.trial050.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-12T12:30:08.323 INFO:teuthology.orchestra.run.trial050.stdout: python3-pytest-6.2.2-7.el9.noarch 2026-02-12T12:30:08.323 INFO:teuthology.orchestra.run.trial050.stdout: python3-rados-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.323 INFO:teuthology.orchestra.run.trial050.stdout: python3-rbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.323 INFO:teuthology.orchestra.run.trial050.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-12T12:30:08.323 INFO:teuthology.orchestra.run.trial050.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-12T12:30:08.323 INFO:teuthology.orchestra.run.trial050.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-12T12:30:08.323 INFO:teuthology.orchestra.run.trial050.stdout: python3-rgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.323 INFO:teuthology.orchestra.run.trial050.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-12T12:30:08.323 INFO:teuthology.orchestra.run.trial050.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-12T12:30:08.323 INFO:teuthology.orchestra.run.trial050.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-12T12:30:08.324 INFO:teuthology.orchestra.run.trial050.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-12T12:30:08.324 INFO:teuthology.orchestra.run.trial050.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-12T12:30:08.324 INFO:teuthology.orchestra.run.trial050.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-12T12:30:08.324 INFO:teuthology.orchestra.run.trial050.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-12T12:30:08.324 INFO:teuthology.orchestra.run.trial050.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-12T12:30:08.324 INFO:teuthology.orchestra.run.trial050.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-12T12:30:08.324 INFO:teuthology.orchestra.run.trial050.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-12T12:30:08.324 INFO:teuthology.orchestra.run.trial050.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-12T12:30:08.324 INFO:teuthology.orchestra.run.trial050.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-12T12:30:08.324 INFO:teuthology.orchestra.run.trial050.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-12T12:30:08.324 INFO:teuthology.orchestra.run.trial050.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-12T12:30:08.324 INFO:teuthology.orchestra.run.trial050.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-12T12:30:08.324 INFO:teuthology.orchestra.run.trial050.stdout: rbd-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.325 INFO:teuthology.orchestra.run.trial050.stdout: rbd-mirror-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.325 INFO:teuthology.orchestra.run.trial050.stdout: rbd-nbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T12:30:08.325 INFO:teuthology.orchestra.run.trial050.stdout: re2-1:20211101-20.el9.x86_64 2026-02-12T12:30:08.325 INFO:teuthology.orchestra.run.trial050.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-12T12:30:08.325 INFO:teuthology.orchestra.run.trial050.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-12T12:30:08.325 INFO:teuthology.orchestra.run.trial050.stdout: unzip-6.0-59.el9.x86_64 2026-02-12T12:30:08.325 INFO:teuthology.orchestra.run.trial050.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-12T12:30:08.325 INFO:teuthology.orchestra.run.trial050.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-12T12:30:08.325 INFO:teuthology.orchestra.run.trial050.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-12T12:30:08.325 INFO:teuthology.orchestra.run.trial050.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-12T12:30:08.325 INFO:teuthology.orchestra.run.trial050.stdout: zip-3.0-35.el9.x86_64 2026-02-12T12:30:08.325 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:08.326 INFO:teuthology.orchestra.run.trial050.stdout:Complete! 2026-02-12T12:30:08.461 DEBUG:teuthology.parallel:result is None 2026-02-12T12:30:08.462 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 2026-02-12T12:30:08.624 DEBUG:teuthology.orchestra.run.trial050:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-12T12:30:08.644 INFO:teuthology.orchestra.run.trial050.stdout:20.3.0-5145.g5fdb7d8f.el9 2026-02-12T12:30:08.645 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5145.g5fdb7d8f.el9 2026-02-12T12:30:08.645 INFO:teuthology.task.install:The correct ceph version 20.3.0-5145.g5fdb7d8f is installed. 2026-02-12T12:30:08.647 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-12T12:30:08.648 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:30:08.648 DEBUG:teuthology.orchestra.run.trial050:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-12T12:30:08.710 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-12T12:30:08.711 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:30:08.711 DEBUG:teuthology.orchestra.run.trial050:> sudo dd of=/usr/bin/daemon-helper 2026-02-12T12:30:08.774 DEBUG:teuthology.orchestra.run.trial050:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-12T12:30:08.839 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-12T12:30:08.839 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:30:08.839 DEBUG:teuthology.orchestra.run.trial050:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-12T12:30:08.902 DEBUG:teuthology.orchestra.run.trial050:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-12T12:30:08.966 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-12T12:30:08.967 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:30:08.967 DEBUG:teuthology.orchestra.run.trial050:> sudo dd of=/usr/bin/stdin-killer 2026-02-12T12:30:09.032 DEBUG:teuthology.orchestra.run.trial050:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-12T12:30:09.096 INFO:teuthology.run_tasks:Running task pexec... 2026-02-12T12:30:09.115 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-12T12:30:09.115 DEBUG:teuthology.orchestra.run.trial050:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-12T12:30:09.139 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial050.front.sepia.ceph.com 2026-02-12T12:30:09.139 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-12T12:30:09.189 DEBUG:teuthology.parallel:result is None 2026-02-12T12:30:09.189 INFO:teuthology.run_tasks:Running task install... 2026-02-12T12:30:09.203 DEBUG:teuthology.task.install:project ceph 2026-02-12T12:30:09.203 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'}} 2026-02-12T12:30:09.203 DEBUG:teuthology.task.install:config {'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'} 2026-02-12T12:30:09.203 INFO:teuthology.task.install:Using flavor: debug 2026-02-12T12:30:09.212 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-12T12:30:09.212 INFO:teuthology.task.install:extra packages: [] 2026-02-12T12:30:09.212 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': ['python3-pytest'], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0', 'tag': None, 'wait_for_package': False} 2026-02-12T12:30:09.213 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 2026-02-12T12:30:09.369 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-moagrawa-seastore-io-uring-again-crimson-only/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/centos/9/flavors/debug/ 2026-02-12T12:30:09.369 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5145.g5fdb7d8f 2026-02-12T12:30:09.613 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-moagrawa-seastore-io-uring-again-crimson-only/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-moagrawa-seastore-io-uring-again-crimson-only/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-moagrawa-seastore-io-uring-again-crimson-only/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-12T12:30:09.613 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:30:09.613 DEBUG:teuthology.orchestra.run.trial050:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-12T12:30:09.637 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-pytest on remote rpm x86_64 2026-02-12T12:30:09.637 DEBUG:teuthology.orchestra.run.trial050:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-12T12:30:09.701 DEBUG:teuthology.orchestra.run.trial050:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-12T12:30:09.766 DEBUG:teuthology.orchestra.run.trial050:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-12T12:30:09.822 INFO:teuthology.orchestra.run.trial050.stdout:check_obsoletes = 1 2026-02-12T12:30:09.831 DEBUG:teuthology.orchestra.run.trial050:> sudo yum clean all 2026-02-12T12:30:09.995 INFO:teuthology.orchestra.run.trial050.stdout:56 files removed 2026-02-12T12:30:10.011 DEBUG:teuthology.orchestra.run.trial050:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-pytest 2026-02-12T12:30:10.563 INFO:teuthology.orchestra.run.trial050.stdout:ceph packages for x86_64 234 kB/s | 94 kB 00:00 2026-02-12T12:30:10.953 INFO:teuthology.orchestra.run.trial050.stdout:ceph noarch packages 67 kB/s | 24 kB 00:00 2026-02-12T12:30:11.308 INFO:teuthology.orchestra.run.trial050.stdout:ceph source packages 6.2 kB/s | 2.1 kB 00:00 2026-02-12T12:30:12.197 INFO:teuthology.orchestra.run.trial050.stdout:CentOS Stream 9 - BaseOS 10 MB/s | 8.9 MB 00:00 2026-02-12T12:30:16.134 INFO:teuthology.orchestra.run.trial050.stdout:CentOS Stream 9 - AppStream 7.9 MB/s | 27 MB 00:03 2026-02-12T12:30:23.744 INFO:teuthology.orchestra.run.trial050.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 7.8 MB 00:05 2026-02-12T12:30:24.577 INFO:teuthology.orchestra.run.trial050.stdout:CentOS Stream 9 - Extras packages 160 kB/s | 20 kB 00:00 2026-02-12T12:30:31.138 INFO:teuthology.orchestra.run.trial050.stdout:Extra Packages for Enterprise Linux 3.1 MB/s | 20 MB 00:06 2026-02-12T12:30:34.368 INFO:teuthology.orchestra.run.trial050.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-12T12:30:35.613 INFO:teuthology.orchestra.run.trial050.stdout:Package ceph-radosgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.614 INFO:teuthology.orchestra.run.trial050.stdout:Package ceph-test-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.614 INFO:teuthology.orchestra.run.trial050.stdout:Package ceph-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.615 INFO:teuthology.orchestra.run.trial050.stdout:Package ceph-base-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.615 INFO:teuthology.orchestra.run.trial050.stdout:Package cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T12:30:35.615 INFO:teuthology.orchestra.run.trial050.stdout:Package ceph-immutable-object-cache-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.616 INFO:teuthology.orchestra.run.trial050.stdout:Package ceph-mgr-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.616 INFO:teuthology.orchestra.run.trial050.stdout:Package ceph-mgr-dashboard-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T12:30:35.616 INFO:teuthology.orchestra.run.trial050.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T12:30:35.617 INFO:teuthology.orchestra.run.trial050.stdout:Package ceph-mgr-rook-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T12:30:35.617 INFO:teuthology.orchestra.run.trial050.stdout:Package ceph-mgr-cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T12:30:35.618 INFO:teuthology.orchestra.run.trial050.stdout:Package ceph-osd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.618 INFO:teuthology.orchestra.run.trial050.stdout:Package ceph-osd-classic-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.619 INFO:teuthology.orchestra.run.trial050.stdout:Package ceph-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.619 INFO:teuthology.orchestra.run.trial050.stdout:Package ceph-volume-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T12:30:35.619 INFO:teuthology.orchestra.run.trial050.stdout:Package librados-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.620 INFO:teuthology.orchestra.run.trial050.stdout:Package libcephfs2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.620 INFO:teuthology.orchestra.run.trial050.stdout:Package libcephfs-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.621 INFO:teuthology.orchestra.run.trial050.stdout:Package librados2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.621 INFO:teuthology.orchestra.run.trial050.stdout:Package librbd1-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.622 INFO:teuthology.orchestra.run.trial050.stdout:Package python3-rados-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.622 INFO:teuthology.orchestra.run.trial050.stdout:Package python3-rgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.622 INFO:teuthology.orchestra.run.trial050.stdout:Package python3-cephfs-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.623 INFO:teuthology.orchestra.run.trial050.stdout:Package python3-rbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.623 INFO:teuthology.orchestra.run.trial050.stdout:Package rbd-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.624 INFO:teuthology.orchestra.run.trial050.stdout:Package rbd-mirror-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.624 INFO:teuthology.orchestra.run.trial050.stdout:Package rbd-nbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T12:30:35.625 INFO:teuthology.orchestra.run.trial050.stdout:Package python3-pytest-6.2.2-7.el9.noarch is already installed. 2026-02-12T12:30:35.643 INFO:teuthology.orchestra.run.trial050.stdout:Dependencies resolved. 2026-02-12T12:30:35.644 INFO:teuthology.orchestra.run.trial050.stdout:Nothing to do. 2026-02-12T12:30:35.644 INFO:teuthology.orchestra.run.trial050.stdout:Complete! 2026-02-12T12:30:35.670 DEBUG:teuthology.parallel:result is None 2026-02-12T12:30:35.670 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 2026-02-12T12:30:44.308 DEBUG:teuthology.orchestra.run.trial050:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-12T12:30:44.329 INFO:teuthology.orchestra.run.trial050.stdout:20.3.0-5145.g5fdb7d8f.el9 2026-02-12T12:30:44.329 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5145.g5fdb7d8f.el9 2026-02-12T12:30:44.329 INFO:teuthology.task.install:The correct ceph version 20.3.0-5145.g5fdb7d8f is installed. 2026-02-12T12:30:44.332 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-12T12:30:44.332 INFO:teuthology.run_tasks:Running task ceph... 2026-02-12T12:30:44.434 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-12T12:30:44.434 DEBUG:teuthology.orchestra.run.trial050:> sudo chmod 777 /var/log/ceph 2026-02-12T12:30:44.459 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-12T12:30:44.459 DEBUG:teuthology.orchestra.run.trial050:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-12T12:30:44.522 INFO:tasks.ceph:Creating extra log directories... 2026-02-12T12:30:44.522 DEBUG:teuthology.orchestra.run.trial050:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-12T12:30:44.598 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-12T12:30:44.598 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-12T12:30:44.598 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/moagrawa-2026-02-10_06:35:43-crimson-rados-main-distro-debug-trial/42514', 'branch': 'main', 'description': 'crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} tasks/rbd_python_api_tests_old_format}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '42514', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'moagrawa-2026-02-10_06:35:43-crimson-rados-main-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'slow request'], 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'}}, 'workunit': {'branch': 'seastore_io_uring', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'}}, 'owner': 'scheduled_moagrawa@soko04.front.sepia.ceph.com', 'priority': 1000, 'repo': 'https://github.com/ceph/ceph.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 2425, 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'seastore_io_uring', 'suite_path': '/home/teuthworker/src/github.com_mohit84_ceph_5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/mohit84/ceph.git', 'suite_sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0', 'targets': {'trial050.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBDG7oXs0XnfkuI46eO/CRk9qtDe7WEwaHkYWg3wGGv1tSLFo/eNvhNvgO1yJA0Qq3+T4CFdwTgl17clpTR+9Xds='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'slow request'], 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rbd/test_librbd_python.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'c433f1062990a0488dc29a553589bc609a460691', 'timestamp': '2026-02-10_06:35:43', 'tube': 'trial', 'user': 'moagrawa', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.2161924'} 2026-02-12T12:30:44.599 DEBUG:teuthology.orchestra.run.trial050:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-12T12:30:44.645 DEBUG:teuthology.orchestra.run.trial050:> sudo install -d -m0777 -- /var/run/ceph 2026-02-12T12:30:44.709 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:30:44.710 DEBUG:teuthology.orchestra.run.trial050:> dd if=/scratch_devs of=/dev/stdout 2026-02-12T12:30:44.767 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-12T12:30:44.768 DEBUG:teuthology.orchestra.run.trial050:> stat /dev/vg_nvme/lv_1 2026-02-12T12:30:44.825 INFO:teuthology.orchestra.run.trial050.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-12T12:30:44.825 INFO:teuthology.orchestra.run.trial050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-12T12:30:44.825 INFO:teuthology.orchestra.run.trial050.stdout:Device: 6h/6d Inode: 1305 Links: 1 2026-02-12T12:30:44.825 INFO:teuthology.orchestra.run.trial050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-12T12:30:44.826 INFO:teuthology.orchestra.run.trial050.stdout:Context: system_u:object_r:device_t:s0 2026-02-12T12:30:44.826 INFO:teuthology.orchestra.run.trial050.stdout:Access: 2026-02-12 12:30:05.239379101 +0000 2026-02-12T12:30:44.826 INFO:teuthology.orchestra.run.trial050.stdout:Modify: 2026-02-12 12:29:18.741168580 +0000 2026-02-12T12:30:44.826 INFO:teuthology.orchestra.run.trial050.stdout:Change: 2026-02-12 12:29:18.741168580 +0000 2026-02-12T12:30:44.826 INFO:teuthology.orchestra.run.trial050.stdout: Birth: 2026-02-12 12:29:18.741168580 +0000 2026-02-12T12:30:44.826 DEBUG:teuthology.orchestra.run.trial050:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-12T12:30:44.889 INFO:teuthology.orchestra.run.trial050.stderr:1+0 records in 2026-02-12T12:30:44.890 INFO:teuthology.orchestra.run.trial050.stderr:1+0 records out 2026-02-12T12:30:44.890 INFO:teuthology.orchestra.run.trial050.stderr:512 bytes copied, 0.00015597 s, 3.3 MB/s 2026-02-12T12:30:44.891 DEBUG:teuthology.orchestra.run.trial050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-12T12:30:44.948 DEBUG:teuthology.orchestra.run.trial050:> stat /dev/vg_nvme/lv_2 2026-02-12T12:30:45.004 INFO:teuthology.orchestra.run.trial050.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-12T12:30:45.005 INFO:teuthology.orchestra.run.trial050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-12T12:30:45.005 INFO:teuthology.orchestra.run.trial050.stdout:Device: 6h/6d Inode: 1315 Links: 1 2026-02-12T12:30:45.005 INFO:teuthology.orchestra.run.trial050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-12T12:30:45.005 INFO:teuthology.orchestra.run.trial050.stdout:Context: system_u:object_r:device_t:s0 2026-02-12T12:30:45.005 INFO:teuthology.orchestra.run.trial050.stdout:Access: 2026-02-12 12:30:05.239379101 +0000 2026-02-12T12:30:45.005 INFO:teuthology.orchestra.run.trial050.stdout:Modify: 2026-02-12 12:29:18.742168563 +0000 2026-02-12T12:30:45.005 INFO:teuthology.orchestra.run.trial050.stdout:Change: 2026-02-12 12:29:18.742168563 +0000 2026-02-12T12:30:45.005 INFO:teuthology.orchestra.run.trial050.stdout: Birth: 2026-02-12 12:29:18.742168563 +0000 2026-02-12T12:30:45.005 DEBUG:teuthology.orchestra.run.trial050:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-12T12:30:45.066 INFO:teuthology.orchestra.run.trial050.stderr:1+0 records in 2026-02-12T12:30:45.066 INFO:teuthology.orchestra.run.trial050.stderr:1+0 records out 2026-02-12T12:30:45.067 INFO:teuthology.orchestra.run.trial050.stderr:512 bytes copied, 0.000130603 s, 3.9 MB/s 2026-02-12T12:30:45.068 DEBUG:teuthology.orchestra.run.trial050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-12T12:30:45.125 DEBUG:teuthology.orchestra.run.trial050:> stat /dev/vg_nvme/lv_3 2026-02-12T12:30:45.182 INFO:teuthology.orchestra.run.trial050.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-12T12:30:45.182 INFO:teuthology.orchestra.run.trial050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-12T12:30:45.182 INFO:teuthology.orchestra.run.trial050.stdout:Device: 6h/6d Inode: 1301 Links: 1 2026-02-12T12:30:45.182 INFO:teuthology.orchestra.run.trial050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-12T12:30:45.182 INFO:teuthology.orchestra.run.trial050.stdout:Context: system_u:object_r:device_t:s0 2026-02-12T12:30:45.182 INFO:teuthology.orchestra.run.trial050.stdout:Access: 2026-02-12 12:30:05.240379083 +0000 2026-02-12T12:30:45.182 INFO:teuthology.orchestra.run.trial050.stdout:Modify: 2026-02-12 12:29:18.741168580 +0000 2026-02-12T12:30:45.182 INFO:teuthology.orchestra.run.trial050.stdout:Change: 2026-02-12 12:29:18.741168580 +0000 2026-02-12T12:30:45.183 INFO:teuthology.orchestra.run.trial050.stdout: Birth: 2026-02-12 12:29:18.741168580 +0000 2026-02-12T12:30:45.183 DEBUG:teuthology.orchestra.run.trial050:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-12T12:30:45.244 INFO:teuthology.orchestra.run.trial050.stderr:1+0 records in 2026-02-12T12:30:45.244 INFO:teuthology.orchestra.run.trial050.stderr:1+0 records out 2026-02-12T12:30:45.244 INFO:teuthology.orchestra.run.trial050.stderr:512 bytes copied, 0.000190364 s, 2.7 MB/s 2026-02-12T12:30:45.246 DEBUG:teuthology.orchestra.run.trial050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-12T12:30:45.303 DEBUG:teuthology.orchestra.run.trial050:> stat /dev/vg_nvme/lv_4 2026-02-12T12:30:45.361 INFO:teuthology.orchestra.run.trial050.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-12T12:30:45.361 INFO:teuthology.orchestra.run.trial050.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-12T12:30:45.361 INFO:teuthology.orchestra.run.trial050.stdout:Device: 6h/6d Inode: 1296 Links: 1 2026-02-12T12:30:45.361 INFO:teuthology.orchestra.run.trial050.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-12T12:30:45.361 INFO:teuthology.orchestra.run.trial050.stdout:Context: system_u:object_r:device_t:s0 2026-02-12T12:30:45.361 INFO:teuthology.orchestra.run.trial050.stdout:Access: 2026-02-12 12:30:05.240379083 +0000 2026-02-12T12:30:45.361 INFO:teuthology.orchestra.run.trial050.stdout:Modify: 2026-02-12 12:29:18.740168598 +0000 2026-02-12T12:30:45.361 INFO:teuthology.orchestra.run.trial050.stdout:Change: 2026-02-12 12:29:18.740168598 +0000 2026-02-12T12:30:45.361 INFO:teuthology.orchestra.run.trial050.stdout: Birth: 2026-02-12 12:29:18.740168598 +0000 2026-02-12T12:30:45.362 DEBUG:teuthology.orchestra.run.trial050:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-12T12:30:45.425 INFO:teuthology.orchestra.run.trial050.stderr:1+0 records in 2026-02-12T12:30:45.426 INFO:teuthology.orchestra.run.trial050.stderr:1+0 records out 2026-02-12T12:30:45.426 INFO:teuthology.orchestra.run.trial050.stderr:512 bytes copied, 0.000137776 s, 3.7 MB/s 2026-02-12T12:30:45.427 DEBUG:teuthology.orchestra.run.trial050:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-12T12:30:45.484 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-12T12:30:45.484 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial050.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-12T12:30:45.484 INFO:tasks.ceph:Generating config... 2026-02-12T12:30:45.486 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2026-02-12T12:30:45.486 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-12T12:30:45.486 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-12T12:30:45.487 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-12T12:30:45.487 INFO:tasks.ceph:[mon] paxos service trim min = 10 2026-02-12T12:30:45.487 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-12T12:30:45.487 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-12T12:30:45.487 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-12T12:30:45.487 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-12T12:30:45.487 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-12T12:30:45.487 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-12T12:30:45.487 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-12T12:30:45.487 INFO:tasks.ceph:[osd] debug ms = 20 2026-02-12T12:30:45.487 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-12T12:30:45.488 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-12T12:30:45.488 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-12T12:30:45.488 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-12T12:30:45.488 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-12T12:30:45.488 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-12T12:30:45.488 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-12T12:30:45.488 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-12T12:30:45.488 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-12T12:30:45.488 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-12T12:30:45.488 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-12T12:30:45.488 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-12T12:30:45.488 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-12T12:30:45.488 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-12T12:30:45.489 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-12T12:30:45.489 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-12T12:30:45.489 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-12T12:30:45.489 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-12T12:30:45.489 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-12T12:30:45.489 INFO:tasks.ceph:[osd] seastore main device type = RANDOM_BLOCK_SSD 2026-02-12T12:30:45.489 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-12T12:30:45.489 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-12T12:30:45.489 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-12T12:30:45.489 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-12T12:30:45.489 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-12T12:30:45.489 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-12T12:30:45.490 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-12T12:30:45.490 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-12T12:30:45.490 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-12T12:30:45.490 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-12T12:30:45.490 INFO:tasks.ceph:Setting up mon.a... 2026-02-12T12:30:45.490 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-12T12:30:45.566 INFO:teuthology.orchestra.run.trial050.stdout:creating /etc/ceph/ceph.keyring 2026-02-12T12:30:45.569 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-12T12:30:45.652 DEBUG:teuthology.orchestra.run.trial050:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-12T12:30:45.716 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.50')] 2026-02-12T12:30:45.717 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.50', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-12T12:30:45.719 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:30:45.719 DEBUG:teuthology.orchestra.run.trial050:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-12T12:30:45.773 DEBUG:teuthology.orchestra.run.trial050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.50 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-12T12:30:45.855 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:30:45.853+0000 7f513aa8ef40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:30:45.855 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:30:45.853+0000 7f513aa8ef40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:30:45.855 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:30:45.853+0000 7f513aa8ef40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:30:45.857 INFO:teuthology.orchestra.run.trial050.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-12T12:30:45.857 INFO:teuthology.orchestra.run.trial050.stdout:monmaptool: generated fsid 2a7ffb28-5872-4384-aac5-3a45ba84b5fb 2026-02-12T12:30:45.857 INFO:teuthology.orchestra.run.trial050.stdout:setting min_mon_release = tentacle 2026-02-12T12:30:45.857 INFO:teuthology.orchestra.run.trial050.stdout:epoch 0 2026-02-12T12:30:45.857 INFO:teuthology.orchestra.run.trial050.stdout:fsid 2a7ffb28-5872-4384-aac5-3a45ba84b5fb 2026-02-12T12:30:45.857 INFO:teuthology.orchestra.run.trial050.stdout:last_changed 2026-02-12T12:30:45.854526+0000 2026-02-12T12:30:45.857 INFO:teuthology.orchestra.run.trial050.stdout:created 2026-02-12T12:30:45.854526+0000 2026-02-12T12:30:45.857 INFO:teuthology.orchestra.run.trial050.stdout:min_mon_release 20 (tentacle) 2026-02-12T12:30:45.857 INFO:teuthology.orchestra.run.trial050.stdout:election_strategy: 1 2026-02-12T12:30:45.858 INFO:teuthology.orchestra.run.trial050.stdout:0: [v2:10.20.193.50:3300/0,v1:10.20.193.50:6789/0] mon.a 2026-02-12T12:30:45.858 INFO:teuthology.orchestra.run.trial050.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-12T12:30:45.858 DEBUG:teuthology.orchestra.run.trial050:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-12T12:30:45.915 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 2a7ffb28-5872-4384-aac5-3a45ba84b5fb... 2026-02-12T12:30:45.916 DEBUG:teuthology.orchestra.run.trial050:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-12T12:30:45.995 INFO:teuthology.orchestra.run.trial050.stdout:[global] 2026-02-12T12:30:45.996 INFO:teuthology.orchestra.run.trial050.stdout: chdir = "" 2026-02-12T12:30:45.996 INFO:teuthology.orchestra.run.trial050.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-12T12:30:45.996 INFO:teuthology.orchestra.run.trial050.stdout: auth supported = cephx 2026-02-12T12:30:45.996 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:45.996 INFO:teuthology.orchestra.run.trial050.stdout: filestore xattr use omap = true 2026-02-12T12:30:45.996 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:45.996 INFO:teuthology.orchestra.run.trial050.stdout: mon clock drift allowed = 1.000 2026-02-12T12:30:45.996 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:45.997 INFO:teuthology.orchestra.run.trial050.stdout: osd crush chooseleaf type = 0 2026-02-12T12:30:45.997 INFO:teuthology.orchestra.run.trial050.stdout: auth debug = true 2026-02-12T12:30:45.997 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:45.997 INFO:teuthology.orchestra.run.trial050.stdout: ms die on old message = true 2026-02-12T12:30:45.997 INFO:teuthology.orchestra.run.trial050.stdout: ms die on bug = true 2026-02-12T12:30:45.997 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:45.997 INFO:teuthology.orchestra.run.trial050.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-12T12:30:45.997 INFO:teuthology.orchestra.run.trial050.stdout: mon pg warn max object skew = 0 2026-02-12T12:30:45.997 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:45.997 INFO:teuthology.orchestra.run.trial050.stdout: # disable pg_autoscaler by default for new pools 2026-02-12T12:30:45.997 INFO:teuthology.orchestra.run.trial050.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-12T12:30:45.997 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:45.998 INFO:teuthology.orchestra.run.trial050.stdout: osd pool default size = 2 2026-02-12T12:30:45.998 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:45.998 INFO:teuthology.orchestra.run.trial050.stdout: mon osd allow primary affinity = true 2026-02-12T12:30:45.998 INFO:teuthology.orchestra.run.trial050.stdout: mon osd allow pg remap = true 2026-02-12T12:30:45.998 INFO:teuthology.orchestra.run.trial050.stdout: mon warn on legacy crush tunables = false 2026-02-12T12:30:45.998 INFO:teuthology.orchestra.run.trial050.stdout: mon warn on crush straw calc version zero = false 2026-02-12T12:30:45.998 INFO:teuthology.orchestra.run.trial050.stdout: mon warn on no sortbitwise = false 2026-02-12T12:30:45.998 INFO:teuthology.orchestra.run.trial050.stdout: mon warn on osd down out interval zero = false 2026-02-12T12:30:45.998 INFO:teuthology.orchestra.run.trial050.stdout: mon warn on too few osds = false 2026-02-12T12:30:45.998 INFO:teuthology.orchestra.run.trial050.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-12T12:30:45.998 INFO:teuthology.orchestra.run.trial050.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-12T12:30:45.998 INFO:teuthology.orchestra.run.trial050.stdout: mon_allow_pool_size_one = true 2026-02-12T12:30:45.998 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:45.999 INFO:teuthology.orchestra.run.trial050.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-12T12:30:45.999 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:45.999 INFO:teuthology.orchestra.run.trial050.stdout: osd default data pool replay window = 5 2026-02-12T12:30:45.999 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:45.999 INFO:teuthology.orchestra.run.trial050.stdout: mon allow pool delete = true 2026-02-12T12:30:45.999 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:45.999 INFO:teuthology.orchestra.run.trial050.stdout: mon cluster log file level = debug 2026-02-12T12:30:45.999 INFO:teuthology.orchestra.run.trial050.stdout: debug asserts on shutdown = true 2026-02-12T12:30:45.999 INFO:teuthology.orchestra.run.trial050.stdout: mon health detail to clog = false 2026-02-12T12:30:45.999 INFO:teuthology.orchestra.run.trial050.stdout: mon host = 10.20.193.50 2026-02-12T12:30:45.999 INFO:teuthology.orchestra.run.trial050.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-12T12:30:45.999 INFO:teuthology.orchestra.run.trial050.stdout: ms client mode = crc 2026-02-12T12:30:46.000 INFO:teuthology.orchestra.run.trial050.stdout: ms cluster mode = crc 2026-02-12T12:30:46.000 INFO:teuthology.orchestra.run.trial050.stdout: ms mon client mode = crc 2026-02-12T12:30:46.000 INFO:teuthology.orchestra.run.trial050.stdout: ms mon cluster mode = crc 2026-02-12T12:30:46.000 INFO:teuthology.orchestra.run.trial050.stdout: ms mon service mode = crc 2026-02-12T12:30:46.000 INFO:teuthology.orchestra.run.trial050.stdout: ms service mode = crc 2026-02-12T12:30:46.000 INFO:teuthology.orchestra.run.trial050.stdout: fsid = 2a7ffb28-5872-4384-aac5-3a45ba84b5fb 2026-02-12T12:30:46.000 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.000 INFO:teuthology.orchestra.run.trial050.stdout:[osd] 2026-02-12T12:30:46.000 INFO:teuthology.orchestra.run.trial050.stdout: osd journal size = 100 2026-02-12T12:30:46.000 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.000 INFO:teuthology.orchestra.run.trial050.stdout: osd scrub load threshold = 5.0 2026-02-12T12:30:46.000 INFO:teuthology.orchestra.run.trial050.stdout: osd scrub max interval = 600 2026-02-12T12:30:46.001 INFO:teuthology.orchestra.run.trial050.stdout: osd mclock profile = high_recovery_ops 2026-02-12T12:30:46.001 INFO:teuthology.orchestra.run.trial050.stdout: osd mclock skip benchmark = true 2026-02-12T12:30:46.001 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.001 INFO:teuthology.orchestra.run.trial050.stdout: osd recover clone overlap = true 2026-02-12T12:30:46.001 INFO:teuthology.orchestra.run.trial050.stdout: osd recovery max chunk = 1048576 2026-02-12T12:30:46.001 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.001 INFO:teuthology.orchestra.run.trial050.stdout: osd debug shutdown = true 2026-02-12T12:30:46.001 INFO:teuthology.orchestra.run.trial050.stdout: osd debug op order = true 2026-02-12T12:30:46.001 INFO:teuthology.orchestra.run.trial050.stdout: osd debug verify stray on activate = true 2026-02-12T12:30:46.001 INFO:teuthology.orchestra.run.trial050.stdout: osd debug trim objects = true 2026-02-12T12:30:46.001 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.001 INFO:teuthology.orchestra.run.trial050.stdout: osd open classes on start = true 2026-02-12T12:30:46.002 INFO:teuthology.orchestra.run.trial050.stdout: osd debug pg log writeout = true 2026-02-12T12:30:46.002 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.002 INFO:teuthology.orchestra.run.trial050.stdout: osd deep scrub update digest min age = 30 2026-02-12T12:30:46.002 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.002 INFO:teuthology.orchestra.run.trial050.stdout: osd map max advance = 10 2026-02-12T12:30:46.002 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.002 INFO:teuthology.orchestra.run.trial050.stdout: journal zero on create = true 2026-02-12T12:30:46.002 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.003 INFO:teuthology.orchestra.run.trial050.stdout: filestore ondisk finisher threads = 3 2026-02-12T12:30:46.003 INFO:teuthology.orchestra.run.trial050.stdout: filestore apply finisher threads = 3 2026-02-12T12:30:46.003 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.003 INFO:teuthology.orchestra.run.trial050.stdout: bdev debug aio = true 2026-02-12T12:30:46.003 INFO:teuthology.orchestra.run.trial050.stdout: osd debug misdirected ops = true 2026-02-12T12:30:46.003 INFO:teuthology.orchestra.run.trial050.stdout: debug monc = 20 2026-02-12T12:30:46.003 INFO:teuthology.orchestra.run.trial050.stdout: crimson cpu num = 4 2026-02-12T12:30:46.003 INFO:teuthology.orchestra.run.trial050.stdout: crimson osd obc lru size = 10 2026-02-12T12:30:46.003 INFO:teuthology.orchestra.run.trial050.stdout: debug ms = 20 2026-02-12T12:30:46.003 INFO:teuthology.orchestra.run.trial050.stdout: debug osd = 20 2026-02-12T12:30:46.003 INFO:teuthology.orchestra.run.trial050.stdout: debug seastore = 20 2026-02-12T12:30:46.003 INFO:teuthology.orchestra.run.trial050.stdout: debug seastore backref = 20 2026-02-12T12:30:46.003 INFO:teuthology.orchestra.run.trial050.stdout: debug seastore cache = 20 2026-02-12T12:30:46.004 INFO:teuthology.orchestra.run.trial050.stdout: debug seastore cleaner = 20 2026-02-12T12:30:46.004 INFO:teuthology.orchestra.run.trial050.stdout: debug seastore device = 20 2026-02-12T12:30:46.004 INFO:teuthology.orchestra.run.trial050.stdout: debug seastore epm = 20 2026-02-12T12:30:46.004 INFO:teuthology.orchestra.run.trial050.stdout: debug seastore fixedkv tree = 20 2026-02-12T12:30:46.004 INFO:teuthology.orchestra.run.trial050.stdout: debug seastore journal = 20 2026-02-12T12:30:46.004 INFO:teuthology.orchestra.run.trial050.stdout: debug seastore lba = 20 2026-02-12T12:30:46.004 INFO:teuthology.orchestra.run.trial050.stdout: debug seastore odata = 20 2026-02-12T12:30:46.004 INFO:teuthology.orchestra.run.trial050.stdout: debug seastore omap = 20 2026-02-12T12:30:46.004 INFO:teuthology.orchestra.run.trial050.stdout: debug seastore onode = 20 2026-02-12T12:30:46.004 INFO:teuthology.orchestra.run.trial050.stdout: debug seastore t = 20 2026-02-12T12:30:46.004 INFO:teuthology.orchestra.run.trial050.stdout: debug seastore tm = 20 2026-02-12T12:30:46.004 INFO:teuthology.orchestra.run.trial050.stdout: osd objectstore = seastore 2026-02-12T12:30:46.004 INFO:teuthology.orchestra.run.trial050.stdout: osd shutdown pgref assert = True 2026-02-12T12:30:46.005 INFO:teuthology.orchestra.run.trial050.stdout: seastore cachepin size pershard = 64M 2026-02-12T12:30:46.005 INFO:teuthology.orchestra.run.trial050.stdout: seastore cachepin type = 2Q 2026-02-12T12:30:46.005 INFO:teuthology.orchestra.run.trial050.stdout: seastore main device type = RANDOM_BLOCK_SSD 2026-02-12T12:30:46.005 INFO:teuthology.orchestra.run.trial050.stdout: seastore max concurrent transactions = 8 2026-02-12T12:30:46.005 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.005 INFO:teuthology.orchestra.run.trial050.stdout:[mgr] 2026-02-12T12:30:46.005 INFO:teuthology.orchestra.run.trial050.stdout: debug ms = 1 2026-02-12T12:30:46.005 INFO:teuthology.orchestra.run.trial050.stdout: debug mgr = 20 2026-02-12T12:30:46.005 INFO:teuthology.orchestra.run.trial050.stdout: debug mon = 20 2026-02-12T12:30:46.005 INFO:teuthology.orchestra.run.trial050.stdout: debug auth = 20 2026-02-12T12:30:46.005 INFO:teuthology.orchestra.run.trial050.stdout: mon reweight min pgs per osd = 4 2026-02-12T12:30:46.005 INFO:teuthology.orchestra.run.trial050.stdout: mon reweight min bytes per osd = 10 2026-02-12T12:30:46.006 INFO:teuthology.orchestra.run.trial050.stdout: mgr/telemetry/nag = false 2026-02-12T12:30:46.006 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.006 INFO:teuthology.orchestra.run.trial050.stdout:[mon] 2026-02-12T12:30:46.006 INFO:teuthology.orchestra.run.trial050.stdout: debug ms = 1 2026-02-12T12:30:46.006 INFO:teuthology.orchestra.run.trial050.stdout: debug mon = 20 2026-02-12T12:30:46.006 INFO:teuthology.orchestra.run.trial050.stdout: debug paxos = 20 2026-02-12T12:30:46.006 INFO:teuthology.orchestra.run.trial050.stdout: debug auth = 20 2026-02-12T12:30:46.006 INFO:teuthology.orchestra.run.trial050.stdout: mon data avail warn = 5 2026-02-12T12:30:46.006 INFO:teuthology.orchestra.run.trial050.stdout: mon mgr mkfs grace = 240 2026-02-12T12:30:46.006 INFO:teuthology.orchestra.run.trial050.stdout: mon reweight min pgs per osd = 4 2026-02-12T12:30:46.006 INFO:teuthology.orchestra.run.trial050.stdout: mon osd reporter subtree level = osd 2026-02-12T12:30:46.006 INFO:teuthology.orchestra.run.trial050.stdout: mon osd prime pg temp = true 2026-02-12T12:30:46.007 INFO:teuthology.orchestra.run.trial050.stdout: mon reweight min bytes per osd = 10 2026-02-12T12:30:46.007 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.007 INFO:teuthology.orchestra.run.trial050.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-12T12:30:46.007 INFO:teuthology.orchestra.run.trial050.stdout: auth mon ticket ttl = 660 # 11m 2026-02-12T12:30:46.007 INFO:teuthology.orchestra.run.trial050.stdout: auth service ticket ttl = 240 # 4m 2026-02-12T12:30:46.007 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.007 INFO:teuthology.orchestra.run.trial050.stdout: # don't complain about insecure global_id in the test suite 2026-02-12T12:30:46.007 INFO:teuthology.orchestra.run.trial050.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-12T12:30:46.007 INFO:teuthology.orchestra.run.trial050.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-12T12:30:46.007 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.007 INFO:teuthology.orchestra.run.trial050.stdout: # 1m isn't quite enough 2026-02-12T12:30:46.007 INFO:teuthology.orchestra.run.trial050.stdout: mon_down_mkfs_grace = 2m 2026-02-12T12:30:46.008 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.008 INFO:teuthology.orchestra.run.trial050.stdout: mon_warn_on_filestore_osds = false 2026-02-12T12:30:46.008 INFO:teuthology.orchestra.run.trial050.stdout: mon min osdmap epochs = 50 2026-02-12T12:30:46.008 INFO:teuthology.orchestra.run.trial050.stdout: mon osdmap full prune interval = 2 2026-02-12T12:30:46.008 INFO:teuthology.orchestra.run.trial050.stdout: mon osdmap full prune min = 15 2026-02-12T12:30:46.008 INFO:teuthology.orchestra.run.trial050.stdout: mon osdmap full prune txsize = 2 2026-02-12T12:30:46.008 INFO:teuthology.orchestra.run.trial050.stdout: paxos service trim min = 10 2026-02-12T12:30:46.008 INFO:teuthology.orchestra.run.trial050.stdout: osd pool default crimson = True 2026-02-12T12:30:46.008 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:30:46.008 INFO:teuthology.orchestra.run.trial050.stdout:[client] 2026-02-12T12:30:46.008 INFO:teuthology.orchestra.run.trial050.stdout: rgw cache enabled = true 2026-02-12T12:30:46.008 INFO:teuthology.orchestra.run.trial050.stdout: rgw enable ops log = true 2026-02-12T12:30:46.008 INFO:teuthology.orchestra.run.trial050.stdout: rgw enable usage log = true 2026-02-12T12:30:46.009 INFO:teuthology.orchestra.run.trial050.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-12T12:30:46.009 INFO:teuthology.orchestra.run.trial050.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-12T12:30:46.009 INFO:teuthology.orchestra.run.trial050.stdout:[mon.a] 2026-02-12T12:30:46.010 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-12T12:30:46.010 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-12T12:30:46.100 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-12T12:30:46.101 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:30:46.101 DEBUG:teuthology.orchestra.run.trial050:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-12T12:30:46.117 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:30:46.118 DEBUG:teuthology.orchestra.run.trial050:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-12T12:30:46.175 INFO:tasks.ceph:Sending monmap to node ubuntu@trial050.front.sepia.ceph.com 2026-02-12T12:30:46.176 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:30:46.176 DEBUG:teuthology.orchestra.run.trial050:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-12T12:30:46.176 DEBUG:teuthology.orchestra.run.trial050:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-12T12:30:46.248 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:30:46.248 DEBUG:teuthology.orchestra.run.trial050:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-12T12:30:46.305 INFO:tasks.ceph:Setting up mon nodes... 2026-02-12T12:30:46.305 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-12T12:30:46.305 DEBUG:teuthology.orchestra.run.trial050:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-12T12:30:46.397 INFO:teuthology.orchestra.run.trial050.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-12T12:30:46.399 INFO:tasks.ceph:Setting up mds nodes... 2026-02-12T12:30:46.399 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-12T12:30:46.400 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-12T12:30:46.442 INFO:teuthology.orchestra.run.trial050.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-12T12:30:46.453 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-12T12:30:46.453 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial050.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-12T12:30:46.454 DEBUG:teuthology.orchestra.run.trial050:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-12T12:30:46.517 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-12T12:30:46.517 INFO:tasks.ceph:role: osd.0 2026-02-12T12:30:46.517 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial050.front.sepia.ceph.com 2026-02-12T12:30:46.517 DEBUG:teuthology.orchestra.run.trial050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-12T12:30:46.584 INFO:teuthology.orchestra.run.trial050.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-12T12:30:46.584 INFO:teuthology.orchestra.run.trial050.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-12T12:30:46.584 INFO:teuthology.orchestra.run.trial050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-12T12:30:46.584 INFO:teuthology.orchestra.run.trial050.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-12T12:30:46.584 INFO:teuthology.orchestra.run.trial050.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-12T12:30:46.584 INFO:teuthology.orchestra.run.trial050.stdout: = sunit=0 swidth=0 blks 2026-02-12T12:30:46.584 INFO:teuthology.orchestra.run.trial050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-12T12:30:46.585 INFO:teuthology.orchestra.run.trial050.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-12T12:30:46.585 INFO:teuthology.orchestra.run.trial050.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-12T12:30:46.585 INFO:teuthology.orchestra.run.trial050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-12T12:30:47.205 INFO:teuthology.orchestra.run.trial050.stdout:Discarding blocks...Done. 2026-02-12T12:30:47.207 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial050.front.sepia.ceph.com -o noatime 2026-02-12T12:30:47.207 DEBUG:teuthology.orchestra.run.trial050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-12T12:30:47.276 DEBUG:teuthology.orchestra.run.trial050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-12T12:30:47.343 DEBUG:teuthology.orchestra.run.trial050:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-12T12:30:47.408 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-12T12:30:47.408 INFO:tasks.ceph:role: osd.1 2026-02-12T12:30:47.409 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial050.front.sepia.ceph.com 2026-02-12T12:30:47.409 DEBUG:teuthology.orchestra.run.trial050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-12T12:30:47.473 INFO:teuthology.orchestra.run.trial050.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-12T12:30:47.473 INFO:teuthology.orchestra.run.trial050.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-12T12:30:47.473 INFO:teuthology.orchestra.run.trial050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-12T12:30:47.473 INFO:teuthology.orchestra.run.trial050.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-12T12:30:47.473 INFO:teuthology.orchestra.run.trial050.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-12T12:30:47.474 INFO:teuthology.orchestra.run.trial050.stdout: = sunit=0 swidth=0 blks 2026-02-12T12:30:47.474 INFO:teuthology.orchestra.run.trial050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-12T12:30:47.474 INFO:teuthology.orchestra.run.trial050.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-12T12:30:47.474 INFO:teuthology.orchestra.run.trial050.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-12T12:30:47.474 INFO:teuthology.orchestra.run.trial050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-12T12:30:48.045 INFO:teuthology.orchestra.run.trial050.stdout:Discarding blocks...Done. 2026-02-12T12:30:48.047 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial050.front.sepia.ceph.com -o noatime 2026-02-12T12:30:48.047 DEBUG:teuthology.orchestra.run.trial050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-12T12:30:48.120 DEBUG:teuthology.orchestra.run.trial050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-12T12:30:48.186 DEBUG:teuthology.orchestra.run.trial050:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-12T12:30:48.251 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-12T12:30:48.251 INFO:tasks.ceph:role: osd.2 2026-02-12T12:30:48.252 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial050.front.sepia.ceph.com 2026-02-12T12:30:48.252 DEBUG:teuthology.orchestra.run.trial050:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-12T12:30:48.316 INFO:teuthology.orchestra.run.trial050.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-12T12:30:48.316 INFO:teuthology.orchestra.run.trial050.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-12T12:30:48.316 INFO:teuthology.orchestra.run.trial050.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-12T12:30:48.316 INFO:teuthology.orchestra.run.trial050.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-12T12:30:48.316 INFO:teuthology.orchestra.run.trial050.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-12T12:30:48.316 INFO:teuthology.orchestra.run.trial050.stdout: = sunit=0 swidth=0 blks 2026-02-12T12:30:48.316 INFO:teuthology.orchestra.run.trial050.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-12T12:30:48.317 INFO:teuthology.orchestra.run.trial050.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-12T12:30:48.317 INFO:teuthology.orchestra.run.trial050.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-12T12:30:48.317 INFO:teuthology.orchestra.run.trial050.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-12T12:30:48.923 INFO:teuthology.orchestra.run.trial050.stdout:Discarding blocks...Done. 2026-02-12T12:30:48.925 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial050.front.sepia.ceph.com -o noatime 2026-02-12T12:30:48.925 DEBUG:teuthology.orchestra.run.trial050:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-12T12:30:48.994 DEBUG:teuthology.orchestra.run.trial050:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-12T12:30:49.060 DEBUG:teuthology.orchestra.run.trial050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-12T12:30:49.209 INFO:teuthology.orchestra.run.trial050.stderr:INFO  osd - OSD::main: parsing early config 2026-02-12T12:30:49.265 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:49,264 seastar - Reactor backend: linux-aio 2026-02-12T12:30:49.283 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:49,282 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T12:30:49.283 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:49,282 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T12:30:49.285 INFO:teuthology.orchestra.run.trial050.stderr:==18171==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T12:30:49.341 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:49,339 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-12T12:30:49.341 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:49,339 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-12T12:30:49.341 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:49,339 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-12T12:30:49.341 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:49,339 [shard 0:main] osd - Configure option_name crimson_reactor_backend with value : linux-aio 2026-02-12T12:30:49.341 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:49,339 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-12T12:30:49.341 INFO:teuthology.orchestra.run.trial050.stderr:DEBUG 2026-02-12 12:30:49,340 [shard 0:main] seastar - reactor::drain 2026-02-12T12:30:51.437 INFO:teuthology.orchestra.run.trial050.stderr:----------------------------------------------------- 2026-02-12T12:30:51.437 INFO:teuthology.orchestra.run.trial050.stderr:Suppressions used: 2026-02-12T12:30:51.437 INFO:teuthology.orchestra.run.trial050.stderr: count bytes template 2026-02-12T12:30:51.437 INFO:teuthology.orchestra.run.trial050.stderr: 1 8 MallocExtension::Register 2026-02-12T12:30:51.437 INFO:teuthology.orchestra.run.trial050.stderr:----------------------------------------------------- 2026-02-12T12:30:51.437 INFO:teuthology.orchestra.run.trial050.stderr: 2026-02-12T12:30:51.456 INFO:teuthology.orchestra.run.trial050.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-12T12:30:51.456 INFO:teuthology.orchestra.run.trial050.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-12T12:30:51.501 INFO:teuthology.orchestra.run.trial050.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-12T12:30:51.501 INFO:teuthology.orchestra.run.trial050.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-12T12:30:51.508 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:51,507 seastar - Reactor backend: linux-aio 2026-02-12T12:30:51.522 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:51,521 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T12:30:51.522 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:51,521 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T12:30:51.525 INFO:teuthology.orchestra.run.trial050.stderr:==18169==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T12:30:51.526 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:51,524 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-12T12:30:51.584 INFO:teuthology.orchestra.run.trial050.stderr:DEBUG 2026-02-12 12:30:51,583 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-12T12:30:51.663 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f 0x9c62298 0x9d2512a 0x9d65ae1 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:30:51.664 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb13e7 0xb6a317 0x1092c80 0xf2fdae 0x9c70450 0x9c7074d 0x9c709c0 0x9c719c0 0x9c79a43 0x9c79dbf 0x9d6290c 0x9d66fb7 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:30:51.664 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x1006bc /lib64/libasan.so.8+0xc6619 /lib64/libasan.so.8+0x151b7 0x112781c 0xf2fda3 0x9c70283 0x9c7074d 0x9c709c0 0x9c726e7 0x9c72c34 0x9c72e33 0x9d58155 0x9d588b0 0x9d62b3e 0x9d66fb7 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:30:51.811 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f 0x9c70b93 0x9c71725 0x9c71c23 0x9c72253 0x9c723bd 0x9d5272d 0x9d65530 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:30:51.811 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc65 0xe1e15b 0x9bc2044 0x9bc2188 0x9bc226d 0x9bc24c9 0x9bc25bc 0x9bc2698 0x9bc2914 0x9bc2ebb 0x9bc32bf 0x9bc357c 0x9bc43c4 0x9d3b018 0x9d3b111 0x7bb2246 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:30:51.811 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f 0x9c7efa7 0x9d45227 0x9d453ef 0x9d4551a 0x9d45a44 0x9d622cf 0x9d66fb7 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:30:53.985 INFO:teuthology.orchestra.run.trial050.stderr:----------------------------------------------------- 2026-02-12T12:30:53.986 INFO:teuthology.orchestra.run.trial050.stderr:Suppressions used: 2026-02-12T12:30:53.986 INFO:teuthology.orchestra.run.trial050.stderr: count bytes template 2026-02-12T12:30:53.986 INFO:teuthology.orchestra.run.trial050.stderr: 1 8 MallocExtension::Register 2026-02-12T12:30:53.986 INFO:teuthology.orchestra.run.trial050.stderr:----------------------------------------------------- 2026-02-12T12:30:53.986 INFO:teuthology.orchestra.run.trial050.stderr: 2026-02-12T12:30:54.016 DEBUG:teuthology.orchestra.run.trial050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-12T12:30:54.041 DEBUG:teuthology.orchestra.run.trial050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-12T12:30:54.191 INFO:teuthology.orchestra.run.trial050.stderr:INFO  osd - OSD::main: parsing early config 2026-02-12T12:30:54.244 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:54,243 seastar - Reactor backend: linux-aio 2026-02-12T12:30:54.262 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:54,261 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T12:30:54.262 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:54,261 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T12:30:54.264 INFO:teuthology.orchestra.run.trial050.stderr:==18238==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T12:30:54.319 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:54,318 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-12T12:30:54.319 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:54,318 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-12T12:30:54.319 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:54,318 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-12T12:30:54.320 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:54,318 [shard 0:main] osd - Configure option_name crimson_reactor_backend with value : linux-aio 2026-02-12T12:30:54.320 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:54,318 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-12T12:30:54.320 INFO:teuthology.orchestra.run.trial050.stderr:DEBUG 2026-02-12 12:30:54,318 [shard 0:main] seastar - reactor::drain 2026-02-12T12:30:56.407 INFO:teuthology.orchestra.run.trial050.stderr:----------------------------------------------------- 2026-02-12T12:30:56.407 INFO:teuthology.orchestra.run.trial050.stderr:Suppressions used: 2026-02-12T12:30:56.407 INFO:teuthology.orchestra.run.trial050.stderr: count bytes template 2026-02-12T12:30:56.407 INFO:teuthology.orchestra.run.trial050.stderr: 1 8 MallocExtension::Register 2026-02-12T12:30:56.407 INFO:teuthology.orchestra.run.trial050.stderr:----------------------------------------------------- 2026-02-12T12:30:56.407 INFO:teuthology.orchestra.run.trial050.stderr: 2026-02-12T12:30:56.426 INFO:teuthology.orchestra.run.trial050.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-12T12:30:56.426 INFO:teuthology.orchestra.run.trial050.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-12T12:30:56.472 INFO:teuthology.orchestra.run.trial050.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-12T12:30:56.472 INFO:teuthology.orchestra.run.trial050.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-12T12:30:56.479 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:56,478 seastar - Reactor backend: linux-aio 2026-02-12T12:30:56.492 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:56,491 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T12:30:56.492 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:56,491 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T12:30:56.496 INFO:teuthology.orchestra.run.trial050.stderr:==18236==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T12:30:56.496 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:56,495 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-12T12:30:56.558 INFO:teuthology.orchestra.run.trial050.stderr:DEBUG 2026-02-12 12:30:56,557 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-12T12:30:56.638 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f 0x1092c44 0xf2fdae 0x9c70450 0x9c7074d 0x9c709c0 0x9c726e7 0x9c72c34 0x9c72e33 0x9d58155 0x9d588b0 0x9d62b3e 0x9d66fb7 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:30:56.638 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7fc 0x9b44d0b 0x9bc24a6 0x9bc25bc 0x9bc2698 0x9bc2d54 0x9bc29cd 0x9bc2ebb 0x9bc32bf 0x9bc357c 0x9d65185 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:30:56.638 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f 0x9c26b0a 0x9d3ad37 0x9d67224 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:30:56.638 INFO:teuthology.orchestra.run.trial050.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff834c5e91 0xffffffff831f91f3 0xffffffff831f9804 0xffffffff831fb799 0xffffffff831fccc1 0xffffffff83224923 0xffffffff83224ae5 0xffffffff831cb89e 0xffffffff831d3b7b 0xffffffff831d3fdb 0xffffffff82e910d2 0xffffffff83b4d451 0xffffffff83c00ba2 2026-02-12T12:30:56.784 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15885 0x9c73cd2 0x9c748ad 0x9d5e7f5 0x9d68edb 0x78a0be9 0x7bb2209 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:30:56.785 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6639 /lib64/libasan.so.8+0x157e7 0x1092cf7 0xf2fdae 0x9c70450 0x9c7074d 0x9c709c0 0x9c726e7 0x9d58641 0x9d588b0 0x9d62b3e 0x9d66fb7 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:30:56.785 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb13e7 0xb6a317 0x1092c80 0xf2fdae 0x9c70283 0x9c7074d 0x9c709c0 0x9c70c76 0x9c71725 0x9c71c23 0x9c72253 0x9c723bd 0x9d5272d 0x9d65530 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:30:58.965 INFO:teuthology.orchestra.run.trial050.stderr:----------------------------------------------------- 2026-02-12T12:30:58.965 INFO:teuthology.orchestra.run.trial050.stderr:Suppressions used: 2026-02-12T12:30:58.965 INFO:teuthology.orchestra.run.trial050.stderr: count bytes template 2026-02-12T12:30:58.965 INFO:teuthology.orchestra.run.trial050.stderr: 1 8 MallocExtension::Register 2026-02-12T12:30:58.965 INFO:teuthology.orchestra.run.trial050.stderr:----------------------------------------------------- 2026-02-12T12:30:58.965 INFO:teuthology.orchestra.run.trial050.stderr: 2026-02-12T12:30:58.996 DEBUG:teuthology.orchestra.run.trial050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-12T12:30:59.021 DEBUG:teuthology.orchestra.run.trial050:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-12T12:30:59.170 INFO:teuthology.orchestra.run.trial050.stderr:INFO  osd - OSD::main: parsing early config 2026-02-12T12:30:59.226 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:59,225 seastar - Reactor backend: linux-aio 2026-02-12T12:30:59.241 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:59,240 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T12:30:59.241 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:59,240 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T12:30:59.243 INFO:teuthology.orchestra.run.trial050.stderr:==18303==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T12:30:59.298 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:59,297 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-12T12:30:59.299 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:59,297 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-12T12:30:59.299 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:59,297 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-12T12:30:59.299 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:59,297 [shard 0:main] osd - Configure option_name crimson_reactor_backend with value : linux-aio 2026-02-12T12:30:59.299 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:30:59,297 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-12T12:30:59.299 INFO:teuthology.orchestra.run.trial050.stderr:DEBUG 2026-02-12 12:30:59,297 [shard 0:main] seastar - reactor::drain 2026-02-12T12:31:01.400 INFO:teuthology.orchestra.run.trial050.stderr:----------------------------------------------------- 2026-02-12T12:31:01.400 INFO:teuthology.orchestra.run.trial050.stderr:Suppressions used: 2026-02-12T12:31:01.400 INFO:teuthology.orchestra.run.trial050.stderr: count bytes template 2026-02-12T12:31:01.401 INFO:teuthology.orchestra.run.trial050.stderr: 1 8 MallocExtension::Register 2026-02-12T12:31:01.401 INFO:teuthology.orchestra.run.trial050.stderr:----------------------------------------------------- 2026-02-12T12:31:01.401 INFO:teuthology.orchestra.run.trial050.stderr: 2026-02-12T12:31:01.420 INFO:teuthology.orchestra.run.trial050.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-12T12:31:01.420 INFO:teuthology.orchestra.run.trial050.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-12T12:31:01.462 INFO:teuthology.orchestra.run.trial050.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-12T12:31:01.462 INFO:teuthology.orchestra.run.trial050.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-12T12:31:01.469 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:31:01,469 seastar - Reactor backend: linux-aio 2026-02-12T12:31:01.480 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:31:01,480 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T12:31:01.480 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:31:01,480 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T12:31:01.485 INFO:teuthology.orchestra.run.trial050.stderr:==18301==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T12:31:01.485 INFO:teuthology.orchestra.run.trial050.stderr:INFO 2026-02-12 12:31:01,484 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-12T12:31:01.543 INFO:teuthology.orchestra.run.trial050.stderr:DEBUG 2026-02-12 12:31:01,542 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-12T12:31:01.622 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7fc 0x9d3b001 0x9d3b111 0x7bb2246 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:31:01.622 INFO:teuthology.orchestra.run.trial050.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff8301f953 0xffffffff8301fb70 0xffffffff82e9525e 0xffffffff831e6239 0xffffffff831c97a5 0xffffffff831d3b7b 0xffffffff831d3fdb 0xffffffff82e910d2 0xffffffff83b4d451 0xffffffff83c00ba2 2026-02-12T12:31:01.622 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1529f 0x112781c 0xf2fda3 0x9c70450 0x9c7074d 0x9c709c0 0x9c70c76 0x9c71238 0x9c796df 0x9c79dbf 0x9d6290c 0x9d66fb7 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:31:01.622 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x129858 /lib64/libasan.so.8+0xc6631 /lib64/libasan.so.8+0x157e7 0x9c70a14 0x9c726e7 0x9d518b2 0x9d5202b 0x9d52516 0x9d528a9 0x9d65530 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:31:01.768 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151d5 0x9c2c85d 0x9d58ce0 0x9d5e3d0 0x9d5ea0e 0x9d68edb 0x78a0be9 0x7bb2209 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:31:01.768 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15885 0x1092cf7 0xf2fdae 0x9c70450 0x9c7074d 0x9c709c0 0x9c70c76 0x9c71725 0x9c71c23 0x9c72253 0x9c723bd 0x9d5272d 0x9d65530 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:31:01.768 INFO:teuthology.orchestra.run.trial050.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f 0xcb89fd 0x1092c72 0xf2fdae 0x9c74caa 0x9c755e9 0x9d615b9 0x9d624d9 0x9d66fb7 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T12:31:03.953 INFO:teuthology.orchestra.run.trial050.stderr:----------------------------------------------------- 2026-02-12T12:31:03.954 INFO:teuthology.orchestra.run.trial050.stderr:Suppressions used: 2026-02-12T12:31:03.954 INFO:teuthology.orchestra.run.trial050.stderr: count bytes template 2026-02-12T12:31:03.954 INFO:teuthology.orchestra.run.trial050.stderr: 1 8 MallocExtension::Register 2026-02-12T12:31:03.954 INFO:teuthology.orchestra.run.trial050.stderr:----------------------------------------------------- 2026-02-12T12:31:03.954 INFO:teuthology.orchestra.run.trial050.stderr: 2026-02-12T12:31:03.984 DEBUG:teuthology.orchestra.run.trial050:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-12T12:31:04.008 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-12T12:31:04.008 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:31:04.008 DEBUG:teuthology.orchestra.run.trial050:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-12T12:31:04.071 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:31:04.071 DEBUG:teuthology.orchestra.run.trial050:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-12T12:31:04.133 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:31:04.133 DEBUG:teuthology.orchestra.run.trial050:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-12T12:31:04.196 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:31:04.196 DEBUG:teuthology.orchestra.run.trial050:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-12T12:31:04.259 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:31:04.259 DEBUG:teuthology.orchestra.run.trial050:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-12T12:31:04.314 INFO:tasks.ceph:Adding keys to all mons... 2026-02-12T12:31:04.314 DEBUG:teuthology.orchestra.run.trial050:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-12T12:31:04.377 INFO:teuthology.orchestra.run.trial050.stdout:[mgr.x] 2026-02-12T12:31:04.377 INFO:teuthology.orchestra.run.trial050.stdout: key = AQD2x41p+8mbFxAALhypNAtcqZOue51jIgg2Ow== 2026-02-12T12:31:04.377 INFO:teuthology.orchestra.run.trial050.stdout:[osd.0] 2026-02-12T12:31:04.377 INFO:teuthology.orchestra.run.trial050.stdout: key = AQD7x41pU0IDIxAAb710EjSQeFw0dlrNy7YXsQ== 2026-02-12T12:31:04.377 INFO:teuthology.orchestra.run.trial050.stdout:[osd.1] 2026-02-12T12:31:04.377 INFO:teuthology.orchestra.run.trial050.stdout: key = AQAAyI1pwuR1IRAA2VcqxSMrQjFnh7l0psoLXg== 2026-02-12T12:31:04.377 INFO:teuthology.orchestra.run.trial050.stdout:[osd.2] 2026-02-12T12:31:04.377 INFO:teuthology.orchestra.run.trial050.stdout: key = AQAFyI1pkGiNIBAApl66wGNEq4ciZ525YVbhvg== 2026-02-12T12:31:04.378 INFO:teuthology.orchestra.run.trial050.stdout:[client.0] 2026-02-12T12:31:04.378 INFO:teuthology.orchestra.run.trial050.stdout: key = AQD2x41pJ9hMGhAAcVtByB/8mxGxjNlSPys4zA== 2026-02-12T12:31:04.379 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-12T12:31:04.466 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-12T12:31:04.514 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-12T12:31:04.563 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-12T12:31:04.612 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-12T12:31:04.662 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-12T12:31:04.662 DEBUG:teuthology.orchestra.run.trial050:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-12T12:31:04.686 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-12T12:31:04.771 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:04.769+0000 7f2d7c038c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:04.772 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:04.769+0000 7f2d7c038c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:04.783 DEBUG:teuthology.orchestra.run.trial050:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-12T12:31:04.808 DEBUG:teuthology.orchestra.run.trial050:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-12T12:31:04.864 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-12T12:31:04.864 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-12T12:31:04.865 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-12T12:31:04.906 INFO:tasks.ceph.mon.a:Started 2026-02-12T12:31:04.907 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-12T12:31:04.907 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-12T12:31:04.907 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-12T12:31:04.909 INFO:tasks.ceph.mgr.x:Started 2026-02-12T12:31:04.910 DEBUG:tasks.ceph:set 0 configs 2026-02-12T12:31:04.910 DEBUG:teuthology.orchestra.run.trial050:> sudo ceph --cluster ceph config dump 2026-02-12T12:31:04.968 INFO:tasks.ceph.mon.a.trial050.stderr:2026-02-12T12:31:04.966+0000 7fcc3e8dbc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:04.969 INFO:tasks.ceph.mon.a.trial050.stderr:2026-02-12T12:31:04.966+0000 7fcc3e8dbc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:04.969 INFO:tasks.ceph.mon.a.trial050.stderr:2026-02-12T12:31:04.966+0000 7fcc3e8dbc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:04.975 INFO:tasks.ceph.mgr.x.trial050.stderr:2026-02-12T12:31:04.973+0000 7f5095a3bfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:04.985 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:04.981+0000 7f79cd2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:04.988 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:04.986+0000 7f79cd2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:05.021 INFO:teuthology.orchestra.run.trial050.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-12T12:31:05.029 INFO:tasks.ceph:Setting crush tunables to default 2026-02-12T12:31:05.030 DEBUG:teuthology.orchestra.run.trial050:> sudo ceph --cluster ceph osd crush tunables default 2026-02-12T12:31:05.104 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:05.098+0000 7f618ca51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:05.106 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:05.104+0000 7f618ca51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:05.140 INFO:teuthology.orchestra.run.trial050.stderr:adjusted tunables profile to default 2026-02-12T12:31:05.149 INFO:tasks.ceph:check_enable_crimson: True 2026-02-12T12:31:05.150 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-12T12:31:05.150 DEBUG:teuthology.orchestra.run.trial050:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-12T12:31:05.179 INFO:tasks.ceph.mgr.x.trial050.stderr:2026-02-12T12:31:05.177+0000 7f5095a3bfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:05.180 INFO:tasks.ceph.mgr.x.trial050.stderr:2026-02-12T12:31:05.177+0000 7f5095a3bfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:05.225 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:05.221+0000 7f9e0fc3d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:05.228 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:05.226+0000 7f9e0fc3d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:05.266 INFO:tasks.ceph.mon.a.trial050.stderr:2026-02-12T12:31:05.264+0000 7fcc35ff1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-12T12:31:05.266 INFO:tasks.ceph.mon.a.trial050.stderr:Please be aware that this feature is experimental, untested, 2026-02-12T12:31:05.266 INFO:tasks.ceph.mon.a.trial050.stderr:unsupported, and may result in data corruption, data loss, 2026-02-12T12:31:05.266 INFO:tasks.ceph.mon.a.trial050.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-12T12:31:05.266 INFO:tasks.ceph.mon.a.trial050.stderr:feature with important data. 2026-02-12T12:31:05.266 INFO:tasks.ceph.mon.a.trial050.stderr: 2026-02-12T12:31:06.027 INFO:tasks.ceph.mgr.x.trial050.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-12T12:31:06.027 INFO:tasks.ceph.mgr.x.trial050.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-12T12:31:06.028 INFO:tasks.ceph.mgr.x.trial050.stderr: from numpy import show_config as show_numpy_config 2026-02-12T12:31:06.150 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-12T12:31:06.151 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:31:06.151 DEBUG:teuthology.orchestra.run.trial050:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-12T12:31:06.214 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:31:06.214 DEBUG:teuthology.orchestra.run.trial050:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-12T12:31:06.278 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:31:06.278 DEBUG:teuthology.orchestra.run.trial050:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-12T12:31:06.343 DEBUG:teuthology.orchestra.run.trial050:> sudo ceph --cluster ceph osd new 899c6ef7-f201-4241-9cc6-0c7a1626b60a 0 2026-02-12T12:31:06.457 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:06.451+0000 7fa3e5fa4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:06.459 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:06.457+0000 7fa3e5fa4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:06.499 INFO:teuthology.orchestra.run.trial050.stdout:0 2026-02-12T12:31:06.508 DEBUG:teuthology.orchestra.run.trial050:> sudo ceph --cluster ceph osd new 0ca533cd-2c69-460f-af47-9ab5f082bdb6 1 2026-02-12T12:31:06.581 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:06.576+0000 7f5a87a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:06.583 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:06.581+0000 7f5a87a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:06.621 INFO:teuthology.orchestra.run.trial050.stdout:1 2026-02-12T12:31:06.629 DEBUG:teuthology.orchestra.run.trial050:> sudo ceph --cluster ceph osd new 3df821e7-05d0-461d-a880-6f185ac9c806 2 2026-02-12T12:31:06.703 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:06.697+0000 7efe8decb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:06.705 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:06.703+0000 7efe8decb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:06.743 INFO:teuthology.orchestra.run.trial050.stdout:2 2026-02-12T12:31:06.752 DEBUG:teuthology.orchestra.run.trial050:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-12T12:31:06.781 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-12T12:31:06.782 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-12T12:31:06.823 INFO:tasks.ceph.osd.0:Started 2026-02-12T12:31:06.824 DEBUG:teuthology.orchestra.run.trial050:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-12T12:31:06.852 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-12T12:31:06.852 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-12T12:31:06.894 INFO:tasks.ceph.osd.1:Started 2026-02-12T12:31:06.894 DEBUG:teuthology.orchestra.run.trial050:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-12T12:31:06.924 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-12T12:31:06.925 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-12T12:31:06.966 INFO:tasks.ceph.osd.2:Started 2026-02-12T12:31:06.967 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-12T12:31:07.046 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:07.042+0000 7f452d2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:07.048 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:07.046+0000 7f452d2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:07.084 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:31:07.084 INFO:teuthology.orchestra.run.trial050.stdout:{"epoch":6,"fsid":"2a7ffb28-5872-4384-aac5-3a45ba84b5fb","created":"2026-02-12T12:31:04.979733+0000","modified":"2026-02-12T12:31:06.741621+0000","last_up_change":"0.000000","last_in_change":"2026-02-12T12:31:06.741621+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"899c6ef7-f201-4241-9cc6-0c7a1626b60a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0ca533cd-2c69-460f-af47-9ab5f082bdb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3df821e7-05d0-461d-a880-6f185ac9c806","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T12:31:07.094 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-12T12:31:07.095 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-12T12:31:07.396 DEBUG:teuthology.orchestra.run.trial050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-12T12:31:07.467 INFO:teuthology.misc.health.trial050.stderr:2026-02-12T12:31:07.463+0000 7fc450a06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:07.470 INFO:teuthology.misc.health.trial050.stderr:2026-02-12T12:31:07.467+0000 7fc450a06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:07.503 INFO:teuthology.misc.health.trial050.stdout: 2026-02-12T12:31:07.503 INFO:teuthology.misc.health.trial050.stdout:{"epoch":6,"fsid":"2a7ffb28-5872-4384-aac5-3a45ba84b5fb","created":"2026-02-12T12:31:04.979733+0000","modified":"2026-02-12T12:31:06.741621+0000","last_up_change":"0.000000","last_in_change":"2026-02-12T12:31:06.741621+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"899c6ef7-f201-4241-9cc6-0c7a1626b60a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0ca533cd-2c69-460f-af47-9ab5f082bdb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3df821e7-05d0-461d-a880-6f185ac9c806","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T12:31:07.510 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-12T12:31:08.440 INFO:tasks.ceph.mgr.x.trial050.stderr:2026-02-12T12:31:08.438+0000 7f4fb065e640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-12T12:31:13.812 DEBUG:teuthology.orchestra.run.trial050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-12T12:31:13.882 INFO:teuthology.misc.health.trial050.stderr:2026-02-12T12:31:13.877+0000 7f1337d64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:13.884 INFO:teuthology.misc.health.trial050.stderr:2026-02-12T12:31:13.882+0000 7f1337d64640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:13.995 INFO:teuthology.misc.health.trial050.stdout: 2026-02-12T12:31:13.995 INFO:teuthology.misc.health.trial050.stdout:{"epoch":7,"fsid":"2a7ffb28-5872-4384-aac5-3a45ba84b5fb","created":"2026-02-12T12:31:04.979733+0000","modified":"2026-02-12T12:31:13.069331+0000","last_up_change":"0.000000","last_in_change":"2026-02-12T12:31:06.741621+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"899c6ef7-f201-4241-9cc6-0c7a1626b60a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"0ca533cd-2c69-460f-af47-9ab5f082bdb6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"3df821e7-05d0-461d-a880-6f185ac9c806","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T12:31:14.003 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-12T12:31:20.305 DEBUG:teuthology.orchestra.run.trial050:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-12T12:31:20.376 INFO:teuthology.misc.health.trial050.stderr:2026-02-12T12:31:20.373+0000 7f6f4136f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:20.378 INFO:teuthology.misc.health.trial050.stderr:2026-02-12T12:31:20.377+0000 7f6f4136f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:20.489 INFO:teuthology.misc.health.trial050.stdout: 2026-02-12T12:31:20.490 INFO:teuthology.misc.health.trial050.stdout:{"epoch":13,"fsid":"2a7ffb28-5872-4384-aac5-3a45ba84b5fb","created":"2026-02-12T12:31:04.979733+0000","modified":"2026-02-12T12:31:20.098082+0000","last_up_change":"2026-02-12T12:31:16.079863+0000","last_in_change":"2026-02-12T12:31:06.741621+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-12T12:31:17.451237+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"899c6ef7-f201-4241-9cc6-0c7a1626b60a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6803","nonce":42187792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6802","nonce":42187792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6809","nonce":42187792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6806","nonce":42187792}]},"public_addr":"10.20.193.50:6803/42187792","cluster_addr":"10.20.193.50:6802/42187792","heartbeat_back_addr":"10.20.193.50:6809/42187792","heartbeat_front_addr":"10.20.193.50:6806/42187792","state":["exists","up"]},{"osd":1,"uuid":"0ca533cd-2c69-460f-af47-9ab5f082bdb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6808","nonce":2033975479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6807","nonce":2033975479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6810","nonce":2033975479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6813","nonce":2033975479}]},"public_addr":"10.20.193.50:6808/2033975479","cluster_addr":"10.20.193.50:6807/2033975479","heartbeat_back_addr":"10.20.193.50:6810/2033975479","heartbeat_front_addr":"10.20.193.50:6813/2033975479","state":["exists","up"]},{"osd":2,"uuid":"3df821e7-05d0-461d-a880-6f185ac9c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6804","nonce":2156715747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6805","nonce":2156715747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6812","nonce":2156715747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6811","nonce":2156715747}]},"public_addr":"10.20.193.50:6804/2156715747","cluster_addr":"10.20.193.50:6805/2156715747","heartbeat_back_addr":"10.20.193.50:6812/2156715747","heartbeat_front_addr":"10.20.193.50:6811/2156715747","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T12:31:20.498 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-12T12:31:20.498 INFO:tasks.ceph:Creating RBD pool 2026-02-12T12:31:20.498 DEBUG:teuthology.orchestra.run.trial050:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-12T12:31:20.573 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:20.567+0000 7f8f8dfd3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:20.575 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:20.574+0000 7f8f8dfd3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:21.104 INFO:teuthology.orchestra.run.trial050.stderr:pool 'rbd' created 2026-02-12T12:31:21.120 DEBUG:teuthology.orchestra.run.trial050:> rbd --cluster ceph pool init rbd 2026-02-12T12:31:21.161 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:21.159+0000 7f0b4f0e4240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:21.161 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:21.160+0000 7f0b4f0e4240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:21.164 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:21.163+0000 7f0b4f0e4240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:24.148 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-12T12:31:24.148 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-12T12:31:24.149 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-12T12:31:24.266 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:24.262+0000 7fd419806640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:24.268 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:24.267+0000 7fd419806640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:24.379 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:31:24.389 INFO:teuthology.orchestra.run.trial050.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-12T12:31:24.390 INFO:tasks.ceph_manager:config epoch is 1 2026-02-12T12:31:24.390 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-12T12:31:24.390 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-12T12:31:24.390 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-12T12:31:24.467 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:24.463+0000 7f16d6c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:24.469 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:24.468+0000 7f16d6c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:24.610 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:31:24.619 INFO:teuthology.orchestra.run.trial050.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6800","nonce":3102251584},{"type":"v1","addr":"10.20.193.50:6801","nonce":3102251584}]},"active_addr":"10.20.193.50:6801/3102251584","active_change":"2026-02-12T12:31:07.433677+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.50:0","nonce":3253230093}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.50:0","nonce":3066773229}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.50:0","nonce":1737704441}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.50:0","nonce":593627368}]}]} 2026-02-12T12:31:24.623 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-12T12:31:24.624 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-12T12:31:24.624 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-12T12:31:24.701 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:24.697+0000 7ff1d2b8c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:24.703 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:24.702+0000 7ff1d2b8c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:24.814 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:31:24.815 INFO:teuthology.orchestra.run.trial050.stdout:{"epoch":17,"fsid":"2a7ffb28-5872-4384-aac5-3a45ba84b5fb","created":"2026-02-12T12:31:04.979733+0000","modified":"2026-02-12T12:31:24.115735+0000","last_up_change":"2026-02-12T12:31:16.079863+0000","last_in_change":"2026-02-12T12:31:06.741621+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-12T12:31:17.451237+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-12T12:31:20.690992+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"899c6ef7-f201-4241-9cc6-0c7a1626b60a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6803","nonce":42187792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6802","nonce":42187792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6809","nonce":42187792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6806","nonce":42187792}]},"public_addr":"10.20.193.50:6803/42187792","cluster_addr":"10.20.193.50:6802/42187792","heartbeat_back_addr":"10.20.193.50:6809/42187792","heartbeat_front_addr":"10.20.193.50:6806/42187792","state":["exists","up"]},{"osd":1,"uuid":"0ca533cd-2c69-460f-af47-9ab5f082bdb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6808","nonce":2033975479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6807","nonce":2033975479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6810","nonce":2033975479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6813","nonce":2033975479}]},"public_addr":"10.20.193.50:6808/2033975479","cluster_addr":"10.20.193.50:6807/2033975479","heartbeat_back_addr":"10.20.193.50:6810/2033975479","heartbeat_front_addr":"10.20.193.50:6813/2033975479","state":["exists","up"]},{"osd":2,"uuid":"3df821e7-05d0-461d-a880-6f185ac9c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6804","nonce":2156715747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6805","nonce":2156715747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6812","nonce":2156715747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6811","nonce":2156715747}]},"public_addr":"10.20.193.50:6804/2156715747","cluster_addr":"10.20.193.50:6805/2156715747","heartbeat_back_addr":"10.20.193.50:6812/2156715747","heartbeat_front_addr":"10.20.193.50:6811/2156715747","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T12:31:24.824 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-12T12:31:24.824 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-12T12:31:24.901 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:24.898+0000 7fa5d4251640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:24.904 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:24.903+0000 7fa5d4251640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.017 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:31:25.018 INFO:teuthology.orchestra.run.trial050.stdout:{"epoch":17,"fsid":"2a7ffb28-5872-4384-aac5-3a45ba84b5fb","created":"2026-02-12T12:31:04.979733+0000","modified":"2026-02-12T12:31:24.115735+0000","last_up_change":"2026-02-12T12:31:16.079863+0000","last_in_change":"2026-02-12T12:31:06.741621+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-12T12:31:17.451237+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-12T12:31:20.690992+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"899c6ef7-f201-4241-9cc6-0c7a1626b60a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6803","nonce":42187792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6802","nonce":42187792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6809","nonce":42187792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6806","nonce":42187792}]},"public_addr":"10.20.193.50:6803/42187792","cluster_addr":"10.20.193.50:6802/42187792","heartbeat_back_addr":"10.20.193.50:6809/42187792","heartbeat_front_addr":"10.20.193.50:6806/42187792","state":["exists","up"]},{"osd":1,"uuid":"0ca533cd-2c69-460f-af47-9ab5f082bdb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6808","nonce":2033975479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6807","nonce":2033975479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6810","nonce":2033975479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6813","nonce":2033975479}]},"public_addr":"10.20.193.50:6808/2033975479","cluster_addr":"10.20.193.50:6807/2033975479","heartbeat_back_addr":"10.20.193.50:6810/2033975479","heartbeat_front_addr":"10.20.193.50:6813/2033975479","state":["exists","up"]},{"osd":2,"uuid":"3df821e7-05d0-461d-a880-6f185ac9c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6804","nonce":2156715747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6805","nonce":2156715747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6812","nonce":2156715747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6811","nonce":2156715747}]},"public_addr":"10.20.193.50:6804/2156715747","cluster_addr":"10.20.193.50:6805/2156715747","heartbeat_back_addr":"10.20.193.50:6812/2156715747","heartbeat_front_addr":"10.20.193.50:6811/2156715747","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T12:31:25.029 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-12T12:31:25.030 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-12T12:31:25.030 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-12T12:31:25.113 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.109+0000 7fae9f9cc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.114 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.110+0000 7f758af0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.114 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.111+0000 7f74bc61a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.115 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.114+0000 7fae9f9cc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.117 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.116+0000 7f758af0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.117 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.116+0000 7f74bc61a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.134 INFO:teuthology.orchestra.run.trial050.stdout:38654705674 2026-02-12T12:31:25.134 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-12T12:31:25.136 INFO:teuthology.orchestra.run.trial050.stdout:42949672969 2026-02-12T12:31:25.136 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-12T12:31:25.137 INFO:teuthology.orchestra.run.trial050.stdout:42949672969 2026-02-12T12:31:25.138 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-12T12:31:25.214 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.210+0000 7fa55f2b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.217 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.216+0000 7fa55f2b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.256 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.252+0000 7fcf56e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.258 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.254+0000 7f0e89c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.259 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.258+0000 7fcf56e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.261 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.260+0000 7f0e89c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.331 INFO:teuthology.orchestra.run.trial050.stdout:38654705674 2026-02-12T12:31:25.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705674 for osd.0 2026-02-12T12:31:25.344 DEBUG:teuthology.parallel:result is None 2026-02-12T12:31:25.370 INFO:teuthology.orchestra.run.trial050.stdout:42949672969 2026-02-12T12:31:25.377 INFO:teuthology.orchestra.run.trial050.stdout:42949672969 2026-02-12T12:31:25.382 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672969 for osd.2 2026-02-12T12:31:25.382 DEBUG:teuthology.parallel:result is None 2026-02-12T12:31:25.388 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672969 for osd.1 2026-02-12T12:31:25.388 DEBUG:teuthology.parallel:result is None 2026-02-12T12:31:25.388 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-12T12:31:25.389 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-12T12:31:25.506 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.502+0000 7fe085964640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.508 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.507+0000 7fe085964640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.617 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:31:25.618 INFO:teuthology.orchestra.run.trial050.stderr:dumped all 2026-02-12T12:31:25.627 INFO:teuthology.orchestra.run.trial050.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-12T12:31:25.438022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62917052,"kb_used_data":62917052,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94369300,"statfs":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.309473"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.130514+0000","last_change":"2026-02-12T12:31:24.133533+0000","last_active":"2026-02-12T12:31:24.130514+0000","last_peered":"2026-02-12T12:31:24.130514+0000","last_clean":"2026-02-12T12:31:24.130514+0000","last_became_active":"2026-02-12T12:31:22.151365+0000","last_became_peered":"2026-02-12T12:31:22.151365+0000","last_unstale":"2026-02-12T12:31:24.130514+0000","last_undegraded":"2026-02-12T12:31:24.130514+0000","last_fullsized":"2026-02-12T12:31:24.130514+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.131669+0000","last_change":"2026-02-12T12:31:24.133965+0000","last_active":"2026-02-12T12:31:24.131669+0000","last_peered":"2026-02-12T12:31:24.131669+0000","last_clean":"2026-02-12T12:31:24.131669+0000","last_became_active":"2026-02-12T12:31:22.149755+0000","last_became_peered":"2026-02-12T12:31:22.149755+0000","last_unstale":"2026-02-12T12:31:24.131669+0000","last_undegraded":"2026-02-12T12:31:24.131669+0000","last_fullsized":"2026-02-12T12:31:24.131669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.131600+0000","last_change":"2026-02-12T12:31:24.135573+0000","last_active":"2026-02-12T12:31:24.131600+0000","last_peered":"2026-02-12T12:31:24.131600+0000","last_clean":"2026-02-12T12:31:24.131600+0000","last_became_active":"2026-02-12T12:31:22.158500+0000","last_became_peered":"2026-02-12T12:31:22.158500+0000","last_unstale":"2026-02-12T12:31:24.131600+0000","last_undegraded":"2026-02-12T12:31:24.131600+0000","last_fullsized":"2026-02-12T12:31:24.131600+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.131598+0000","last_change":"2026-02-12T12:31:24.134224+0000","last_active":"2026-02-12T12:31:24.131598+0000","last_peered":"2026-02-12T12:31:24.131598+0000","last_clean":"2026-02-12T12:31:24.131598+0000","last_became_active":"2026-02-12T12:31:22.150491+0000","last_became_peered":"2026-02-12T12:31:22.150491+0000","last_unstale":"2026-02-12T12:31:24.131598+0000","last_undegraded":"2026-02-12T12:31:24.131598+0000","last_fullsized":"2026-02-12T12:31:24.131598+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.133829+0000","last_change":"2026-02-12T12:31:24.133829+0000","last_active":"2026-02-12T12:31:24.133829+0000","last_peered":"2026-02-12T12:31:24.133829+0000","last_clean":"2026-02-12T12:31:24.133829+0000","last_became_active":"2026-02-12T12:31:22.170355+0000","last_became_peered":"2026-02-12T12:31:22.170355+0000","last_unstale":"2026-02-12T12:31:24.133829+0000","last_undegraded":"2026-02-12T12:31:24.133829+0000","last_fullsized":"2026-02-12T12:31:24.133829+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.131733+0000","last_change":"2026-02-12T12:31:24.134494+0000","last_active":"2026-02-12T12:31:24.131733+0000","last_peered":"2026-02-12T12:31:24.131733+0000","last_clean":"2026-02-12T12:31:24.131733+0000","last_became_active":"2026-02-12T12:31:22.170811+0000","last_became_peered":"2026-02-12T12:31:22.170811+0000","last_unstale":"2026-02-12T12:31:24.131733+0000","last_undegraded":"2026-02-12T12:31:24.131733+0000","last_fullsized":"2026-02-12T12:31:24.131733+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.129610+0000","last_change":"2026-02-12T12:31:24.133038+0000","last_active":"2026-02-12T12:31:24.129610+0000","last_peered":"2026-02-12T12:31:24.129610+0000","last_clean":"2026-02-12T12:31:24.129610+0000","last_became_active":"2026-02-12T12:31:22.166160+0000","last_became_peered":"2026-02-12T12:31:22.166160+0000","last_unstale":"2026-02-12T12:31:24.129610+0000","last_undegraded":"2026-02-12T12:31:24.129610+0000","last_fullsized":"2026-02-12T12:31:24.129610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.131830+0000","last_change":"2026-02-12T12:31:24.135685+0000","last_active":"2026-02-12T12:31:24.131830+0000","last_peered":"2026-02-12T12:31:24.131830+0000","last_clean":"2026-02-12T12:31:24.131830+0000","last_became_active":"2026-02-12T12:31:22.149374+0000","last_became_peered":"2026-02-12T12:31:22.149374+0000","last_unstale":"2026-02-12T12:31:24.131830+0000","last_undegraded":"2026-02-12T12:31:24.131830+0000","last_fullsized":"2026-02-12T12:31:24.131830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.130259+0000","last_change":"2026-02-12T12:31:19.128575+0000","last_active":"2026-02-12T12:31:24.130259+0000","last_peered":"2026-02-12T12:31:24.130259+0000","last_clean":"2026-02-12T12:31:24.130259+0000","last_became_active":"2026-02-12T12:31:19.118740+0000","last_became_peered":"2026-02-12T12:31:19.118740+0000","last_unstale":"2026-02-12T12:31:24.130259+0000","last_undegraded":"2026-02-12T12:31:24.130259+0000","last_fullsized":"2026-02-12T12:31:24.130259+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:18.085509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:18.085509+0000","last_clean_scrub_stamp":"2026-02-12T12:31:18.085509+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971820,"kb_used_data":20971820,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456964,"statfs":{"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972652,"kb_used_data":20972652,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456132,"statfs":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972580,"kb_used_data":20972580,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456204,"statfs":{"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-12T12:31:25.628 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-12T12:31:25.705 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.701+0000 7f445703d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.707 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.706+0000 7f445703d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.821 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:31:25.822 INFO:teuthology.orchestra.run.trial050.stderr:dumped all 2026-02-12T12:31:25.830 INFO:teuthology.orchestra.run.trial050.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-12T12:31:25.438022+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62917052,"kb_used_data":62917052,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94369300,"statfs":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.309473"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.130514+0000","last_change":"2026-02-12T12:31:24.133533+0000","last_active":"2026-02-12T12:31:24.130514+0000","last_peered":"2026-02-12T12:31:24.130514+0000","last_clean":"2026-02-12T12:31:24.130514+0000","last_became_active":"2026-02-12T12:31:22.151365+0000","last_became_peered":"2026-02-12T12:31:22.151365+0000","last_unstale":"2026-02-12T12:31:24.130514+0000","last_undegraded":"2026-02-12T12:31:24.130514+0000","last_fullsized":"2026-02-12T12:31:24.130514+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.131669+0000","last_change":"2026-02-12T12:31:24.133965+0000","last_active":"2026-02-12T12:31:24.131669+0000","last_peered":"2026-02-12T12:31:24.131669+0000","last_clean":"2026-02-12T12:31:24.131669+0000","last_became_active":"2026-02-12T12:31:22.149755+0000","last_became_peered":"2026-02-12T12:31:22.149755+0000","last_unstale":"2026-02-12T12:31:24.131669+0000","last_undegraded":"2026-02-12T12:31:24.131669+0000","last_fullsized":"2026-02-12T12:31:24.131669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.131600+0000","last_change":"2026-02-12T12:31:24.135573+0000","last_active":"2026-02-12T12:31:24.131600+0000","last_peered":"2026-02-12T12:31:24.131600+0000","last_clean":"2026-02-12T12:31:24.131600+0000","last_became_active":"2026-02-12T12:31:22.158500+0000","last_became_peered":"2026-02-12T12:31:22.158500+0000","last_unstale":"2026-02-12T12:31:24.131600+0000","last_undegraded":"2026-02-12T12:31:24.131600+0000","last_fullsized":"2026-02-12T12:31:24.131600+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.131598+0000","last_change":"2026-02-12T12:31:24.134224+0000","last_active":"2026-02-12T12:31:24.131598+0000","last_peered":"2026-02-12T12:31:24.131598+0000","last_clean":"2026-02-12T12:31:24.131598+0000","last_became_active":"2026-02-12T12:31:22.150491+0000","last_became_peered":"2026-02-12T12:31:22.150491+0000","last_unstale":"2026-02-12T12:31:24.131598+0000","last_undegraded":"2026-02-12T12:31:24.131598+0000","last_fullsized":"2026-02-12T12:31:24.131598+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.133829+0000","last_change":"2026-02-12T12:31:24.133829+0000","last_active":"2026-02-12T12:31:24.133829+0000","last_peered":"2026-02-12T12:31:24.133829+0000","last_clean":"2026-02-12T12:31:24.133829+0000","last_became_active":"2026-02-12T12:31:22.170355+0000","last_became_peered":"2026-02-12T12:31:22.170355+0000","last_unstale":"2026-02-12T12:31:24.133829+0000","last_undegraded":"2026-02-12T12:31:24.133829+0000","last_fullsized":"2026-02-12T12:31:24.133829+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.131733+0000","last_change":"2026-02-12T12:31:24.134494+0000","last_active":"2026-02-12T12:31:24.131733+0000","last_peered":"2026-02-12T12:31:24.131733+0000","last_clean":"2026-02-12T12:31:24.131733+0000","last_became_active":"2026-02-12T12:31:22.170811+0000","last_became_peered":"2026-02-12T12:31:22.170811+0000","last_unstale":"2026-02-12T12:31:24.131733+0000","last_undegraded":"2026-02-12T12:31:24.131733+0000","last_fullsized":"2026-02-12T12:31:24.131733+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.129610+0000","last_change":"2026-02-12T12:31:24.133038+0000","last_active":"2026-02-12T12:31:24.129610+0000","last_peered":"2026-02-12T12:31:24.129610+0000","last_clean":"2026-02-12T12:31:24.129610+0000","last_became_active":"2026-02-12T12:31:22.166160+0000","last_became_peered":"2026-02-12T12:31:22.166160+0000","last_unstale":"2026-02-12T12:31:24.129610+0000","last_undegraded":"2026-02-12T12:31:24.129610+0000","last_fullsized":"2026-02-12T12:31:24.129610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.131830+0000","last_change":"2026-02-12T12:31:24.135685+0000","last_active":"2026-02-12T12:31:24.131830+0000","last_peered":"2026-02-12T12:31:24.131830+0000","last_clean":"2026-02-12T12:31:24.131830+0000","last_became_active":"2026-02-12T12:31:22.149374+0000","last_became_peered":"2026-02-12T12:31:22.149374+0000","last_unstale":"2026-02-12T12:31:24.131830+0000","last_undegraded":"2026-02-12T12:31:24.131830+0000","last_fullsized":"2026-02-12T12:31:24.131830+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T12:31:24.130259+0000","last_change":"2026-02-12T12:31:19.128575+0000","last_active":"2026-02-12T12:31:24.130259+0000","last_peered":"2026-02-12T12:31:24.130259+0000","last_clean":"2026-02-12T12:31:24.130259+0000","last_became_active":"2026-02-12T12:31:19.118740+0000","last_became_peered":"2026-02-12T12:31:19.118740+0000","last_unstale":"2026-02-12T12:31:24.130259+0000","last_undegraded":"2026-02-12T12:31:24.130259+0000","last_fullsized":"2026-02-12T12:31:24.130259+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:18.085509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:18.085509+0000","last_clean_scrub_stamp":"2026-02-12T12:31:18.085509+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971820,"kb_used_data":20971820,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456964,"statfs":{"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972652,"kb_used_data":20972652,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456132,"statfs":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972580,"kb_used_data":20972580,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456204,"statfs":{"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-12T12:31:25.832 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-12T12:31:25.832 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-12T12:31:25.832 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-12T12:31:25.832 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-12T12:31:25.909 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.906+0000 7ff82597b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:25.911 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:31:25.910+0000 7ff82597b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:31:26.049 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:31:26.049 INFO:teuthology.orchestra.run.trial050.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-12T12:31:26.058 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-12T12:31:26.059 INFO:teuthology.run_tasks:Running task workunit... 2026-02-12T12:31:26.085 INFO:tasks.workunit:Pulling workunits from ref 5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 2026-02-12T12:31:26.085 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-12T12:31:26.086 DEBUG:teuthology.orchestra.run.trial050:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-12T12:31:26.103 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T12:31:26.103 INFO:teuthology.orchestra.run.trial050.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-12T12:31:26.104 DEBUG:teuthology.orchestra.run.trial050:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-12T12:31:26.159 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-12T12:31:26.160 DEBUG:teuthology.orchestra.run.trial050:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-12T12:31:26.215 INFO:tasks.workunit:timeout=3h 2026-02-12T12:31:26.215 INFO:tasks.workunit:cleanup=True 2026-02-12T12:31:26.216 DEBUG:teuthology.orchestra.run.trial050:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/mohit84/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout 5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 2026-02-12T12:31:26.279 INFO:tasks.workunit.client.0.trial050.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-12T12:31:44.619 INFO:tasks.workunit.client.0.trial050.stderr:Note: switching to '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'. 2026-02-12T12:31:44.619 INFO:tasks.workunit.client.0.trial050.stderr: 2026-02-12T12:31:44.620 INFO:tasks.workunit.client.0.trial050.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-12T12:31:44.620 INFO:tasks.workunit.client.0.trial050.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-12T12:31:44.620 INFO:tasks.workunit.client.0.trial050.stderr:state without impacting any branches by switching back to a branch. 2026-02-12T12:31:44.620 INFO:tasks.workunit.client.0.trial050.stderr: 2026-02-12T12:31:44.620 INFO:tasks.workunit.client.0.trial050.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-12T12:31:44.620 INFO:tasks.workunit.client.0.trial050.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-12T12:31:44.620 INFO:tasks.workunit.client.0.trial050.stderr: 2026-02-12T12:31:44.620 INFO:tasks.workunit.client.0.trial050.stderr: git switch -c 2026-02-12T12:31:44.620 INFO:tasks.workunit.client.0.trial050.stderr: 2026-02-12T12:31:44.620 INFO:tasks.workunit.client.0.trial050.stderr:Or undo this operation with: 2026-02-12T12:31:44.620 INFO:tasks.workunit.client.0.trial050.stderr: 2026-02-12T12:31:44.621 INFO:tasks.workunit.client.0.trial050.stderr: git switch - 2026-02-12T12:31:44.621 INFO:tasks.workunit.client.0.trial050.stderr: 2026-02-12T12:31:44.621 INFO:tasks.workunit.client.0.trial050.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-12T12:31:44.621 INFO:tasks.workunit.client.0.trial050.stderr: 2026-02-12T12:31:44.621 INFO:tasks.workunit.client.0.trial050.stderr:HEAD is now at 5fdb7d8f131 crimson: Support io_uring as reactor backend option 2026-02-12T12:31:44.623 DEBUG:teuthology.orchestra.run.trial050:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-12T12:31:44.680 INFO:tasks.workunit.client.0.trial050.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-12T12:31:44.682 INFO:tasks.workunit.client.0.trial050.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-12T12:31:44.682 INFO:tasks.workunit.client.0.trial050.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-12T12:31:44.770 INFO:tasks.workunit.client.0.trial050.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-12T12:31:44.800 INFO:tasks.workunit.client.0.trial050.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-12T12:31:44.824 INFO:tasks.workunit.client.0.trial050.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-12T12:31:44.826 INFO:tasks.workunit.client.0.trial050.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-12T12:31:44.826 INFO:tasks.workunit.client.0.trial050.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-12T12:31:44.851 INFO:tasks.workunit.client.0.trial050.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-12T12:31:44.854 DEBUG:teuthology.orchestra.run.trial050:> set -ex 2026-02-12T12:31:44.854 DEBUG:teuthology.orchestra.run.trial050:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-12T12:31:44.911 INFO:tasks.workunit:Running workunits matching rbd/test_librbd_python.sh on client.0... 2026-02-12T12:31:44.914 INFO:tasks.workunit:Running workunit rbd/test_librbd_python.sh... 2026-02-12T12:31:44.914 DEBUG:teuthology.orchestra.run.trial050:workunit test rbd/test_librbd_python.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd_python.sh 2026-02-12T12:31:44.971 INFO:tasks.workunit.client.0.trial050.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd_python.sh 2026-02-12T12:31:44.972 INFO:tasks.workunit.client.0.trial050.stderr:+ relpath=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../../src/test/pybind 2026-02-12T12:31:44.973 INFO:tasks.workunit.client.0.trial050.stderr:+ '[' -n '' ']' 2026-02-12T12:31:44.973 INFO:tasks.workunit.client.0.trial050.stderr:+ python3 -m pytest -v /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../../src/test/pybind/test_rbd.py 2026-02-12T12:31:45.046 INFO:tasks.workunit.client.0.trial050.stdout:============================= test session starts ============================== 2026-02-12T12:31:45.047 INFO:tasks.workunit.client.0.trial050.stdout:platform linux -- Python 3.9.25, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3 2026-02-12T12:31:45.047 INFO:tasks.workunit.client.0.trial050.stdout:cachedir: .pytest_cache 2026-02-12T12:31:45.047 INFO:tasks.workunit.client.0.trial050.stdout:rootdir: /home/ubuntu/cephtest/clone.client.0/src/test/pybind, configfile: pytest.ini 2026-02-12T12:31:45.343 INFO:tasks.workunit.client.0.trial050.stdout:collecting ... collected 196 items 2026-02-12T12:31:45.344 INFO:tasks.workunit.client.0.trial050.stdout: 2026-02-12T12:31:49.194 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_version PASSED [ 0%] 2026-02-12T12:31:49.702 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_create PASSED [ 1%] 2026-02-12T12:31:54.188 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_create_defaults PASSED [ 1%] 2026-02-12T12:31:54.886 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_context_manager PASSED [ 2%] 2026-02-12T12:31:57.753 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_open_read_only PASSED [ 2%] 2026-02-12T12:31:58.917 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_open_dne PASSED [ 3%] 2026-02-12T12:32:00.098 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_open_readonly_dne PASSED [ 3%] 2026-02-12T12:32:00.099 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_open_by_id SKIPPED [ 4%] 2026-02-12T12:32:00.705 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_aio_open PASSED [ 4%] 2026-02-12T12:32:00.767 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_remove_dne PASSED [ 5%] 2026-02-12T12:32:00.776 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list_empty PASSED [ 5%] 2026-02-12T12:32:01.363 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list PASSED [ 6%] 2026-02-12T12:32:01.863 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_remove_with_progress PASSED [ 6%] 2026-02-12T12:32:02.519 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_remove_canceled PASSED [ 7%] 2026-02-12T12:32:03.015 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_remove_with_progress_except PASSED [ 7%] 2026-02-12T12:32:03.856 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_rename PASSED [ 8%] 2026-02-12T12:32:05.696 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_pool_metadata PASSED [ 8%] 2026-02-12T12:32:05.744 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_config_list PASSED [ 9%] 2026-02-12T12:32:05.780 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_pool_config_set_and_get_and_remove PASSED [ 9%] 2026-02-12T12:32:05.962 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_namespaces PASSED [ 10%] 2026-02-12T12:32:05.963 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_pool_stats SKIPPED [ 10%] 2026-02-12T12:32:05.963 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_features_to_string SKIPPED [ 11%] 2026-02-12T12:32:05.963 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_features_from_string SKIPPED [ 11%] 2026-02-12T12:32:06.548 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_update_features SKIPPED [ 12%] 2026-02-12T12:32:07.131 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_with_params SKIPPED [ 12%] 2026-02-12T12:32:07.710 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_id SKIPPED [ 13%] 2026-02-12T12:32:08.272 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_block_name_prefix PASSED [ 13%] 2026-02-12T12:32:08.867 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_data_pool_id PASSED [ 14%] 2026-02-12T12:32:09.436 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_timestamp PASSED [ 14%] 2026-02-12T12:32:10.007 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_access_timestamp PASSED [ 15%] 2026-02-12T12:32:10.615 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_modify_timestamp PASSED [ 15%] 2026-02-12T12:32:11.339 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_invalidate_cache PASSED [ 16%] 2026-02-12T12:32:11.914 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_stat PASSED [ 16%] 2026-02-12T12:32:12.516 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_flags PASSED [ 17%] 2026-02-12T12:32:13.149 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_image_auto_close PASSED [ 17%] 2026-02-12T12:32:13.724 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_use_after_close PASSED [ 18%] 2026-02-12T12:32:14.405 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write PASSED [ 18%] 2026-02-12T12:32:15.111 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write_with_fadvise_flags PASSED [ 19%] 2026-02-12T12:32:15.804 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write_zeroes PASSED [ 19%] 2026-02-12T12:32:16.496 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write_zeroes_thick_provision PASSED [ 20%] 2026-02-12T12:32:17.077 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_read PASSED [ 20%] 2026-02-12T12:32:17.670 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_read_with_fadvise_flags PASSED [ 21%] 2026-02-12T12:32:18.428 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_large_write PASSED [ 21%] 2026-02-12T12:32:19.002 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_large_read PASSED [ 22%] 2026-02-12T12:32:19.760 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write_read PASSED [ 22%] 2026-02-12T12:32:20.328 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_read_bad_offset PASSED [ 23%] 2026-02-12T12:32:20.987 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_resize PASSED [ 23%] 2026-02-12T12:32:21.664 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_resize_allow_shrink_False PASSED [ 24%] 2026-02-12T12:32:25.538 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_size PASSED [ 25%] 2026-02-12T12:32:26.340 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_resize_down PASSED [ 25%] 2026-02-12T12:32:27.227 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_resize_bytes PASSED [ 26%] 2026-02-12T12:32:28.745 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_copy PASSED [ 26%] 2026-02-12T12:32:30.278 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_copy2 PASSED [ 27%] 2026-02-12T12:32:30.853 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_copy3 SKIPPED [ 27%] 2026-02-12T12:32:36.158 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy PASSED [ 28%] 2026-02-12T12:32:36.756 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v1_to_v1 SKIPPED [ 28%] 2026-02-12T12:32:37.368 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v1_to_v2 SKIPPED [ 29%] 2026-02-12T12:32:37.963 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v2_to_v1 SKIPPED [ 29%] 2026-02-12T12:32:38.541 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v2_to_v2 SKIPPED [ 30%] 2026-02-12T12:32:39.127 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v1_flatten SKIPPED [ 30%] 2026-02-12T12:32:39.726 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v2_flatten SKIPPED [ 31%] 2026-02-12T12:32:41.658 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_snap PASSED [ 31%] 2026-02-12T12:32:43.639 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_snap_exists PASSED [ 32%] 2026-02-12T12:32:49.652 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_snap_flags PASSED [ 32%] 2026-02-12T12:32:53.636 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_list_snaps PASSED [ 33%] 2026-02-12T12:32:55.644 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_list_snaps_iterator_auto_close PASSED [ 33%] 2026-02-12T12:32:57.693 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_snap PASSED [ 34%] 2026-02-12T12:32:58.321 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_snap_not_found PASSED [ 34%] 2026-02-12T12:32:58.922 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_snap2 SKIPPED [ 35%] 2026-02-12T12:33:00.656 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_snap_by_id PASSED [ 35%] 2026-02-12T12:33:02.723 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_rename_snap PASSED [ 36%] 2026-02-12T12:33:03.314 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_protect_snap SKIPPED [ 36%] 2026-02-12T12:33:05.707 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_snap_exists PASSED [ 37%] 2026-02-12T12:33:07.723 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_snap_timestamp PASSED [ 37%] 2026-02-12T12:33:15.736 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_limit_snaps PASSED [ 38%] 2026-02-12T12:33:16.326 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_with_exclusive_lock SKIPPED [ 38%] 2026-02-12T12:33:18.721 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_with_snap PASSED [ 39%] 2026-02-12T12:33:19.615 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_with_watcher PASSED [ 39%] 2026-02-12T12:33:21.805 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_rollback_to_snap PASSED [ 40%] 2026-02-12T12:33:23.748 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_rollback_to_snap_sparse PASSED [ 40%] 2026-02-12T12:33:27.812 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_rollback_with_resize PASSED [ 41%] 2026-02-12T12:33:29.772 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap PASSED [ 41%] 2026-02-12T12:33:31.842 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_no_snap PASSED [ 42%] 2026-02-12T12:33:33.814 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap_by_id PASSED [ 42%] 2026-02-12T12:33:39.817 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_snap_get_name PASSED [ 43%] 2026-02-12T12:33:45.631 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_snap_get_id PASSED [ 43%] 2026-02-12T12:33:47.604 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap_sparse PASSED [ 44%] 2026-02-12T12:42:37.786 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_many_snaps PASSED [ 44%] 2026-02-12T12:42:39.849 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap_deleted PASSED [ 45%] 2026-02-12T12:42:43.842 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap_recreated PASSED [ 45%] 2026-02-12T12:42:44.546 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_lock_unlock PASSED [ 46%] 2026-02-12T12:42:46.105 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_list_lockers PASSED [ 46%] 2026-02-12T12:42:52.842 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_diff_iterate PASSED [ 47%] 2026-02-12T12:42:53.466 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_diff_iterate_from_trash_snap SKIPPED [ 47%] 2026-02-12T12:42:54.072 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_diff_iterate_exclude_parent SKIPPED [ 48%] 2026-02-12T12:42:54.690 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_read PASSED [ 48%] 2026-02-12T12:42:55.420 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_write PASSED [ 49%] 2026-02-12T12:42:56.173 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_discard PASSED [ 50%] 2026-02-12T12:42:56.904 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_write_zeroes PASSED [ 50%] 2026-02-12T12:42:57.568 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_flush PASSED [ 51%] 2026-02-12T12:43:04.637 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_metadata PASSED [ 51%] 2026-02-12T12:43:05.263 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_watchers_list PASSED [ 52%] 2026-02-12T12:43:06.126 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_config_list PASSED [ 52%] 2026-02-12T12:43:06.967 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_image_config_set_and_get_and_remove PASSED [ 53%] 2026-02-12T12:43:07.658 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_sparsify PASSED [ 53%] 2026-02-12T12:43:17.042 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_encryption_luks1 PASSED [ 54%] 2026-02-12T12:43:30.185 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_encryption_luks2 PASSED [ 54%] 2026-02-12T12:43:30.852 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImageId::test_read PASSED [ 55%] 2026-02-12T12:43:31.686 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImageId::test_write PASSED [ 55%] 2026-02-12T12:43:32.499 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImageId::test_resize PASSED [ 56%] 2026-02-12T12:43:32.500 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_with_params SKIPPED [ 56%] 2026-02-12T12:43:32.500 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_with_params2 SKIPPED [ 57%] 2026-02-12T12:43:32.500 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_with_params3 SKIPPED [ 57%] 2026-02-12T12:43:32.501 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_stripe_unit_and_count SKIPPED [ 58%] 2026-02-12T12:43:32.502 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_clone_format SKIPPED [ 58%] 2026-02-12T12:43:32.502 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_unprotected SKIPPED [ 59%] 2026-02-12T12:43:32.503 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_unprotect_with_children SKIPPED [ 59%] 2026-02-12T12:43:32.503 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_clone_by_snap_id SKIPPED [ 60%] 2026-02-12T12:43:32.504 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_stat SKIPPED [ 60%] 2026-02-12T12:43:32.504 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_resize_stat SKIPPED [ 61%] 2026-02-12T12:43:32.504 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_resize_io SKIPPED [ 61%] 2026-02-12T12:43:32.505 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_read SKIPPED [ 62%] 2026-02-12T12:43:32.505 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_write SKIPPED [ 62%] 2026-02-12T12:43:32.506 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_list_children SKIPPED [ 63%] 2026-02-12T12:43:32.506 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_errors SKIPPED [ 63%] 2026-02-12T12:43:32.507 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_basic SKIPPED [ 64%] 2026-02-12T12:43:32.507 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_smaller_order SKIPPED [ 64%] 2026-02-12T12:43:32.508 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_larger_order SKIPPED [ 65%] 2026-02-12T12:43:32.508 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_drops_cache SKIPPED [ 65%] 2026-02-12T12:43:32.508 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_multi_level SKIPPED [ 66%] 2026-02-12T12:43:32.509 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_with_progress SKIPPED [ 66%] 2026-02-12T12:43:32.509 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_resize_flatten_multi_level SKIPPED [ 67%] 2026-02-12T12:43:32.509 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_trash_snapshot SKIPPED [ 67%] 2026-02-12T12:43:32.510 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_encryption_luks1 SKIPPED [ 68%] 2026-02-12T12:43:32.511 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_encryption_luks2 SKIPPED [ 68%] 2026-02-12T12:43:32.511 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_ownership SKIPPED [ 69%] 2026-02-12T12:43:32.511 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_snapshot_leadership SKIPPED [ 69%] 2026-02-12T12:43:32.512 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_read_only_leadership SKIPPED [ 70%] 2026-02-12T12:43:32.512 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_flatten SKIPPED [ 70%] 2026-02-12T12:43:32.512 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_resize SKIPPED [ 71%] 2026-02-12T12:43:32.512 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_snap_create SKIPPED [ 71%] 2026-02-12T12:43:32.512 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_snap_rollback SKIPPED [ 72%] 2026-02-12T12:43:32.513 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_discard SKIPPED [ 72%] 2026-02-12T12:43:32.513 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_write SKIPPED [ 73%] 2026-02-12T12:43:32.514 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_acquire_release_lock SKIPPED [ 73%] 2026-02-12T12:43:32.514 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_break_lock SKIPPED [ 74%] 2026-02-12T12:43:33.236 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_uuid PASSED [ 75%] 2026-02-12T12:43:33.972 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_site_name PASSED [ 75%] 2026-02-12T12:43:35.330 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_remote_namespace PASSED [ 76%] 2026-02-12T12:43:36.129 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_peer_bootstrap PASSED [ 76%] 2026-02-12T12:43:36.926 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_peer PASSED [ 77%] 2026-02-12T12:43:37.632 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_image SKIPPED [ 77%] 2026-02-12T12:43:38.353 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_image_status SKIPPED [ 78%] 2026-02-12T12:43:46.969 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_image_create_snapshot PASSED [ 78%] 2026-02-12T12:43:51.470 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_aio_mirror_image_create_snapshot PASSED [ 79%] 2026-02-12T12:43:52.006 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_move PASSED [ 79%] 2026-02-12T12:43:53.010 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_purge PASSED [ 80%] 2026-02-12T12:43:53.546 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_remove_denied PASSED [ 80%] 2026-02-12T12:43:54.053 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_remove PASSED [ 81%] 2026-02-12T12:43:54.558 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_remove_with_progress PASSED [ 81%] 2026-02-12T12:43:55.099 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_get PASSED [ 82%] 2026-02-12T12:43:56.153 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_list PASSED [ 82%] 2026-02-12T12:43:57.015 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_restore PASSED [ 83%] 2026-02-12T12:43:57.070 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_create_group PASSED [ 83%] 2026-02-12T12:43:57.151 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_rename_group PASSED [ 84%] 2026-02-12T12:43:57.155 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list_groups_empty PASSED [ 84%] 2026-02-12T12:43:57.210 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list_groups PASSED [ 85%] 2026-02-12T12:43:57.270 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list_groups_after_removed PASSED [ 85%] 2026-02-12T12:43:58.060 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_add PASSED [ 86%] 2026-02-12T12:43:58.749 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_list_empty PASSED [ 86%] 2026-02-12T12:43:59.531 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_list PASSED [ 87%] 2026-02-12T12:44:00.450 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_list_move_to_trash PASSED [ 87%] 2026-02-12T12:44:01.911 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_list_remove PASSED [ 88%] 2026-02-12T12:44:02.600 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_get_id PASSED [ 88%] 2026-02-12T12:44:16.475 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_many_images PASSED [ 89%] 2026-02-12T12:44:17.439 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_remove PASSED [ 89%] 2026-02-12T12:44:20.427 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_get_info PASSED [ 90%] 2026-02-12T12:44:21.204 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_get_info_no_member_images PASSED [ 90%] 2026-02-12T12:44:21.982 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap PASSED [ 91%] 2026-02-12T12:44:22.875 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_flags PASSED [ 91%] 2026-02-12T12:45:04.546 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_list_many PASSED [ 92%] 2026-02-12T12:45:06.960 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_namespace PASSED [ 92%] 2026-02-12T12:45:07.815 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_rename PASSED [ 93%] 2026-02-12T12:45:08.508 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_clone SKIPPED [ 93%] 2026-02-12T12:45:09.190 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_clone_flatten SKIPPED [ 94%] 2026-02-12T12:45:12.066 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_diff_iterate PASSED [ 94%] 2026-02-12T12:45:26.061 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_rollback PASSED [ 95%] 2026-02-12T12:45:28.390 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration PASSED [ 95%] 2026-02-12T12:45:28.391 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_clone_v1_to_v1 SKIPPED [ 96%] 2026-02-12T12:45:28.391 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_clone_v1_to_v2 SKIPPED [ 96%] 2026-02-12T12:45:28.392 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_clone_v2_to_v1 SKIPPED [ 97%] 2026-02-12T12:45:28.392 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_clone_v2_to_v2 SKIPPED [ 97%] 2026-02-12T12:45:34.262 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_import PASSED [ 98%] 2026-02-12T12:45:36.226 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_with_progress PASSED [ 98%] 2026-02-12T12:45:37.882 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migrate_abort PASSED [ 99%] 2026-02-12T12:45:40.152 INFO:tasks.workunit.client.0.trial050.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migrate_abort_with_progress PASSED [100%] 2026-02-12T12:45:40.153 INFO:tasks.workunit.client.0.trial050.stdout: 2026-02-12T12:45:40.153 INFO:tasks.workunit.client.0.trial050.stdout:=============================== warnings summary =============================== 2026-02-12T12:45:40.153 INFO:tasks.workunit.client.0.trial050.stdout:test_rbd.py::test_remove_with_progress_except 2026-02-12T12:45:40.153 INFO:tasks.workunit.client.0.trial050.stdout: /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: 'rbd.progress_callback' 2026-02-12T12:45:40.153 INFO:tasks.workunit.client.0.trial050.stdout: 2026-02-12T12:45:40.153 INFO:tasks.workunit.client.0.trial050.stdout: Traceback (most recent call last): 2026-02-12T12:45:40.153 INFO:tasks.workunit.client.0.trial050.stdout: File "/home/ubuntu/cephtest/clone.client.0/src/test/pybind/test_rbd.py", line 429, in progress_cb 2026-02-12T12:45:40.153 INFO:tasks.workunit.client.0.trial050.stdout: raise Exception() 2026-02-12T12:45:40.153 INFO:tasks.workunit.client.0.trial050.stdout: Exception 2026-02-12T12:45:40.154 INFO:tasks.workunit.client.0.trial050.stdout: 2026-02-12T12:45:40.154 INFO:tasks.workunit.client.0.trial050.stdout: warnings.warn(pytest.PytestUnraisableExceptionWarning(msg)) 2026-02-12T12:45:40.154 INFO:tasks.workunit.client.0.trial050.stdout: 2026-02-12T12:45:40.154 INFO:tasks.workunit.client.0.trial050.stdout:-- Docs: https://docs.pytest.org/en/stable/warnings.html 2026-02-12T12:45:40.154 INFO:tasks.workunit.client.0.trial050.stdout:============ 133 passed, 63 skipped, 1 warning in 835.11s (0:13:55) ============ 2026-02-12T12:45:40.218 INFO:tasks.workunit.client.0.trial050.stderr:+ exit 0 2026-02-12T12:45:40.218 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-12T12:45:40.218 DEBUG:teuthology.orchestra.run.trial050:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-12T12:45:40.289 INFO:tasks.workunit:Stopping ['rbd/test_librbd_python.sh'] on client.0... 2026-02-12T12:45:40.290 DEBUG:teuthology.orchestra.run.trial050:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-12T12:45:40.664 DEBUG:teuthology.parallel:result is None 2026-02-12T12:45:40.665 DEBUG:teuthology.orchestra.run.trial050:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-12T12:45:40.691 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-12T12:45:40.691 DEBUG:teuthology.orchestra.run.trial050:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-12T12:45:40.748 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-12T12:45:40.748 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-12T12:45:40.763 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-12T12:45:40.763 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-12T12:45:40.864 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:40.860+0000 7f732c63d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:40.867 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:40.865+0000 7f732c63d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:40.978 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:45:40.978 INFO:teuthology.orchestra.run.trial050.stderr:dumped all 2026-02-12T12:45:40.988 INFO:teuthology.orchestra.run.trial050.stdout:{"pg_ready":true,"pg_map":{"version":1024,"stamp":"2026-02-12T12:45:40.145585+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62935132,"kb_used_data":62935132,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94351220,"statfs":{"total":161061224448,"available":96615649280,"internally_reserved":0,"allocated":64445575168,"data_stored":64445575168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-164,"num_objects":-13,"num_object_clones":0,"num_object_copies":-26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":-339876,"num_read_kb":-37719,"num_write":-8136,"num_write_kb":-41427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.013032"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.149366+0000","last_change":"2026-02-12T12:31:24.133533+0000","last_active":"2026-02-12T12:45:33.149366+0000","last_peered":"2026-02-12T12:45:33.149366+0000","last_clean":"2026-02-12T12:45:33.149366+0000","last_became_active":"2026-02-12T12:31:22.151365+0000","last_became_peered":"2026-02-12T12:31:22.151365+0000","last_unstale":"2026-02-12T12:45:33.149366+0000","last_undegraded":"2026-02-12T12:45:33.149366+0000","last_fullsized":"2026-02-12T12:45:33.149366+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.147819+0000","last_change":"2026-02-12T12:31:24.133965+0000","last_active":"2026-02-12T12:45:33.147819+0000","last_peered":"2026-02-12T12:45:33.147819+0000","last_clean":"2026-02-12T12:45:33.147819+0000","last_became_active":"2026-02-12T12:31:22.149755+0000","last_became_peered":"2026-02-12T12:31:22.149755+0000","last_unstale":"2026-02-12T12:45:33.147819+0000","last_undegraded":"2026-02-12T12:45:33.147819+0000","last_fullsized":"2026-02-12T12:45:33.147819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.152097+0000","last_change":"2026-02-12T12:31:24.135573+0000","last_active":"2026-02-12T12:45:33.152097+0000","last_peered":"2026-02-12T12:45:33.152097+0000","last_clean":"2026-02-12T12:45:33.152097+0000","last_became_active":"2026-02-12T12:31:22.158500+0000","last_became_peered":"2026-02-12T12:31:22.158500+0000","last_unstale":"2026-02-12T12:45:33.152097+0000","last_undegraded":"2026-02-12T12:45:33.152097+0000","last_fullsized":"2026-02-12T12:45:33.152097+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.150601+0000","last_change":"2026-02-12T12:31:24.134224+0000","last_active":"2026-02-12T12:45:33.150601+0000","last_peered":"2026-02-12T12:45:33.150601+0000","last_clean":"2026-02-12T12:45:33.150601+0000","last_became_active":"2026-02-12T12:31:22.150491+0000","last_became_peered":"2026-02-12T12:31:22.150491+0000","last_unstale":"2026-02-12T12:45:33.150601+0000","last_undegraded":"2026-02-12T12:45:33.150601+0000","last_fullsized":"2026-02-12T12:45:33.150601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":1760,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.150597+0000","last_change":"2026-02-12T12:31:24.133829+0000","last_active":"2026-02-12T12:45:33.150597+0000","last_peered":"2026-02-12T12:45:33.150597+0000","last_clean":"2026-02-12T12:45:33.150597+0000","last_became_active":"2026-02-12T12:31:22.170355+0000","last_became_peered":"2026-02-12T12:31:22.170355+0000","last_unstale":"2026-02-12T12:45:33.150597+0000","last_undegraded":"2026-02-12T12:45:33.150597+0000","last_fullsized":"2026-02-12T12:45:33.150597+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.146047+0000","last_change":"2026-02-12T12:31:24.134494+0000","last_active":"2026-02-12T12:45:33.146047+0000","last_peered":"2026-02-12T12:45:33.146047+0000","last_clean":"2026-02-12T12:45:33.146047+0000","last_became_active":"2026-02-12T12:31:22.170811+0000","last_became_peered":"2026-02-12T12:31:22.170811+0000","last_unstale":"2026-02-12T12:45:33.146047+0000","last_undegraded":"2026-02-12T12:45:33.146047+0000","last_fullsized":"2026-02-12T12:45:33.146047+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.144282+0000","last_change":"2026-02-12T12:31:24.133038+0000","last_active":"2026-02-12T12:45:33.144282+0000","last_peered":"2026-02-12T12:45:33.144282+0000","last_clean":"2026-02-12T12:45:33.144282+0000","last_became_active":"2026-02-12T12:31:22.166160+0000","last_became_peered":"2026-02-12T12:31:22.166160+0000","last_unstale":"2026-02-12T12:45:33.144282+0000","last_undegraded":"2026-02-12T12:45:33.144282+0000","last_fullsized":"2026-02-12T12:45:33.144282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":1759,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.149280+0000","last_change":"2026-02-12T12:31:24.135685+0000","last_active":"2026-02-12T12:45:33.149280+0000","last_peered":"2026-02-12T12:45:33.149280+0000","last_clean":"2026-02-12T12:45:33.149280+0000","last_became_active":"2026-02-12T12:31:22.149374+0000","last_became_peered":"2026-02-12T12:31:22.149374+0000","last_unstale":"2026-02-12T12:45:33.149280+0000","last_undegraded":"2026-02-12T12:45:33.149280+0000","last_fullsized":"2026-02-12T12:45:33.149280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":1833,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.147977+0000","last_change":"2026-02-12T12:31:19.128575+0000","last_active":"2026-02-12T12:45:33.147977+0000","last_peered":"2026-02-12T12:45:33.147977+0000","last_clean":"2026-02-12T12:45:33.147977+0000","last_became_active":"2026-02-12T12:31:19.118740+0000","last_became_peered":"2026-02-12T12:31:19.118740+0000","last_unstale":"2026-02-12T12:45:33.147977+0000","last_undegraded":"2026-02-12T12:45:33.147977+0000","last_fullsized":"2026-02-12T12:45:33.147977+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:18.085509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:18.085509+0000","last_clean_scrub_stamp":"2026-02-12T12:31:18.085509+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96616067072,"internally_reserved":0,"allocated":64445157376,"data_stored":64445157376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32204001280,"internally_reserved":0,"allocated":21483073536,"data_stored":21483073536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949673819,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20977196,"kb_used_data":20977196,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31451588,"statfs":{"total":53687074816,"available":32206426112,"internally_reserved":0,"allocated":21480648704,"data_stored":21480648704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673819,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20979680,"kb_used_data":20979680,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31449104,"statfs":{"total":53687074816,"available":32203882496,"internally_reserved":0,"allocated":21483192320,"data_stored":21483192320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706528,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20978256,"kb_used_data":20978256,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31450528,"statfs":{"total":53687074816,"available":32205340672,"internally_reserved":0,"allocated":21481734144,"data_stored":21481734144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32204001280,"internally_reserved":0,"allocated":21483073536,"data_stored":21483073536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32205455360,"internally_reserved":0,"allocated":21481619456,"data_stored":21481619456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32204001280,"internally_reserved":0,"allocated":21483073536,"data_stored":21483073536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32206610432,"internally_reserved":0,"allocated":21480464384,"data_stored":21480464384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-12T12:45:40.989 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-12T12:45:41.067 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:41.063+0000 7fa2e950b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:41.069 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:41.067+0000 7fa2e950b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:41.179 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:45:41.179 INFO:teuthology.orchestra.run.trial050.stderr:dumped all 2026-02-12T12:45:41.190 INFO:teuthology.orchestra.run.trial050.stdout:{"pg_ready":true,"pg_map":{"version":1024,"stamp":"2026-02-12T12:45:40.145585+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62935132,"kb_used_data":62935132,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94351220,"statfs":{"total":161061224448,"available":96615649280,"internally_reserved":0,"allocated":64445575168,"data_stored":64445575168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-164,"num_objects":-13,"num_object_clones":0,"num_object_copies":-26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":-339876,"num_read_kb":-37719,"num_write":-8136,"num_write_kb":-41427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.013032"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.149366+0000","last_change":"2026-02-12T12:31:24.133533+0000","last_active":"2026-02-12T12:45:33.149366+0000","last_peered":"2026-02-12T12:45:33.149366+0000","last_clean":"2026-02-12T12:45:33.149366+0000","last_became_active":"2026-02-12T12:31:22.151365+0000","last_became_peered":"2026-02-12T12:31:22.151365+0000","last_unstale":"2026-02-12T12:45:33.149366+0000","last_undegraded":"2026-02-12T12:45:33.149366+0000","last_fullsized":"2026-02-12T12:45:33.149366+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.147819+0000","last_change":"2026-02-12T12:31:24.133965+0000","last_active":"2026-02-12T12:45:33.147819+0000","last_peered":"2026-02-12T12:45:33.147819+0000","last_clean":"2026-02-12T12:45:33.147819+0000","last_became_active":"2026-02-12T12:31:22.149755+0000","last_became_peered":"2026-02-12T12:31:22.149755+0000","last_unstale":"2026-02-12T12:45:33.147819+0000","last_undegraded":"2026-02-12T12:45:33.147819+0000","last_fullsized":"2026-02-12T12:45:33.147819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.152097+0000","last_change":"2026-02-12T12:31:24.135573+0000","last_active":"2026-02-12T12:45:33.152097+0000","last_peered":"2026-02-12T12:45:33.152097+0000","last_clean":"2026-02-12T12:45:33.152097+0000","last_became_active":"2026-02-12T12:31:22.158500+0000","last_became_peered":"2026-02-12T12:31:22.158500+0000","last_unstale":"2026-02-12T12:45:33.152097+0000","last_undegraded":"2026-02-12T12:45:33.152097+0000","last_fullsized":"2026-02-12T12:45:33.152097+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.150601+0000","last_change":"2026-02-12T12:31:24.134224+0000","last_active":"2026-02-12T12:45:33.150601+0000","last_peered":"2026-02-12T12:45:33.150601+0000","last_clean":"2026-02-12T12:45:33.150601+0000","last_became_active":"2026-02-12T12:31:22.150491+0000","last_became_peered":"2026-02-12T12:31:22.150491+0000","last_unstale":"2026-02-12T12:45:33.150601+0000","last_undegraded":"2026-02-12T12:45:33.150601+0000","last_fullsized":"2026-02-12T12:45:33.150601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":1760,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.150597+0000","last_change":"2026-02-12T12:31:24.133829+0000","last_active":"2026-02-12T12:45:33.150597+0000","last_peered":"2026-02-12T12:45:33.150597+0000","last_clean":"2026-02-12T12:45:33.150597+0000","last_became_active":"2026-02-12T12:31:22.170355+0000","last_became_peered":"2026-02-12T12:31:22.170355+0000","last_unstale":"2026-02-12T12:45:33.150597+0000","last_undegraded":"2026-02-12T12:45:33.150597+0000","last_fullsized":"2026-02-12T12:45:33.150597+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.146047+0000","last_change":"2026-02-12T12:31:24.134494+0000","last_active":"2026-02-12T12:45:33.146047+0000","last_peered":"2026-02-12T12:45:33.146047+0000","last_clean":"2026-02-12T12:45:33.146047+0000","last_became_active":"2026-02-12T12:31:22.170811+0000","last_became_peered":"2026-02-12T12:31:22.170811+0000","last_unstale":"2026-02-12T12:45:33.146047+0000","last_undegraded":"2026-02-12T12:45:33.146047+0000","last_fullsized":"2026-02-12T12:45:33.146047+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.144282+0000","last_change":"2026-02-12T12:31:24.133038+0000","last_active":"2026-02-12T12:45:33.144282+0000","last_peered":"2026-02-12T12:45:33.144282+0000","last_clean":"2026-02-12T12:45:33.144282+0000","last_became_active":"2026-02-12T12:31:22.166160+0000","last_became_peered":"2026-02-12T12:31:22.166160+0000","last_unstale":"2026-02-12T12:45:33.144282+0000","last_undegraded":"2026-02-12T12:45:33.144282+0000","last_fullsized":"2026-02-12T12:45:33.144282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":1759,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.149280+0000","last_change":"2026-02-12T12:31:24.135685+0000","last_active":"2026-02-12T12:45:33.149280+0000","last_peered":"2026-02-12T12:45:33.149280+0000","last_clean":"2026-02-12T12:45:33.149280+0000","last_became_active":"2026-02-12T12:31:22.149374+0000","last_became_peered":"2026-02-12T12:31:22.149374+0000","last_unstale":"2026-02-12T12:45:33.149280+0000","last_undegraded":"2026-02-12T12:45:33.149280+0000","last_fullsized":"2026-02-12T12:45:33.149280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":1833,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.147977+0000","last_change":"2026-02-12T12:31:19.128575+0000","last_active":"2026-02-12T12:45:33.147977+0000","last_peered":"2026-02-12T12:45:33.147977+0000","last_clean":"2026-02-12T12:45:33.147977+0000","last_became_active":"2026-02-12T12:31:19.118740+0000","last_became_peered":"2026-02-12T12:31:19.118740+0000","last_unstale":"2026-02-12T12:45:33.147977+0000","last_undegraded":"2026-02-12T12:45:33.147977+0000","last_fullsized":"2026-02-12T12:45:33.147977+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:18.085509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:18.085509+0000","last_clean_scrub_stamp":"2026-02-12T12:31:18.085509+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96616067072,"internally_reserved":0,"allocated":64445157376,"data_stored":64445157376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32204001280,"internally_reserved":0,"allocated":21483073536,"data_stored":21483073536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949673819,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20977196,"kb_used_data":20977196,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31451588,"statfs":{"total":53687074816,"available":32206426112,"internally_reserved":0,"allocated":21480648704,"data_stored":21480648704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673819,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20979680,"kb_used_data":20979680,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31449104,"statfs":{"total":53687074816,"available":32203882496,"internally_reserved":0,"allocated":21483192320,"data_stored":21483192320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706528,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20978256,"kb_used_data":20978256,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31450528,"statfs":{"total":53687074816,"available":32205340672,"internally_reserved":0,"allocated":21481734144,"data_stored":21481734144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32204001280,"internally_reserved":0,"allocated":21483073536,"data_stored":21483073536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32205455360,"internally_reserved":0,"allocated":21481619456,"data_stored":21481619456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32204001280,"internally_reserved":0,"allocated":21483073536,"data_stored":21483073536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32206610432,"internally_reserved":0,"allocated":21480464384,"data_stored":21480464384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-12T12:45:41.191 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-12T12:45:41.191 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-12T12:45:41.269 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:41.264+0000 7fa05b9ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:41.271 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:41.269+0000 7fa05b9ec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:41.383 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:45:41.383 INFO:teuthology.orchestra.run.trial050.stderr:dumped all 2026-02-12T12:45:41.393 INFO:teuthology.orchestra.run.trial050.stdout:{"pg_ready":true,"pg_map":{"version":1024,"stamp":"2026-02-12T12:45:40.145585+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62935132,"kb_used_data":62935132,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94351220,"statfs":{"total":161061224448,"available":96615649280,"internally_reserved":0,"allocated":64445575168,"data_stored":64445575168,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-164,"num_objects":-13,"num_object_clones":0,"num_object_copies":-26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":-339876,"num_read_kb":-37719,"num_write":-8136,"num_write_kb":-41427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.013032"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.149366+0000","last_change":"2026-02-12T12:31:24.133533+0000","last_active":"2026-02-12T12:45:33.149366+0000","last_peered":"2026-02-12T12:45:33.149366+0000","last_clean":"2026-02-12T12:45:33.149366+0000","last_became_active":"2026-02-12T12:31:22.151365+0000","last_became_peered":"2026-02-12T12:31:22.151365+0000","last_unstale":"2026-02-12T12:45:33.149366+0000","last_undegraded":"2026-02-12T12:45:33.149366+0000","last_fullsized":"2026-02-12T12:45:33.149366+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.147819+0000","last_change":"2026-02-12T12:31:24.133965+0000","last_active":"2026-02-12T12:45:33.147819+0000","last_peered":"2026-02-12T12:45:33.147819+0000","last_clean":"2026-02-12T12:45:33.147819+0000","last_became_active":"2026-02-12T12:31:22.149755+0000","last_became_peered":"2026-02-12T12:31:22.149755+0000","last_unstale":"2026-02-12T12:45:33.147819+0000","last_undegraded":"2026-02-12T12:45:33.147819+0000","last_fullsized":"2026-02-12T12:45:33.147819+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.152097+0000","last_change":"2026-02-12T12:31:24.135573+0000","last_active":"2026-02-12T12:45:33.152097+0000","last_peered":"2026-02-12T12:45:33.152097+0000","last_clean":"2026-02-12T12:45:33.152097+0000","last_became_active":"2026-02-12T12:31:22.158500+0000","last_became_peered":"2026-02-12T12:31:22.158500+0000","last_unstale":"2026-02-12T12:45:33.152097+0000","last_undegraded":"2026-02-12T12:45:33.152097+0000","last_fullsized":"2026-02-12T12:45:33.152097+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.150601+0000","last_change":"2026-02-12T12:31:24.134224+0000","last_active":"2026-02-12T12:45:33.150601+0000","last_peered":"2026-02-12T12:45:33.150601+0000","last_clean":"2026-02-12T12:45:33.150601+0000","last_became_active":"2026-02-12T12:31:22.150491+0000","last_became_peered":"2026-02-12T12:31:22.150491+0000","last_unstale":"2026-02-12T12:45:33.150601+0000","last_undegraded":"2026-02-12T12:45:33.150601+0000","last_fullsized":"2026-02-12T12:45:33.150601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":1760,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.150597+0000","last_change":"2026-02-12T12:31:24.133829+0000","last_active":"2026-02-12T12:45:33.150597+0000","last_peered":"2026-02-12T12:45:33.150597+0000","last_clean":"2026-02-12T12:45:33.150597+0000","last_became_active":"2026-02-12T12:31:22.170355+0000","last_became_peered":"2026-02-12T12:31:22.170355+0000","last_unstale":"2026-02-12T12:45:33.150597+0000","last_undegraded":"2026-02-12T12:45:33.150597+0000","last_fullsized":"2026-02-12T12:45:33.150597+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.146047+0000","last_change":"2026-02-12T12:31:24.134494+0000","last_active":"2026-02-12T12:45:33.146047+0000","last_peered":"2026-02-12T12:45:33.146047+0000","last_clean":"2026-02-12T12:45:33.146047+0000","last_became_active":"2026-02-12T12:31:22.170811+0000","last_became_peered":"2026-02-12T12:31:22.170811+0000","last_unstale":"2026-02-12T12:45:33.146047+0000","last_undegraded":"2026-02-12T12:45:33.146047+0000","last_fullsized":"2026-02-12T12:45:33.146047+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1758,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.144282+0000","last_change":"2026-02-12T12:31:24.133038+0000","last_active":"2026-02-12T12:45:33.144282+0000","last_peered":"2026-02-12T12:45:33.144282+0000","last_clean":"2026-02-12T12:45:33.144282+0000","last_became_active":"2026-02-12T12:31:22.166160+0000","last_became_peered":"2026-02-12T12:31:22.166160+0000","last_unstale":"2026-02-12T12:45:33.144282+0000","last_undegraded":"2026-02-12T12:45:33.144282+0000","last_fullsized":"2026-02-12T12:45:33.144282+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":1759,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.149280+0000","last_change":"2026-02-12T12:31:24.135685+0000","last_active":"2026-02-12T12:45:33.149280+0000","last_peered":"2026-02-12T12:45:33.149280+0000","last_clean":"2026-02-12T12:45:33.149280+0000","last_became_active":"2026-02-12T12:31:22.149374+0000","last_became_peered":"2026-02-12T12:31:22.149374+0000","last_unstale":"2026-02-12T12:45:33.149280+0000","last_undegraded":"2026-02-12T12:45:33.149280+0000","last_fullsized":"2026-02-12T12:45:33.149280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":1833,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.147977+0000","last_change":"2026-02-12T12:31:19.128575+0000","last_active":"2026-02-12T12:45:33.147977+0000","last_peered":"2026-02-12T12:45:33.147977+0000","last_clean":"2026-02-12T12:45:33.147977+0000","last_became_active":"2026-02-12T12:31:19.118740+0000","last_became_peered":"2026-02-12T12:31:19.118740+0000","last_unstale":"2026-02-12T12:45:33.147977+0000","last_undegraded":"2026-02-12T12:45:33.147977+0000","last_fullsized":"2026-02-12T12:45:33.147977+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:18.085509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:18.085509+0000","last_clean_scrub_stamp":"2026-02-12T12:31:18.085509+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96616067072,"internally_reserved":0,"allocated":64445157376,"data_stored":64445157376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32204001280,"internally_reserved":0,"allocated":21483073536,"data_stored":21483073536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949673819,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20977196,"kb_used_data":20977196,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31451588,"statfs":{"total":53687074816,"available":32206426112,"internally_reserved":0,"allocated":21480648704,"data_stored":21480648704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673819,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20979680,"kb_used_data":20979680,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31449104,"statfs":{"total":53687074816,"available":32203882496,"internally_reserved":0,"allocated":21483192320,"data_stored":21483192320,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706528,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20978256,"kb_used_data":20978256,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31450528,"statfs":{"total":53687074816,"available":32205340672,"internally_reserved":0,"allocated":21481734144,"data_stored":21481734144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32204001280,"internally_reserved":0,"allocated":21483073536,"data_stored":21483073536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32205455360,"internally_reserved":0,"allocated":21481619456,"data_stored":21481619456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32204001280,"internally_reserved":0,"allocated":21483073536,"data_stored":21483073536,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32206610432,"internally_reserved":0,"allocated":21480464384,"data_stored":21480464384,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-12T12:45:41.395 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-12T12:45:41.472 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:41.467+0000 7fd09910b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:41.474 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:41.472+0000 7fd09910b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:41.585 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:45:41.585 INFO:teuthology.orchestra.run.trial050.stdout:{"epoch":599,"fsid":"2a7ffb28-5872-4384-aac5-3a45ba84b5fb","created":"2026-02-12T12:31:04.979733+0000","modified":"2026-02-12T12:45:40.144725+0000","last_up_change":"2026-02-12T12:31:16.079863+0000","last_in_change":"2026-02-12T12:31:06.741621+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-12T12:31:17.451237+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-12T12:31:20.690992+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"899c6ef7-f201-4241-9cc6-0c7a1626b60a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6803","nonce":42187792}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6802","nonce":42187792}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6809","nonce":42187792}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6806","nonce":42187792}]},"public_addr":"10.20.193.50:6803/42187792","cluster_addr":"10.20.193.50:6802/42187792","heartbeat_back_addr":"10.20.193.50:6809/42187792","heartbeat_front_addr":"10.20.193.50:6806/42187792","state":["exists","up"]},{"osd":1,"uuid":"0ca533cd-2c69-460f-af47-9ab5f082bdb6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6808","nonce":2033975479}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6807","nonce":2033975479}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6810","nonce":2033975479}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6813","nonce":2033975479}]},"public_addr":"10.20.193.50:6808/2033975479","cluster_addr":"10.20.193.50:6807/2033975479","heartbeat_back_addr":"10.20.193.50:6810/2033975479","heartbeat_front_addr":"10.20.193.50:6813/2033975479","state":["exists","up"]},{"osd":2,"uuid":"3df821e7-05d0-461d-a880-6f185ac9c806","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":18,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6804","nonce":2156715747}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6805","nonce":2156715747}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6812","nonce":2156715747}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.50:6811","nonce":2156715747}]},"public_addr":"10.20.193.50:6804/2156715747","cluster_addr":"10.20.193.50:6805/2156715747","heartbeat_back_addr":"10.20.193.50:6812/2156715747","heartbeat_front_addr":"10.20.193.50:6811/2156715747","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]},{"pool":3,"snaps":[{"begin":2,"length":302}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T12:45:42.597 INFO:tasks.ceph:Scrubbing osd.0 2026-02-12T12:45:42.597 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-12T12:45:42.679 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:42.672+0000 7f80b3c11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:42.681 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:42.679+0000 7f80b3c11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:42.691 INFO:teuthology.orchestra.run.trial050.stdout:{ 2026-02-12T12:45:42.691 INFO:teuthology.orchestra.run.trial050.stdout: "success": "" 2026-02-12T12:45:42.691 INFO:teuthology.orchestra.run.trial050.stdout:} 2026-02-12T12:45:42.698 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-12T12:45:42.775 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:42.771+0000 7fc13f57b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:42.777 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:42.775+0000 7fc13f57b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:42.887 INFO:teuthology.orchestra.run.trial050.stderr:instructed osd(s) 0 to deep-scrub 2026-02-12T12:45:42.899 INFO:tasks.ceph:Scrubbing osd.1 2026-02-12T12:45:42.900 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-12T12:45:42.979 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:42.972+0000 7f7b9dec2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:42.982 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:42.980+0000 7f7b9dec2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:42.991 INFO:teuthology.orchestra.run.trial050.stdout:{ 2026-02-12T12:45:42.991 INFO:teuthology.orchestra.run.trial050.stdout: "success": "" 2026-02-12T12:45:42.991 INFO:teuthology.orchestra.run.trial050.stdout:} 2026-02-12T12:45:42.999 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-12T12:45:43.078 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:43.073+0000 7fde8bfdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:43.080 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:43.078+0000 7fde8bfdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:43.194 INFO:teuthology.orchestra.run.trial050.stderr:instructed osd(s) 1 to deep-scrub 2026-02-12T12:45:43.208 INFO:tasks.ceph:Scrubbing osd.2 2026-02-12T12:45:43.208 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-12T12:45:43.288 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:43.281+0000 7f2114851640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:43.290 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:43.288+0000 7f2114851640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:43.300 INFO:teuthology.orchestra.run.trial050.stdout:{ 2026-02-12T12:45:43.300 INFO:teuthology.orchestra.run.trial050.stdout: "success": "" 2026-02-12T12:45:43.300 INFO:teuthology.orchestra.run.trial050.stdout:} 2026-02-12T12:45:43.309 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-12T12:45:43.387 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:43.382+0000 7f6677406640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:43.390 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:43.388+0000 7f6677406640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:43.503 INFO:teuthology.orchestra.run.trial050.stderr:instructed osd(s) 2 to deep-scrub 2026-02-12T12:45:43.514 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-12T12:45:43.591 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:43.587+0000 7f57fdb0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:43.594 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:45:43.592+0000 7f57fdb0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:45:43.704 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:45:43.705 INFO:teuthology.orchestra.run.trial050.stderr:dumped all 2026-02-12T12:45:43.714 INFO:teuthology.orchestra.run.trial050.stdout:{"pg_ready":true,"pg_map":{"version":1026,"stamp":"2026-02-12T12:45:43.570560+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62930376,"kb_used_data":62930376,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94355976,"statfs":{"total":161061224448,"available":96620519424,"internally_reserved":0,"allocated":64440705024,"data_stored":64440705024,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-164,"num_objects":-13,"num_object_clones":0,"num_object_copies":-26,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":-339876,"num_read_kb":-37719,"num_write":-8136,"num_write_kb":-41427,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"10.001564"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":1761,"reported_epoch":599,"state":"active+clean","last_fresh":"2026-02-12T12:45:40.173827+0000","last_change":"2026-02-12T12:31:24.133533+0000","last_active":"2026-02-12T12:45:40.173827+0000","last_peered":"2026-02-12T12:45:40.173827+0000","last_clean":"2026-02-12T12:45:40.173827+0000","last_became_active":"2026-02-12T12:31:22.151365+0000","last_became_peered":"2026-02-12T12:31:22.151365+0000","last_unstale":"2026-02-12T12:45:40.173827+0000","last_undegraded":"2026-02-12T12:45:40.173827+0000","last_fullsized":"2026-02-12T12:45:40.173827+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1761,"reported_epoch":599,"state":"active+clean","last_fresh":"2026-02-12T12:45:40.168895+0000","last_change":"2026-02-12T12:31:24.133965+0000","last_active":"2026-02-12T12:45:40.168895+0000","last_peered":"2026-02-12T12:45:40.168895+0000","last_clean":"2026-02-12T12:45:40.168895+0000","last_became_active":"2026-02-12T12:31:22.149755+0000","last_became_peered":"2026-02-12T12:31:22.149755+0000","last_unstale":"2026-02-12T12:45:40.168895+0000","last_undegraded":"2026-02-12T12:45:40.168895+0000","last_fullsized":"2026-02-12T12:45:40.168895+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1761,"reported_epoch":599,"state":"active+clean","last_fresh":"2026-02-12T12:45:40.174551+0000","last_change":"2026-02-12T12:31:24.135573+0000","last_active":"2026-02-12T12:45:40.174551+0000","last_peered":"2026-02-12T12:45:40.174551+0000","last_clean":"2026-02-12T12:45:40.174551+0000","last_became_active":"2026-02-12T12:31:22.158500+0000","last_became_peered":"2026-02-12T12:31:22.158500+0000","last_unstale":"2026-02-12T12:45:40.174551+0000","last_undegraded":"2026-02-12T12:45:40.174551+0000","last_fullsized":"2026-02-12T12:45:40.174551+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1761,"reported_epoch":599,"state":"active+clean","last_fresh":"2026-02-12T12:45:40.172609+0000","last_change":"2026-02-12T12:31:24.134224+0000","last_active":"2026-02-12T12:45:40.172609+0000","last_peered":"2026-02-12T12:45:40.172609+0000","last_clean":"2026-02-12T12:45:40.172609+0000","last_became_active":"2026-02-12T12:31:22.150491+0000","last_became_peered":"2026-02-12T12:31:22.150491+0000","last_unstale":"2026-02-12T12:45:40.172609+0000","last_undegraded":"2026-02-12T12:45:40.172609+0000","last_fullsized":"2026-02-12T12:45:40.172609+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":1760,"reported_epoch":598,"state":"active+clean","last_fresh":"2026-02-12T12:45:33.150597+0000","last_change":"2026-02-12T12:31:24.133829+0000","last_active":"2026-02-12T12:45:33.150597+0000","last_peered":"2026-02-12T12:45:33.150597+0000","last_clean":"2026-02-12T12:45:33.150597+0000","last_became_active":"2026-02-12T12:31:22.170355+0000","last_became_peered":"2026-02-12T12:31:22.170355+0000","last_unstale":"2026-02-12T12:45:33.150597+0000","last_undegraded":"2026-02-12T12:45:33.150597+0000","last_fullsized":"2026-02-12T12:45:33.150597+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1761,"reported_epoch":599,"state":"active+clean","last_fresh":"2026-02-12T12:45:40.168241+0000","last_change":"2026-02-12T12:31:24.134494+0000","last_active":"2026-02-12T12:45:40.168241+0000","last_peered":"2026-02-12T12:45:40.168241+0000","last_clean":"2026-02-12T12:45:40.168241+0000","last_became_active":"2026-02-12T12:31:22.170811+0000","last_became_peered":"2026-02-12T12:31:22.170811+0000","last_unstale":"2026-02-12T12:45:40.168241+0000","last_undegraded":"2026-02-12T12:45:40.168241+0000","last_fullsized":"2026-02-12T12:45:40.168241+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1761,"reported_epoch":599,"state":"active+clean","last_fresh":"2026-02-12T12:45:40.171681+0000","last_change":"2026-02-12T12:31:24.133038+0000","last_active":"2026-02-12T12:45:40.171681+0000","last_peered":"2026-02-12T12:45:40.171681+0000","last_clean":"2026-02-12T12:45:40.171681+0000","last_became_active":"2026-02-12T12:31:22.166160+0000","last_became_peered":"2026-02-12T12:31:22.166160+0000","last_unstale":"2026-02-12T12:45:40.171681+0000","last_undegraded":"2026-02-12T12:45:40.171681+0000","last_fullsized":"2026-02-12T12:45:40.171681+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":1762,"reported_epoch":599,"state":"active+clean","last_fresh":"2026-02-12T12:45:40.172803+0000","last_change":"2026-02-12T12:31:24.135685+0000","last_active":"2026-02-12T12:45:40.172803+0000","last_peered":"2026-02-12T12:45:40.172803+0000","last_clean":"2026-02-12T12:45:40.172803+0000","last_became_active":"2026-02-12T12:31:22.149374+0000","last_became_peered":"2026-02-12T12:31:22.149374+0000","last_unstale":"2026-02-12T12:45:40.172803+0000","last_undegraded":"2026-02-12T12:45:40.172803+0000","last_fullsized":"2026-02-12T12:45:40.172803+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:21.102225+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":1836,"reported_epoch":599,"state":"active+clean","last_fresh":"2026-02-12T12:45:40.168309+0000","last_change":"2026-02-12T12:31:19.128575+0000","last_active":"2026-02-12T12:45:40.168309+0000","last_peered":"2026-02-12T12:45:40.168309+0000","last_clean":"2026-02-12T12:45:40.168309+0000","last_became_active":"2026-02-12T12:31:19.118740+0000","last_became_peered":"2026-02-12T12:31:19.118740+0000","last_unstale":"2026-02-12T12:45:40.168309+0000","last_undegraded":"2026-02-12T12:45:40.168309+0000","last_fullsized":"2026-02-12T12:45:40.168309+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:31:18.085509+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:31:18.085509+0000","last_clean_scrub_stamp":"2026-02-12T12:31:18.085509+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96625209344,"internally_reserved":0,"allocated":64436015104,"data_stored":64436015104,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32209457152,"internally_reserved":0,"allocated":21477617664,"data_stored":21477617664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949673824,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20974616,"kb_used_data":20974616,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454168,"statfs":{"total":53687074816,"available":32209068032,"internally_reserved":0,"allocated":21478006784,"data_stored":21478006784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673824,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20977504,"kb_used_data":20977504,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31451280,"statfs":{"total":53687074816,"available":32206110720,"internally_reserved":0,"allocated":21480964096,"data_stored":21480964096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706528,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20978256,"kb_used_data":20978256,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31450528,"statfs":{"total":53687074816,"available":32205340672,"internally_reserved":0,"allocated":21481734144,"data_stored":21481734144,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32209457152,"internally_reserved":0,"allocated":21477617664,"data_stored":21477617664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32205455360,"internally_reserved":0,"allocated":21481619456,"data_stored":21481619456,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32209457152,"internally_reserved":0,"allocated":21477617664,"data_stored":21477617664,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32210296832,"internally_reserved":0,"allocated":21476777984,"data_stored":21476777984,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-12T12:45:43.719 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-12T12:31:21.102225+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=31, tm_sec=21, tm_wday=3, tm_yday=43, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=45, tm_sec=41, tm_wday=3, tm_yday=43, tm_isdst=0) 2026-02-12T12:45:43.719 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-12T12:31:21.102225+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=31, tm_sec=21, tm_wday=3, tm_yday=43, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=45, tm_sec=41, tm_wday=3, tm_yday=43, tm_isdst=0) 2026-02-12T12:45:43.719 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-12T12:31:21.102225+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=31, tm_sec=21, tm_wday=3, tm_yday=43, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=45, tm_sec=41, tm_wday=3, tm_yday=43, tm_isdst=0) 2026-02-12T12:45:43.720 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-12T12:31:21.102225+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=31, tm_sec=21, tm_wday=3, tm_yday=43, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=45, tm_sec=41, tm_wday=3, tm_yday=43, tm_isdst=0) 2026-02-12T12:45:43.720 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-12T12:31:21.102225+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=31, tm_sec=21, tm_wday=3, tm_yday=43, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=45, tm_sec=41, tm_wday=3, tm_yday=43, tm_isdst=0) 2026-02-12T12:45:43.720 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2026-02-12T12:31:21.102225+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=31, tm_sec=21, tm_wday=3, tm_yday=43, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=45, tm_sec=41, tm_wday=3, tm_yday=43, tm_isdst=0) 2026-02-12T12:45:43.720 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-12T12:31:21.102225+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=31, tm_sec=21, tm_wday=3, tm_yday=43, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=45, tm_sec=41, tm_wday=3, tm_yday=43, tm_isdst=0) 2026-02-12T12:45:43.720 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2026-02-12T12:31:21.102225+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=31, tm_sec=21, tm_wday=3, tm_yday=43, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=45, tm_sec=41, tm_wday=3, tm_yday=43, tm_isdst=0) 2026-02-12T12:45:43.720 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-12T12:31:18.085509+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=31, tm_sec=18, tm_wday=3, tm_yday=43, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=12, tm_hour=12, tm_min=45, tm_sec=41, tm_wday=3, tm_yday=43, tm_isdst=0) 2026-02-12T12:45:43.720 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-12T12:46:03.721 DEBUG:teuthology.orchestra.run.trial050:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-12T12:46:03.800 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:46:03.796+0000 7fe8820c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:46:03.802 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:46:03.800+0000 7fe8820c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:46:03.913 INFO:teuthology.orchestra.run.trial050.stdout: 2026-02-12T12:46:03.914 INFO:teuthology.orchestra.run.trial050.stderr:dumped all 2026-02-12T12:46:03.923 INFO:teuthology.orchestra.run.trial050.stdout:{"pg_ready":true,"pg_map":{"version":1037,"stamp":"2026-02-12T12:46:03.573846+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62921836,"kb_used_data":62921836,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94364516,"statfs":{"total":161061224448,"available":96629264384,"internally_reserved":0,"allocated":64431960064,"data_stored":64431960064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001893"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":1766,"reported_epoch":600,"state":"active+clean","last_fresh":"2026-02-12T12:45:45.123851+0000","last_change":"2026-02-12T12:45:43.210036+0000","last_active":"2026-02-12T12:45:45.123851+0000","last_peered":"2026-02-12T12:45:45.123851+0000","last_clean":"2026-02-12T12:45:45.123851+0000","last_became_active":"2026-02-12T12:31:22.151365+0000","last_became_peered":"2026-02-12T12:31:22.151365+0000","last_unstale":"2026-02-12T12:45:45.123851+0000","last_undegraded":"2026-02-12T12:45:45.123851+0000","last_fullsized":"2026-02-12T12:45:45.123851+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:45:43.209986+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:45:43.209986+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":1766,"reported_epoch":600,"state":"active+clean","last_fresh":"2026-02-12T12:45:45.124838+0000","last_change":"2026-02-12T12:45:43.209265+0000","last_active":"2026-02-12T12:45:45.124838+0000","last_peered":"2026-02-12T12:45:45.124838+0000","last_clean":"2026-02-12T12:45:45.124838+0000","last_became_active":"2026-02-12T12:31:22.149755+0000","last_became_peered":"2026-02-12T12:31:22.149755+0000","last_unstale":"2026-02-12T12:45:45.124838+0000","last_undegraded":"2026-02-12T12:45:45.124838+0000","last_fullsized":"2026-02-12T12:45:45.124838+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:45:43.209183+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:45:43.209183+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":1766,"reported_epoch":600,"state":"active+clean","last_fresh":"2026-02-12T12:45:45.124640+0000","last_change":"2026-02-12T12:45:43.208461+0000","last_active":"2026-02-12T12:45:45.124640+0000","last_peered":"2026-02-12T12:45:45.124640+0000","last_clean":"2026-02-12T12:45:45.124640+0000","last_became_active":"2026-02-12T12:31:22.158500+0000","last_became_peered":"2026-02-12T12:31:22.158500+0000","last_unstale":"2026-02-12T12:45:45.124640+0000","last_undegraded":"2026-02-12T12:45:45.124640+0000","last_fullsized":"2026-02-12T12:45:45.124640+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:45:43.208406+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:45:43.208406+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":1766,"reported_epoch":600,"state":"active+clean","last_fresh":"2026-02-12T12:45:45.124802+0000","last_change":"2026-02-12T12:45:43.205042+0000","last_active":"2026-02-12T12:45:45.124802+0000","last_peered":"2026-02-12T12:45:45.124802+0000","last_clean":"2026-02-12T12:45:45.124802+0000","last_became_active":"2026-02-12T12:31:22.150491+0000","last_became_peered":"2026-02-12T12:31:22.150491+0000","last_unstale":"2026-02-12T12:45:45.124802+0000","last_undegraded":"2026-02-12T12:45:45.124802+0000","last_fullsized":"2026-02-12T12:45:45.124802+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:45:43.204962+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:45:43.204962+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":1768,"reported_epoch":600,"state":"active+clean","last_fresh":"2026-02-12T12:45:45.125034+0000","last_change":"2026-02-12T12:45:42.898256+0000","last_active":"2026-02-12T12:45:45.125034+0000","last_peered":"2026-02-12T12:45:45.125034+0000","last_clean":"2026-02-12T12:45:45.125034+0000","last_became_active":"2026-02-12T12:31:22.170355+0000","last_became_peered":"2026-02-12T12:31:22.170355+0000","last_unstale":"2026-02-12T12:45:45.125034+0000","last_undegraded":"2026-02-12T12:45:45.125034+0000","last_fullsized":"2026-02-12T12:45:45.125034+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"17'2","last_scrub_stamp":"2026-02-12T12:45:42.898171+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:45:42.898171+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":1766,"reported_epoch":600,"state":"active+clean","last_fresh":"2026-02-12T12:45:45.123685+0000","last_change":"2026-02-12T12:45:43.511021+0000","last_active":"2026-02-12T12:45:45.123685+0000","last_peered":"2026-02-12T12:45:45.123685+0000","last_clean":"2026-02-12T12:45:45.123685+0000","last_became_active":"2026-02-12T12:31:22.170811+0000","last_became_peered":"2026-02-12T12:31:22.170811+0000","last_unstale":"2026-02-12T12:45:45.123685+0000","last_undegraded":"2026-02-12T12:45:45.123685+0000","last_fullsized":"2026-02-12T12:45:45.123685+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:45:43.510938+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:45:43.510938+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":1766,"reported_epoch":600,"state":"active+clean","last_fresh":"2026-02-12T12:45:45.122921+0000","last_change":"2026-02-12T12:45:43.511532+0000","last_active":"2026-02-12T12:45:45.122921+0000","last_peered":"2026-02-12T12:45:45.122921+0000","last_clean":"2026-02-12T12:45:45.122921+0000","last_became_active":"2026-02-12T12:31:22.166160+0000","last_became_peered":"2026-02-12T12:31:22.166160+0000","last_unstale":"2026-02-12T12:45:45.122921+0000","last_undegraded":"2026-02-12T12:45:45.122921+0000","last_fullsized":"2026-02-12T12:45:45.122921+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T12:45:43.511477+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:45:43.511477+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":1767,"reported_epoch":600,"state":"active+clean","last_fresh":"2026-02-12T12:45:45.124914+0000","last_change":"2026-02-12T12:45:43.207788+0000","last_active":"2026-02-12T12:45:45.124914+0000","last_peered":"2026-02-12T12:45:45.124914+0000","last_clean":"2026-02-12T12:45:45.124914+0000","last_became_active":"2026-02-12T12:31:22.149374+0000","last_became_peered":"2026-02-12T12:31:22.149374+0000","last_unstale":"2026-02-12T12:45:45.124914+0000","last_undegraded":"2026-02-12T12:45:45.124914+0000","last_fullsized":"2026-02-12T12:45:45.124914+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'1","last_scrub_stamp":"2026-02-12T12:45:43.207722+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:45:43.207722+0000","last_clean_scrub_stamp":"2026-02-12T12:31:21.102225+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":1841,"reported_epoch":600,"state":"active+clean","last_fresh":"2026-02-12T12:45:45.123481+0000","last_change":"2026-02-12T12:45:43.221020+0000","last_active":"2026-02-12T12:45:45.123481+0000","last_peered":"2026-02-12T12:45:45.123481+0000","last_clean":"2026-02-12T12:45:45.123481+0000","last_became_active":"2026-02-12T12:31:19.118740+0000","last_became_peered":"2026-02-12T12:31:19.118740+0000","last_unstale":"2026-02-12T12:45:45.123481+0000","last_undegraded":"2026-02-12T12:45:45.123481+0000","last_fullsized":"2026-02-12T12:45:45.123481+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"12'52","last_scrub_stamp":"2026-02-12T12:45:43.220940+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T12:45:43.220940+0000","last_clean_scrub_stamp":"2026-02-12T12:31:18.085509+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96629264384,"internally_reserved":0,"allocated":64431960064,"data_stored":64431960064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32209448960,"internally_reserved":0,"allocated":21477625856,"data_stored":21477625856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949673844,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20973424,"kb_used_data":20973424,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31455360,"statfs":{"total":53687074816,"available":32210288640,"internally_reserved":0,"allocated":21476786176,"data_stored":21476786176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949673844,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20974244,"kb_used_data":20974244,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454540,"statfs":{"total":53687074816,"available":32209448960,"internally_reserved":0,"allocated":21477625856,"data_stored":21477625856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654706548,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20974168,"kb_used_data":20974168,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454616,"statfs":{"total":53687074816,"available":32209526784,"internally_reserved":0,"allocated":21477548032,"data_stored":21477548032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32209448960,"internally_reserved":0,"allocated":21477625856,"data_stored":21477625856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32209526784,"internally_reserved":0,"allocated":21477548032,"data_stored":21477548032,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32209448960,"internally_reserved":0,"allocated":21477625856,"data_stored":21477625856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32210288640,"internally_reserved":0,"allocated":21476786176,"data_stored":21476786176,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-12T12:46:03.925 DEBUG:teuthology.orchestra.run.trial050:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-12T12:46:04.001 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:46:03.995+0000 7f543437b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:46:04.003 INFO:teuthology.orchestra.run.trial050.stderr:2026-02-12T12:46:04.001+0000 7f543437b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T12:46:04.131 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-12T12:46:04.131 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-12T12:46:04.132 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-12T12:46:04.132 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T12:46:04.204 INFO:tasks.ceph.osd.0:Stopped 2026-02-12T12:46:04.204 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-12T12:46:04.204 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T12:46:04.273 INFO:tasks.ceph.osd.1:Stopped 2026-02-12T12:46:04.273 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-12T12:46:04.273 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T12:46:04.343 INFO:tasks.ceph.osd.2:Stopped 2026-02-12T12:46:04.344 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-12T12:46:04.344 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-12T12:46:04.344 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T12:46:04.379 INFO:tasks.ceph.mgr.x:Stopped 2026-02-12T12:46:04.379 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-12T12:46:04.380 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-12T12:46:04.380 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T12:46:04.431 INFO:tasks.ceph.mon.a:Stopped 2026-02-12T12:46:04.431 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-12T12:46:04.432 DEBUG:teuthology.orchestra.run.trial050:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v '\(SLOW_OPS\)' | grep -E -v 'slow request' | head -n 1 2026-02-12T12:46:04.497 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial050.front.sepia.ceph.com 2026-02-12T12:46:04.497 DEBUG:teuthology.orchestra.run.trial050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-12T12:46:04.617 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial050.front.sepia.ceph.com 2026-02-12T12:46:04.618 DEBUG:teuthology.orchestra.run.trial050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-12T12:46:04.643 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial050.front.sepia.ceph.com 2026-02-12T12:46:04.644 DEBUG:teuthology.orchestra.run.trial050:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-12T12:46:04.709 INFO:tasks.ceph:Archiving mon data... 2026-02-12T12:46:04.710 DEBUG:teuthology.misc:Transferring archived files from trial050:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/moagrawa-2026-02-10_06:35:43-crimson-rados-main-distro-debug-trial/42514/data/mon.a.tgz 2026-02-12T12:46:04.710 DEBUG:teuthology.orchestra.run.trial050:> mktemp 2026-02-12T12:46:04.765 INFO:teuthology.orchestra.run.trial050.stdout:/tmp/tmp.PikbJeckPS 2026-02-12T12:46:04.765 DEBUG:teuthology.orchestra.run.trial050:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.PikbJeckPS 2026-02-12T12:46:04.860 DEBUG:teuthology.orchestra.run.trial050:> sudo chmod 0666 /tmp/tmp.PikbJeckPS 2026-02-12T12:46:04.941 DEBUG:teuthology.orchestra.remote:trial050:/tmp/tmp.PikbJeckPS is 262KB 2026-02-12T12:46:05.035 DEBUG:teuthology.orchestra.run.trial050:> rm -fr /tmp/tmp.PikbJeckPS 2026-02-12T12:46:05.052 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-12T12:46:05.053 DEBUG:teuthology.orchestra.run.trial050:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-12T12:46:05.175 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-12T12:46:05.175 INFO:tasks.ceph:Archiving crash dumps... 2026-02-12T12:46:05.177 DEBUG:teuthology.misc:Transferring archived files from trial050:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/moagrawa-2026-02-10_06:35:43-crimson-rados-main-distro-debug-trial/42514/remote/trial050/crash 2026-02-12T12:46:05.179 DEBUG:teuthology.orchestra.run.trial050:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-12T12:46:05.240 INFO:tasks.ceph:Compressing logs... 2026-02-12T12:46:05.240 DEBUG:teuthology.orchestra.run.trial050:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-12T12:46:05.303 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-12T12:46:05.317 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-12T12:46:05.317 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20073.log 2026-02-12T12:46:05.317 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23028.log 2026-02-12T12:46:05.317 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.20073.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22489.log 2026-02-12T12:46:05.317 INFO:teuthology.orchestra.run.trial050.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.20073.log.gz 2026-02-12T12:46:05.317 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.23028.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-12T12:46:05.317 INFO:teuthology.orchestra.run.trial050.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.23028.log.gz 2026-02-12T12:46:05.317 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19105.log/var/log/ceph/ceph-client.admin.22489.log: 2026-02-12T12:46:05.317 INFO:teuthology.orchestra.run.trial050.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22489.log.gz 2026-02-12T12:46:05.317 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20072.log 2026-02-12T12:46:05.318 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.19105.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19872.log 2026-02-12T12:46:05.318 INFO:teuthology.orchestra.run.trial050.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.19105.log.gz 2026-02-12T12:46:05.318 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22930.log 2026-02-12T12:46:05.318 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.19872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22734.log 2026-02-12T12:46:05.318 INFO:teuthology.orchestra.run.trial050.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.19872.log.gz 2026-02-12T12:46:05.318 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.20072.log: /var/log/ceph/ceph-client.admin.22930.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22538.log 2026-02-12T12:46:05.318 INFO:teuthology.orchestra.run.trial050.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22930.log.gz 2026-02-12T12:46:05.318 INFO:teuthology.orchestra.run.trial050.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.20072.log.gz 2026-02-12T12:46:05.318 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.22734.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19921.log 2026-02-12T12:46:05.318 INFO:teuthology.orchestra.run.trial050.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22734.log.gz 2026-02-12T12:46:05.318 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.22538.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18968.log 2026-02-12T12:46:05.318 INFO:teuthology.orchestra.run.trial050.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22538.log.gz 2026-02-12T12:46:05.319 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.19921.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20071.log 2026-02-12T12:46:05.319 INFO:teuthology.orchestra.run.trial050.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.19921.log.gz 2026-02-12T12:46:05.319 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23075.log 2026-02-12T12:46:05.319 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.18968.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.18968.log.gz 2026-02-12T12:46:05.319 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.20071.log: gzip 50.0% -5 -- replaced with /var/log/ceph/ceph-client.admin.20071.log.gz --verbose 2026-02-12T12:46:05.319 INFO:teuthology.orchestra.run.trial050.stderr: -- /var/log/ceph/ceph-client.admin.20219.log 2026-02-12T12:46:05.319 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.23075.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22587.log 2026-02-12T12:46:05.319 INFO:teuthology.orchestra.run.trial050.stderr: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.23075.log.gz 2026-02-12T12:46:05.319 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.20219.log: gzip -5 50.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.20219.log.gz /var/log/ceph/ceph-client.admin.20362.log 2026-02-12T12:46:05.319 INFO:teuthology.orchestra.run.trial050.stderr: 2026-02-12T12:46:05.319 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.22587.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22587.log.gz 2026-02-12T12:46:05.319 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19448.log 2026-02-12T12:46:05.320 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.20362.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20313.log 2026-02-12T12:46:05.320 INFO:teuthology.orchestra.run.trial050.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.20362.log.gz 2026-02-12T12:46:05.320 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.19448.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.19448.log.gz 2026-02-12T12:46:05.320 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19595.log 2026-02-12T12:46:05.320 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.20313.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19823.log 2026-02-12T12:46:05.320 INFO:teuthology.orchestra.run.trial050.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.20313.log.gz 2026-02-12T12:46:05.320 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.19595.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.19595.log.gz 2026-02-12T12:46:05.320 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20411.log 2026-02-12T12:46:05.320 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.19823.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22979.log 2026-02-12T12:46:05.320 INFO:teuthology.orchestra.run.trial050.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.19823.log.gz 2026-02-12T12:46:05.320 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.20411.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.20411.log.gz 2026-02-12T12:46:05.320 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19152.log 2026-02-12T12:46:05.321 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.22979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20687.log 2026-02-12T12:46:05.321 INFO:teuthology.orchestra.run.trial050.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.22979.log.gz 2026-02-12T12:46:05.321 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.19152.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.19152.log.gz 2026-02-12T12:46:05.321 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19970.log 2026-02-12T12:46:05.321 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.20687.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-12T12:46:05.321 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.19970.log: 87.9% -- replaced with /var/log/ceph/ceph-client.admin.20687.log.gz 2026-02-12T12:46:05.321 INFO:teuthology.orchestra.run.trial050.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.19970.log.gz 2026-02-12T12:46:05.321 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18921.log 2026-02-12T12:46:05.321 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19733.log 2026-02-12T12:46:05.321 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.18921.log: 86.9% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-12T12:46:05.321 INFO:teuthology.orchestra.run.trial050.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.18921.log.gz 2026-02-12T12:46:05.321 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19663.log 2026-02-12T12:46:05.321 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.19733.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-12T12:46:05.321 INFO:teuthology.orchestra.run.trial050.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.19733.log.gz 2026-02-12T12:46:05.322 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.19663.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.19663.log.gz 2026-02-12T12:46:05.322 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19418.log 2026-02-12T12:46:05.322 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.18826.log 2026-02-12T12:46:05.322 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.19418.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.19418.log.gz 2026-02-12T12:46:05.322 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22783.log 2026-02-12T12:46:05.322 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.18826.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19199.log 2026-02-12T12:46:05.322 INFO:teuthology.orchestra.run.trial050.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.18826.log.gz 2026-02-12T12:46:05.322 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.22783.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.22783.log.gz 2026-02-12T12:46:05.322 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22685.log 2026-02-12T12:46:05.322 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.19199.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.19199.log.gz 2026-02-12T12:46:05.322 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20166.log 2026-02-12T12:46:05.322 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.22685.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22685.log.gz 2026-02-12T12:46:05.323 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22636.log 2026-02-12T12:46:05.323 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.20166.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.20166.log.gz 2026-02-12T12:46:05.323 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.19758.log 2026-02-12T12:46:05.323 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.22636.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22636.log.gz 2026-02-12T12:46:05.323 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-12T12:46:05.323 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.19758.log: 65.9% -- replaced with /var/log/ceph/ceph-client.admin.19758.log.gz 2026-02-12T12:46:05.323 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22881.log 2026-02-12T12:46:05.323 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-12T12:46:05.323 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.22881.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22881.log.gz 2026-02-12T12:46:05.323 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.20220.log 2026-02-12T12:46:05.323 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22832.log 2026-02-12T12:46:05.323 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.20220.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.20220.log.gz 2026-02-12T12:46:05.323 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.17536.log 2026-02-12T12:46:05.324 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph-client.admin.22832.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22832.log.gz 2026-02-12T12:46:05.324 INFO:teuthology.orchestra.run.trial050.stderr:/var/log/ceph/ceph.tmp-client.admin.17536.log: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.17536.log.gz 2026-02-12T12:46:05.324 INFO:teuthology.orchestra.run.trial050.stderr: 87.2% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-12T12:46:05.360 INFO:teuthology.orchestra.run.trial050.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-12T12:46:05.560 INFO:teuthology.orchestra.run.trial050.stderr: 89.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-12T12:46:09.099 INFO:teuthology.orchestra.run.trial050.stderr: 92.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-12T12:46:09.952 INFO:teuthology.orchestra.run.trial050.stderr: 93.1% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-12T12:46:16.948 INFO:teuthology.orchestra.run.trial050.stderr: 95.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-12T12:46:16.950 INFO:teuthology.orchestra.run.trial050.stderr: 2026-02-12T12:46:16.950 INFO:teuthology.orchestra.run.trial050.stderr:real 0m11.654s 2026-02-12T12:46:16.951 INFO:teuthology.orchestra.run.trial050.stderr:user 0m19.174s 2026-02-12T12:46:16.951 INFO:teuthology.orchestra.run.trial050.stderr:sys 0m1.205s 2026-02-12T12:46:16.951 INFO:tasks.ceph:Archiving logs... 2026-02-12T12:46:16.951 DEBUG:teuthology.misc:Transferring archived files from trial050:/var/log/ceph to /home/teuthworker/mnt/teuthology/moagrawa-2026-02-10_06:35:43-crimson-rados-main-distro-debug-trial/42514/remote/trial050/log 2026-02-12T12:46:16.953 DEBUG:teuthology.orchestra.run.trial050:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-12T12:46:18.489 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-12T12:46:18.562 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-12T12:46:18.573 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-12T12:46:18.573 DEBUG:teuthology.orchestra.run.trial050:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-12T12:46:18.599 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-12T12:46:18.610 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-12T12:46:18.611 DEBUG:teuthology.orchestra.run.trial050:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-12T12:46:18.655 INFO:teuthology.orchestra.run.trial050.stderr:bash: line 1: ntpq: command not found 2026-02-12T12:46:18.677 INFO:teuthology.orchestra.run.trial050.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-12T12:46:18.677 INFO:teuthology.orchestra.run.trial050.stdout:=============================================================================== 2026-02-12T12:46:18.677 INFO:teuthology.orchestra.run.trial050.stdout:^- time4.lshiy.com 2 7 377 42 +7929ns[+7929ns] +/- 122ms 2026-02-12T12:46:18.677 INFO:teuthology.orchestra.run.trial050.stdout:^* nyc3.us.ntp.li 2 6 377 47 -887us[ -884us] +/- 3382us 2026-02-12T12:46:18.677 INFO:teuthology.orchestra.run.trial050.stdout:^- kc1cloud.147.72.155.23.m> 3 6 377 45 -375us[ -375us] +/- 53ms 2026-02-12T12:46:18.677 INFO:teuthology.orchestra.run.trial050.stdout:^- 172-238-164-57.ip.linode> 4 7 377 109 -4249us[-4245us] +/- 33ms 2026-02-12T12:46:18.678 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-12T12:46:18.692 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-12T12:46:18.692 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-12T12:46:18.707 DEBUG:teuthology.orchestra.run.trial050:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-12T12:46:18.762 DEBUG:teuthology.orchestra.run.trial050:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-12T12:46:18.828 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T12:46:18.829 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-12T12:46:18.844 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-12T12:46:18.860 INFO:teuthology.task.internal:Duration was 1290.939529 seconds 2026-02-12T12:46:18.860 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-12T12:46:18.875 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-12T12:46:18.875 DEBUG:teuthology.orchestra.run.trial050:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-12T12:46:18.913 INFO:teuthology.orchestra.run.trial050.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-12T12:46:19.158 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-12T12:46:19.158 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial050.front.sepia.ceph.com 2026-02-12T12:46:19.159 DEBUG:teuthology.orchestra.run.trial050:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-12T12:46:19.218 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-12T12:46:19.219 DEBUG:teuthology.orchestra.run.trial050:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-12T12:46:19.299 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-12T12:46:19.300 DEBUG:teuthology.orchestra.run.trial050:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-12T12:46:19.362 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T12:46:19.363 INFO:teuthology.orchestra.run.trial050.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T12:46:19.363 INFO:teuthology.orchestra.run.trial050.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-12T12:46:19.364 INFO:teuthology.orchestra.run.trial050.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-12T12:46:19.364 INFO:teuthology.orchestra.run.trial050.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-12T12:46:19.366 INFO:teuthology.orchestra.run.trial050.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 87.6% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-12T12:46:19.368 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-12T12:46:19.385 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-12T12:46:19.385 DEBUG:teuthology.orchestra.run.trial050:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-12T12:46:19.432 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-12T12:46:19.450 DEBUG:teuthology.orchestra.run.trial050:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-12T12:46:19.495 INFO:teuthology.orchestra.run.trial050.stdout:kernel.core_pattern = core 2026-02-12T12:46:19.513 DEBUG:teuthology.orchestra.run.trial050:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-12T12:46:19.568 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T12:46:19.569 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-12T12:46:19.586 INFO:teuthology.task.internal:Transferring archived files... 2026-02-12T12:46:19.587 DEBUG:teuthology.misc:Transferring archived files from trial050:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/moagrawa-2026-02-10_06:35:43-crimson-rados-main-distro-debug-trial/42514/remote/trial050 2026-02-12T12:46:19.587 DEBUG:teuthology.orchestra.run.trial050:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-12T12:46:19.643 INFO:teuthology.task.internal:Removing archive directory... 2026-02-12T12:46:19.643 DEBUG:teuthology.orchestra.run.trial050:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-12T12:46:19.692 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-12T12:46:19.710 INFO:teuthology.task.internal:Not uploading archives. 2026-02-12T12:46:19.710 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-12T12:46:19.726 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-12T12:46:19.727 DEBUG:teuthology.orchestra.run.trial050:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-12T12:46:19.748 INFO:teuthology.orchestra.run.trial050.stdout: 791883 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 12 12:46 /home/ubuntu/cephtest 2026-02-12T12:46:19.749 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-12T12:46:19.767 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-12T12:46:19.804 INFO:teuthology.run:Summary data: description: crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} tasks/rbd_python_api_tests_old_format} duration: 1290.9395289421082 flavor: debug owner: scheduled_moagrawa@soko04.front.sepia.ceph.com success: true 2026-02-12T12:46:19.804 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-12T12:46:19.918 INFO:teuthology.run:pass