2026-02-20T23:36:04.757 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-20T23:36:04.771 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-20T23:36:04.850 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61870 branch: wip-liucm-osd-shard-redesign-crimson-only description: crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} tasks/rbd_python_api_tests} email: null first_in_suite: false flavor: debug job_id: '61870' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-liucm-osd-shard-redesign-crimson-only ceph: conf: global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug ms: 20 debug osd: 20 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore main device type: RANDOM_BLOCK_SSD seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - \(SLOW_OPS\) - slow request sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: extra_system_packages: - python3-pytest flavor: debug sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f workunit: branch: wip-liucm-osd-shard-redesign-crimson-only env: CRIMSON_COMPAT: '1' sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f owner: scheduled_liucm@soko04.front.sepia.ceph.com priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 6283 sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f sleep_before_teardown: 0 suite: crimson-rados suite_branch: wip-liucm-osd-shard-redesign-crimson-only suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f targets: trial057.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJSHHyhJtv/dDcwMHqjCODxpvw3rgBPcmUMWFhPBegk6A9G7XF5/Lc0g+rv+/H6xw+i3I0SzBFxHy4ybUa/TAYY= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 - workunit: clients: client.0: - rbd/test_librbd_python.sh env: RBD_FEATURES: '61' teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-20_22:16:48 tube: trial user: liucm verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-20T23:36:04.850 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/qa; will attempt to use it 2026-02-20T23:36:04.851 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/qa/tasks 2026-02-20T23:36:04.851 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-20T23:36:04.852 INFO:teuthology.task.internal:Checking packages... 2026-02-20T23:36:04.852 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f' 2026-02-20T23:36:04.852 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-20T23:36:04.852 INFO:teuthology.packaging:ref: None 2026-02-20T23:36:04.852 INFO:teuthology.packaging:tag: None 2026-02-20T23:36:04.853 INFO:teuthology.packaging:branch: wip-liucm-osd-shard-redesign-crimson-only 2026-02-20T23:36:04.853 INFO:teuthology.packaging:sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-20T23:36:04.853 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=wip-liucm-osd-shard-redesign-crimson-only 2026-02-20T23:36:04.990 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5343.gbcdd429b 2026-02-20T23:36:04.992 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-20T23:36:05.003 INFO:teuthology.task.internal:no buildpackages task found 2026-02-20T23:36:05.003 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-20T23:36:05.017 INFO:teuthology.task.internal:Saving configuration 2026-02-20T23:36:05.029 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-20T23:36:05.041 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-20T23:36:05.124 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial057.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61870', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-20 23:32:35.040834', 'locked_by': 'scheduled_liucm@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJSHHyhJtv/dDcwMHqjCODxpvw3rgBPcmUMWFhPBegk6A9G7XF5/Lc0g+rv+/H6xw+i3I0SzBFxHy4ybUa/TAYY='} 2026-02-20T23:36:05.124 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-20T23:36:05.135 INFO:teuthology.task.internal:roles: ubuntu@trial057.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-20T23:36:05.135 INFO:teuthology.run_tasks:Running task console_log... 2026-02-20T23:36:05.222 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x78a893e17ba0>, signals=[15]) 2026-02-20T23:36:05.222 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-20T23:36:05.233 INFO:teuthology.task.internal:Opening connections... 2026-02-20T23:36:05.233 DEBUG:teuthology.task.internal:connecting to ubuntu@trial057.front.sepia.ceph.com 2026-02-20T23:36:05.234 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T23:36:05.296 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-20T23:36:05.305 DEBUG:teuthology.orchestra.run.trial057:> uname -m 2026-02-20T23:36:05.373 INFO:teuthology.orchestra.run.trial057.stdout:x86_64 2026-02-20T23:36:05.374 DEBUG:teuthology.orchestra.run.trial057:> cat /etc/os-release 2026-02-20T23:36:05.429 INFO:teuthology.orchestra.run.trial057.stdout:NAME="CentOS Stream" 2026-02-20T23:36:05.430 INFO:teuthology.orchestra.run.trial057.stdout:VERSION="9" 2026-02-20T23:36:05.430 INFO:teuthology.orchestra.run.trial057.stdout:ID="centos" 2026-02-20T23:36:05.430 INFO:teuthology.orchestra.run.trial057.stdout:ID_LIKE="rhel fedora" 2026-02-20T23:36:05.430 INFO:teuthology.orchestra.run.trial057.stdout:VERSION_ID="9" 2026-02-20T23:36:05.430 INFO:teuthology.orchestra.run.trial057.stdout:PLATFORM_ID="platform:el9" 2026-02-20T23:36:05.430 INFO:teuthology.orchestra.run.trial057.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-20T23:36:05.430 INFO:teuthology.orchestra.run.trial057.stdout:ANSI_COLOR="0;31" 2026-02-20T23:36:05.430 INFO:teuthology.orchestra.run.trial057.stdout:LOGO="fedora-logo-icon" 2026-02-20T23:36:05.430 INFO:teuthology.orchestra.run.trial057.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-20T23:36:05.430 INFO:teuthology.orchestra.run.trial057.stdout:HOME_URL="https://centos.org/" 2026-02-20T23:36:05.430 INFO:teuthology.orchestra.run.trial057.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-20T23:36:05.430 INFO:teuthology.orchestra.run.trial057.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-20T23:36:05.430 INFO:teuthology.orchestra.run.trial057.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-20T23:36:05.431 INFO:teuthology.lock.ops:Updating trial057.front.sepia.ceph.com on lock server 2026-02-20T23:36:05.521 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-20T23:36:05.534 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-20T23:36:05.545 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-20T23:36:05.545 DEBUG:teuthology.orchestra.run.trial057:> test '!' -e /home/ubuntu/cephtest 2026-02-20T23:36:05.560 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-20T23:36:05.572 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-20T23:36:05.572 DEBUG:teuthology.orchestra.run.trial057:> test -z $(ls -A /var/lib/ceph) 2026-02-20T23:36:05.617 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-20T23:36:05.853 INFO:teuthology.run_tasks:Running task kernel... 2026-02-20T23:36:05.877 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-20T23:36:05.877 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-20T23:36:05.877 DEBUG:teuthology.orchestra.run.trial057:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-20T23:36:05.892 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T23:36:05.892 DEBUG:teuthology.orchestra.run.trial057:> uname -r 2026-02-20T23:36:05.947 INFO:teuthology.orchestra.run.trial057.stdout:5.14.0-681.el9.x86_64 2026-02-20T23:36:05.947 INFO:teuthology.task.kernel:Running kernel on trial057: 5.14.0-681.el9.x86_64 2026-02-20T23:36:05.947 DEBUG:teuthology.orchestra.run.trial057:> sudo yum install -y kernel 2026-02-20T23:36:07.140 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-20T23:36:10.378 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 27 MB 00:02 2026-02-20T23:36:13.757 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 7.8 MB 00:01 2026-02-20T23:36:14.592 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - Extras packages 188 kB/s | 20 kB 00:00 2026-02-20T23:36:21.515 INFO:teuthology.orchestra.run.trial057.stdout:Extra Packages for Enterprise Linux 3.0 MB/s | 20 MB 00:06 2026-02-20T23:36:24.724 INFO:teuthology.orchestra.run.trial057.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-20T23:36:25.732 INFO:teuthology.orchestra.run.trial057.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-20T23:36:25.732 INFO:teuthology.orchestra.run.trial057.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-20T23:36:25.751 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-20T23:36:25.752 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-20T23:36:25.752 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-20T23:36:25.774 DEBUG:teuthology.orchestra.run.trial057:> echo no | sudo yum reinstall kernel || true 2026-02-20T23:36:26.075 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 11:36:24 PM UTC. 2026-02-20T23:36:26.155 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-20T23:36:26.155 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-20T23:36:26.155 INFO:teuthology.orchestra.run.trial057.stdout: Package Architecture Version Repository Size 2026-02-20T23:36:26.155 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-20T23:36:26.155 INFO:teuthology.orchestra.run.trial057.stdout:Reinstalling: 2026-02-20T23:36:26.155 INFO:teuthology.orchestra.run.trial057.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T23:36:26.155 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:36:26.155 INFO:teuthology.orchestra.run.trial057.stdout:Transaction Summary 2026-02-20T23:36:26.155 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-20T23:36:26.156 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:36:26.156 INFO:teuthology.orchestra.run.trial057.stdout:Total download size: 803 k 2026-02-20T23:36:26.156 INFO:teuthology.orchestra.run.trial057.stdout:Installed size: 0 2026-02-20T23:36:26.156 INFO:teuthology.orchestra.run.trial057.stderr:Operation aborted. 2026-02-20T23:36:26.175 INFO:teuthology.orchestra.run.trial057.stdout:Is this ok [y/N]: 2026-02-20T23:36:26.176 DEBUG:teuthology.orchestra.run.trial057:> sudo yum reinstall -y kernel || true 2026-02-20T23:36:26.476 INFO:teuthology.orchestra.run.trial057.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 11:36:24 PM UTC. 2026-02-20T23:36:26.554 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-20T23:36:26.554 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-20T23:36:26.555 INFO:teuthology.orchestra.run.trial057.stdout: Package Architecture Version Repository Size 2026-02-20T23:36:26.555 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-20T23:36:26.555 INFO:teuthology.orchestra.run.trial057.stdout:Reinstalling: 2026-02-20T23:36:26.555 INFO:teuthology.orchestra.run.trial057.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T23:36:26.555 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:36:26.555 INFO:teuthology.orchestra.run.trial057.stdout:Transaction Summary 2026-02-20T23:36:26.555 INFO:teuthology.orchestra.run.trial057.stdout:================================================================================ 2026-02-20T23:36:26.555 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:36:26.555 INFO:teuthology.orchestra.run.trial057.stdout:Total download size: 803 k 2026-02-20T23:36:26.555 INFO:teuthology.orchestra.run.trial057.stdout:Installed size: 0 2026-02-20T23:36:26.555 INFO:teuthology.orchestra.run.trial057.stdout:Downloading Packages: 2026-02-20T23:36:26.644 INFO:teuthology.orchestra.run.trial057.stdout:kernel-5.14.0-681.el9.x86_64.rpm 29 MB/s | 803 kB 00:00 2026-02-20T23:36:26.644 INFO:teuthology.orchestra.run.trial057.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:36:26.644 INFO:teuthology.orchestra.run.trial057.stdout:Total 8.8 MB/s | 803 kB 00:00 2026-02-20T23:36:26.654 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction check 2026-02-20T23:36:26.687 INFO:teuthology.orchestra.run.trial057.stdout:Transaction check succeeded. 2026-02-20T23:36:26.687 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction test 2026-02-20T23:36:26.691 INFO:teuthology.orchestra.run.trial057.stdout:Transaction test succeeded. 2026-02-20T23:36:26.691 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction 2026-02-20T23:36:26.715 INFO:teuthology.orchestra.run.trial057.stdout: Preparing : 1/1 2026-02-20T23:36:26.719 INFO:teuthology.orchestra.run.trial057.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T23:36:26.719 INFO:teuthology.orchestra.run.trial057.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T23:36:26.828 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T23:36:26.828 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T23:36:26.879 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T23:36:26.879 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:36:26.879 INFO:teuthology.orchestra.run.trial057.stdout:Reinstalled: 2026-02-20T23:36:26.879 INFO:teuthology.orchestra.run.trial057.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-20T23:36:26.879 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:36:26.879 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-20T23:36:26.905 DEBUG:teuthology.orchestra.run.trial057:> rpm -q kernel | sort -rV | head -n 1 2026-02-20T23:36:26.929 INFO:teuthology.orchestra.run.trial057.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-20T23:36:26.929 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-20T23:36:26.929 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-20T23:36:26.929 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-20T23:36:26.930 DEBUG:teuthology.orchestra.run.trial057:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-20T23:36:26.994 INFO:teuthology.orchestra.run.trial057.stdout:ttyS1 2026-02-20T23:36:27.011 DEBUG:teuthology.parallel:result is None 2026-02-20T23:36:27.011 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-20T23:36:27.022 INFO:teuthology.task.internal:Creating test directory... 2026-02-20T23:36:27.023 DEBUG:teuthology.orchestra.run.trial057:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-20T23:36:27.066 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-20T23:36:27.076 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-20T23:36:27.090 INFO:teuthology.task.internal:Creating archive directory... 2026-02-20T23:36:27.090 DEBUG:teuthology.orchestra.run.trial057:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-20T23:36:27.128 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-20T23:36:27.139 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-20T23:36:27.139 DEBUG:teuthology.orchestra.run.trial057:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-20T23:36:27.187 INFO:teuthology.orchestra.run.trial057.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T23:36:27.195 INFO:teuthology.orchestra.run.trial057.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T23:36:27.197 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-20T23:36:27.208 INFO:teuthology.task.internal:Configuring sudo... 2026-02-20T23:36:27.208 DEBUG:teuthology.orchestra.run.trial057:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-20T23:36:27.262 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-20T23:36:27.280 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-20T23:36:27.281 DEBUG:teuthology.orchestra.run.trial057:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-20T23:36:27.318 DEBUG:teuthology.orchestra.run.trial057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T23:36:27.480 DEBUG:teuthology.orchestra.run.trial057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T23:36:27.504 DEBUG:teuthology.orchestra.run.trial057:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T23:36:27.647 DEBUG:teuthology.orchestra.run.trial057:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T23:36:27.671 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:36:27.671 DEBUG:teuthology.orchestra.run.trial057:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-20T23:36:27.733 DEBUG:teuthology.orchestra.run.trial057:> sudo service rsyslog restart 2026-02-20T23:36:27.801 INFO:teuthology.orchestra.run.trial057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-20T23:36:28.076 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-20T23:36:28.088 INFO:teuthology.task.internal:Starting timer... 2026-02-20T23:36:28.088 INFO:teuthology.run_tasks:Running task pcp... 2026-02-20T23:36:28.108 INFO:teuthology.run_tasks:Running task selinux... 2026-02-20T23:36:28.126 DEBUG:teuthology.orchestra.run.trial057:> sudo service auditd rotate 2026-02-20T23:36:28.155 INFO:teuthology.orchestra.run.trial057.stdout:Rotating logs: 2026-02-20T23:36:28.157 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-20T23:36:28.157 DEBUG:teuthology.orchestra.run.trial057:> /usr/sbin/getenforce 2026-02-20T23:36:28.214 INFO:teuthology.orchestra.run.trial057.stdout:Permissive 2026-02-20T23:36:28.214 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial057.front.sepia.ceph.com': 'permissive'} 2026-02-20T23:36:28.214 DEBUG:teuthology.orchestra.run.trial057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-20T23:36:28.278 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T23:36:28.279 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-20T23:36:28.279 DEBUG:teuthology.orchestra.run.trial057:> sudo /usr/sbin/setenforce permissive 2026-02-20T23:36:28.344 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-20T23:36:28.357 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-20T23:36:28.364 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-20T23:36:28.364 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial057.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-20T23:39:06.506 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial057.front.sepia.ceph.com')] 2026-02-20T23:39:06.507 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial057.front.sepia.ceph.com' 2026-02-20T23:39:06.507 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial057.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T23:39:06.608 DEBUG:teuthology.orchestra.run.trial057:> true 2026-02-20T23:39:06.687 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial057.front.sepia.ceph.com' 2026-02-20T23:39:06.687 INFO:teuthology.run_tasks:Running task clock... 2026-02-20T23:39:06.700 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-20T23:39:06.700 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-20T23:39:06.700 DEBUG:teuthology.orchestra.run.trial057:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-20T23:39:06.755 INFO:teuthology.orchestra.run.trial057.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-20T23:39:06.767 INFO:teuthology.orchestra.run.trial057.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-20T23:39:06.787 INFO:teuthology.orchestra.run.trial057.stderr:sudo: ntpd: command not found 2026-02-20T23:39:06.796 INFO:teuthology.orchestra.run.trial057.stdout:506 Cannot talk to daemon 2026-02-20T23:39:06.808 INFO:teuthology.orchestra.run.trial057.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-20T23:39:06.820 INFO:teuthology.orchestra.run.trial057.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-20T23:39:06.865 INFO:teuthology.orchestra.run.trial057.stderr:bash: line 1: ntpq: command not found 2026-02-20T23:39:06.883 INFO:teuthology.orchestra.run.trial057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-20T23:39:06.883 INFO:teuthology.orchestra.run.trial057.stdout:=============================================================================== 2026-02-20T23:39:06.884 INFO:teuthology.orchestra.run.trial057.stdout:^? time.trtnw.net 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-20T23:39:06.884 INFO:teuthology.orchestra.run.trial057.stdout:^? wdc1.us.ntp.li 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-20T23:39:06.884 INFO:teuthology.orchestra.run.trial057.stdout:^? h96-60-247-29.mdsnwi.bro> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-20T23:39:06.884 INFO:teuthology.orchestra.run.trial057.stdout:^? t2.time.bf1.yahoo.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-20T23:39:06.885 INFO:teuthology.run_tasks:Running task install... 2026-02-20T23:39:06.900 DEBUG:teuthology.task.install:project ceph 2026-02-20T23:39:06.900 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}} 2026-02-20T23:39:06.900 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'} 2026-02-20T23:39:06.900 INFO:teuthology.task.install:Using flavor: debug 2026-02-20T23:39:06.904 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-20T23:39:06.904 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-20T23:39:06.904 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': ['python3-pytest'], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'tag': None, 'wait_for_package': False} 2026-02-20T23:39:06.904 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-20T23:39:07.037 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/ 2026-02-20T23:39:07.038 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5343.gbcdd429b 2026-02-20T23:39:07.419 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-20T23:39:07.419 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:39:07.419 DEBUG:teuthology.orchestra.run.trial057:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-20T23:39:07.446 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo, python3-pytest on remote rpm x86_64 2026-02-20T23:39:07.446 DEBUG:teuthology.orchestra.run.trial057:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-20T23:39:07.511 DEBUG:teuthology.orchestra.run.trial057:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-20T23:39:07.590 DEBUG:teuthology.orchestra.run.trial057:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-20T23:39:07.664 INFO:teuthology.orchestra.run.trial057.stdout:check_obsoletes = 1 2026-02-20T23:39:07.665 DEBUG:teuthology.orchestra.run.trial057:> sudo yum clean all 2026-02-20T23:39:07.827 INFO:teuthology.orchestra.run.trial057.stdout:41 files removed 2026-02-20T23:39:07.842 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd ceph-osd-crimson ceph-osd-crimson-debuginfo python3-pytest 2026-02-20T23:39:09.152 INFO:teuthology.orchestra.run.trial057.stdout:ceph packages for x86_64 79 kB/s | 92 kB 00:01 2026-02-20T23:39:09.816 INFO:teuthology.orchestra.run.trial057.stdout:ceph noarch packages 37 kB/s | 24 kB 00:00 2026-02-20T23:39:10.411 INFO:teuthology.orchestra.run.trial057.stdout:ceph source packages 3.6 kB/s | 2.1 kB 00:00 2026-02-20T23:39:11.338 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - BaseOS 9.8 MB/s | 8.9 MB 00:00 2026-02-20T23:39:38.536 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 27 MB 00:26 2026-02-20T23:39:49.803 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - CRB 905 kB/s | 7.8 MB 00:08 2026-02-20T23:39:50.932 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - Extras packages 50 kB/s | 20 kB 00:00 2026-02-20T23:39:52.243 INFO:teuthology.orchestra.run.trial057.stdout:Extra Packages for Enterprise Linux 16 MB/s | 20 MB 00:01 2026-02-20T23:39:55.491 INFO:teuthology.orchestra.run.trial057.stdout:lab-extras 2.8 MB/s | 50 kB 00:00 2026-02-20T23:39:56.682 INFO:teuthology.orchestra.run.trial057.stdout:Package librados2-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-20T23:39:56.683 INFO:teuthology.orchestra.run.trial057.stdout:Package librbd1-2:16.2.4-5.el9.x86_64 is already installed. 2026-02-20T23:39:56.709 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-20T23:39:56.714 INFO:teuthology.orchestra.run.trial057.stdout:======================================================================================= 2026-02-20T23:39:56.714 INFO:teuthology.orchestra.run.trial057.stdout: Package Arch Version Repository Size 2026-02-20T23:39:56.714 INFO:teuthology.orchestra.run.trial057.stdout:======================================================================================= 2026-02-20T23:39:56.714 INFO:teuthology.orchestra.run.trial057.stdout:Installing: 2026-02-20T23:39:56.714 INFO:teuthology.orchestra.run.trial057.stdout: ceph x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 6.5 k 2026-02-20T23:39:56.714 INFO:teuthology.orchestra.run.trial057.stdout: ceph-base x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 5.1 M 2026-02-20T23:39:56.714 INFO:teuthology.orchestra.run.trial057.stdout: ceph-fuse x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 914 k 2026-02-20T23:39:56.715 INFO:teuthology.orchestra.run.trial057.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 149 k 2026-02-20T23:39:56.715 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 937 k 2026-02-20T23:39:56.715 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 184 k 2026-02-20T23:39:56.715 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 51 M 2026-02-20T23:39:56.715 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 7.4 M 2026-02-20T23:39:56.715 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-rook noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 50 k 2026-02-20T23:39:56.715 INFO:teuthology.orchestra.run.trial057.stdout: ceph-osd x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 163 k 2026-02-20T23:39:56.715 INFO:teuthology.orchestra.run.trial057.stdout: ceph-osd-classic x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 15 M 2026-02-20T23:39:56.715 INFO:teuthology.orchestra.run.trial057.stdout: ceph-osd-crimson x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 88 M 2026-02-20T23:39:56.715 INFO:teuthology.orchestra.run.trial057.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 433 M 2026-02-20T23:39:56.715 INFO:teuthology.orchestra.run.trial057.stdout: ceph-radosgw x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 11 M 2026-02-20T23:39:56.716 INFO:teuthology.orchestra.run.trial057.stdout: ceph-test x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 126 M 2026-02-20T23:39:56.716 INFO:teuthology.orchestra.run.trial057.stdout: ceph-volume noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 301 k 2026-02-20T23:39:56.716 INFO:teuthology.orchestra.run.trial057.stdout: cephadm noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 1.0 M 2026-02-20T23:39:56.716 INFO:teuthology.orchestra.run.trial057.stdout: libcephfs-devel x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 37 k 2026-02-20T23:39:56.716 INFO:teuthology.orchestra.run.trial057.stdout: libcephfs2 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 860 k 2026-02-20T23:39:56.716 INFO:teuthology.orchestra.run.trial057.stdout: librados-devel x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 124 k 2026-02-20T23:39:56.716 INFO:teuthology.orchestra.run.trial057.stdout: python3-cephfs x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 203 k 2026-02-20T23:39:56.716 INFO:teuthology.orchestra.run.trial057.stdout: python3-pytest noarch 6.2.2-7.el9 appstream 519 k 2026-02-20T23:39:56.716 INFO:teuthology.orchestra.run.trial057.stdout: python3-rados x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 324 k 2026-02-20T23:39:56.717 INFO:teuthology.orchestra.run.trial057.stdout: python3-rbd x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 305 k 2026-02-20T23:39:56.717 INFO:teuthology.orchestra.run.trial057.stdout: python3-rgw x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 100 k 2026-02-20T23:39:56.717 INFO:teuthology.orchestra.run.trial057.stdout: rbd-fuse x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 86 k 2026-02-20T23:39:56.717 INFO:teuthology.orchestra.run.trial057.stdout: rbd-mirror x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 2.6 M 2026-02-20T23:39:56.717 INFO:teuthology.orchestra.run.trial057.stdout: rbd-nbd x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 166 k 2026-02-20T23:39:56.717 INFO:teuthology.orchestra.run.trial057.stdout:Upgrading: 2026-02-20T23:39:56.717 INFO:teuthology.orchestra.run.trial057.stdout: librados2 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 3.3 M 2026-02-20T23:39:56.717 INFO:teuthology.orchestra.run.trial057.stdout: librbd1 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 2.7 M 2026-02-20T23:39:56.717 INFO:teuthology.orchestra.run.trial057.stdout:Installing dependencies: 2026-02-20T23:39:56.717 INFO:teuthology.orchestra.run.trial057.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-20T23:39:56.718 INFO:teuthology.orchestra.run.trial057.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-20T23:39:56.718 INFO:teuthology.orchestra.run.trial057.stdout: ceph-common x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 21 M 2026-02-20T23:39:56.718 INFO:teuthology.orchestra.run.trial057.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 57 k 2026-02-20T23:39:56.718 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mds x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 2.0 M 2026-02-20T23:39:56.718 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 297 k 2026-02-20T23:39:56.718 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mon x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 4.9 M 2026-02-20T23:39:56.718 INFO:teuthology.orchestra.run.trial057.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 17 k 2026-02-20T23:39:56.718 INFO:teuthology.orchestra.run.trial057.stdout: ceph-selinux x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 25 k 2026-02-20T23:39:56.718 INFO:teuthology.orchestra.run.trial057.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-20T23:39:56.718 INFO:teuthology.orchestra.run.trial057.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-20T23:39:56.718 INFO:teuthology.orchestra.run.trial057.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-20T23:39:56.718 INFO:teuthology.orchestra.run.trial057.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-20T23:39:56.719 INFO:teuthology.orchestra.run.trial057.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-20T23:39:56.719 INFO:teuthology.orchestra.run.trial057.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-20T23:39:56.719 INFO:teuthology.orchestra.run.trial057.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-20T23:39:56.719 INFO:teuthology.orchestra.run.trial057.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-20T23:39:56.719 INFO:teuthology.orchestra.run.trial057.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-20T23:39:56.719 INFO:teuthology.orchestra.run.trial057.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-20T23:39:56.719 INFO:teuthology.orchestra.run.trial057.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-20T23:39:56.719 INFO:teuthology.orchestra.run.trial057.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-20T23:39:56.719 INFO:teuthology.orchestra.run.trial057.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 26 k 2026-02-20T23:39:56.719 INFO:teuthology.orchestra.run.trial057.stdout: libcephsqlite x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 153 k 2026-02-20T23:39:56.719 INFO:teuthology.orchestra.run.trial057.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-20T23:39:56.719 INFO:teuthology.orchestra.run.trial057.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-20T23:39:56.720 INFO:teuthology.orchestra.run.trial057.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-20T23:39:56.720 INFO:teuthology.orchestra.run.trial057.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-20T23:39:56.720 INFO:teuthology.orchestra.run.trial057.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-20T23:39:56.720 INFO:teuthology.orchestra.run.trial057.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-20T23:39:56.720 INFO:teuthology.orchestra.run.trial057.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-20T23:39:56.720 INFO:teuthology.orchestra.run.trial057.stdout: libradosstriper1 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 263 k 2026-02-20T23:39:56.720 INFO:teuthology.orchestra.run.trial057.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-20T23:39:56.720 INFO:teuthology.orchestra.run.trial057.stdout: librgw2 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 5.6 M 2026-02-20T23:39:56.720 INFO:teuthology.orchestra.run.trial057.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-20T23:39:56.720 INFO:teuthology.orchestra.run.trial057.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-20T23:39:56.720 INFO:teuthology.orchestra.run.trial057.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-20T23:39:56.720 INFO:teuthology.orchestra.run.trial057.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-20T23:39:56.720 INFO:teuthology.orchestra.run.trial057.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-20T23:39:56.721 INFO:teuthology.orchestra.run.trial057.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-20T23:39:56.721 INFO:teuthology.orchestra.run.trial057.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-20T23:39:56.721 INFO:teuthology.orchestra.run.trial057.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-20T23:39:56.721 INFO:teuthology.orchestra.run.trial057.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-20T23:39:56.721 INFO:teuthology.orchestra.run.trial057.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-20T23:39:56.721 INFO:teuthology.orchestra.run.trial057.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-20T23:39:56.721 INFO:teuthology.orchestra.run.trial057.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-20T23:39:56.721 INFO:teuthology.orchestra.run.trial057.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-20T23:39:56.721 INFO:teuthology.orchestra.run.trial057.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-20T23:39:56.721 INFO:teuthology.orchestra.run.trial057.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-20T23:39:56.721 INFO:teuthology.orchestra.run.trial057.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-20T23:39:56.722 INFO:teuthology.orchestra.run.trial057.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-20T23:39:56.722 INFO:teuthology.orchestra.run.trial057.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-20T23:39:56.722 INFO:teuthology.orchestra.run.trial057.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-20T23:39:56.722 INFO:teuthology.orchestra.run.trial057.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-20T23:39:56.722 INFO:teuthology.orchestra.run.trial057.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-20T23:39:56.722 INFO:teuthology.orchestra.run.trial057.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-20T23:39:56.722 INFO:teuthology.orchestra.run.trial057.stdout: python3-ceph-argparse x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 45 k 2026-02-20T23:39:56.722 INFO:teuthology.orchestra.run.trial057.stdout: python3-ceph-common x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 185 k 2026-02-20T23:39:56.722 INFO:teuthology.orchestra.run.trial057.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-20T23:39:56.722 INFO:teuthology.orchestra.run.trial057.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-20T23:39:56.722 INFO:teuthology.orchestra.run.trial057.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-20T23:39:56.722 INFO:teuthology.orchestra.run.trial057.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-20T23:39:56.723 INFO:teuthology.orchestra.run.trial057.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-20T23:39:56.723 INFO:teuthology.orchestra.run.trial057.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-20T23:39:56.723 INFO:teuthology.orchestra.run.trial057.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-20T23:39:56.723 INFO:teuthology.orchestra.run.trial057.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-20T23:39:56.723 INFO:teuthology.orchestra.run.trial057.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-20T23:39:56.723 INFO:teuthology.orchestra.run.trial057.stdout: python3-iniconfig noarch 1.1.1-7.el9 appstream 17 k 2026-02-20T23:39:56.723 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-20T23:39:56.723 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-20T23:39:56.724 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-20T23:39:56.724 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-20T23:39:56.724 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-20T23:39:56.724 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-20T23:39:56.724 INFO:teuthology.orchestra.run.trial057.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-20T23:39:56.724 INFO:teuthology.orchestra.run.trial057.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-20T23:39:56.724 INFO:teuthology.orchestra.run.trial057.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-20T23:39:56.724 INFO:teuthology.orchestra.run.trial057.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-20T23:39:56.724 INFO:teuthology.orchestra.run.trial057.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-20T23:39:56.724 INFO:teuthology.orchestra.run.trial057.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-20T23:39:56.724 INFO:teuthology.orchestra.run.trial057.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-20T23:39:56.724 INFO:teuthology.orchestra.run.trial057.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-20T23:39:56.725 INFO:teuthology.orchestra.run.trial057.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-20T23:39:56.725 INFO:teuthology.orchestra.run.trial057.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-20T23:39:56.725 INFO:teuthology.orchestra.run.trial057.stdout: python3-pluggy noarch 0.13.1-7.el9 appstream 41 k 2026-02-20T23:39:56.725 INFO:teuthology.orchestra.run.trial057.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-20T23:39:56.725 INFO:teuthology.orchestra.run.trial057.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-20T23:39:56.725 INFO:teuthology.orchestra.run.trial057.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-20T23:39:56.725 INFO:teuthology.orchestra.run.trial057.stdout: python3-py noarch 1.10.0-6.el9 appstream 477 k 2026-02-20T23:39:56.725 INFO:teuthology.orchestra.run.trial057.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-20T23:39:56.725 INFO:teuthology.orchestra.run.trial057.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-20T23:39:56.725 INFO:teuthology.orchestra.run.trial057.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-20T23:39:56.726 INFO:teuthology.orchestra.run.trial057.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-20T23:39:56.726 INFO:teuthology.orchestra.run.trial057.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-20T23:39:56.726 INFO:teuthology.orchestra.run.trial057.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-20T23:39:56.726 INFO:teuthology.orchestra.run.trial057.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-20T23:39:56.726 INFO:teuthology.orchestra.run.trial057.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-20T23:39:56.726 INFO:teuthology.orchestra.run.trial057.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-20T23:39:56.726 INFO:teuthology.orchestra.run.trial057.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-20T23:39:56.726 INFO:teuthology.orchestra.run.trial057.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-20T23:39:56.726 INFO:teuthology.orchestra.run.trial057.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-20T23:39:56.726 INFO:teuthology.orchestra.run.trial057.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-20T23:39:56.726 INFO:teuthology.orchestra.run.trial057.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-20T23:39:56.726 INFO:teuthology.orchestra.run.trial057.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-20T23:39:56.727 INFO:teuthology.orchestra.run.trial057.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-20T23:39:56.727 INFO:teuthology.orchestra.run.trial057.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-20T23:39:56.727 INFO:teuthology.orchestra.run.trial057.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-20T23:39:56.727 INFO:teuthology.orchestra.run.trial057.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-20T23:39:56.727 INFO:teuthology.orchestra.run.trial057.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-20T23:39:56.727 INFO:teuthology.orchestra.run.trial057.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-20T23:39:56.727 INFO:teuthology.orchestra.run.trial057.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-20T23:39:56.727 INFO:teuthology.orchestra.run.trial057.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-20T23:39:56.727 INFO:teuthology.orchestra.run.trial057.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-20T23:39:56.727 INFO:teuthology.orchestra.run.trial057.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-20T23:39:56.727 INFO:teuthology.orchestra.run.trial057.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-20T23:39:56.727 INFO:teuthology.orchestra.run.trial057.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-20T23:39:56.727 INFO:teuthology.orchestra.run.trial057.stdout:Installing weak dependencies: 2026-02-20T23:39:56.728 INFO:teuthology.orchestra.run.trial057.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-20T23:39:56.728 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:39:56.728 INFO:teuthology.orchestra.run.trial057.stdout:Transaction Summary 2026-02-20T23:39:56.728 INFO:teuthology.orchestra.run.trial057.stdout:======================================================================================= 2026-02-20T23:39:56.728 INFO:teuthology.orchestra.run.trial057.stdout:Install 144 Packages 2026-02-20T23:39:56.728 INFO:teuthology.orchestra.run.trial057.stdout:Upgrade 2 Packages 2026-02-20T23:39:56.728 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:39:56.729 INFO:teuthology.orchestra.run.trial057.stdout:Total download size: 853 M 2026-02-20T23:39:56.729 INFO:teuthology.orchestra.run.trial057.stdout:Downloading Packages: 2026-02-20T23:39:57.760 INFO:teuthology.orchestra.run.trial057.stdout:(1/146): ceph-20.3.0-5343.gbcdd429b.el9.x86_64. 23 kB/s | 6.5 kB 00:00 2026-02-20T23:39:58.868 INFO:teuthology.orchestra.run.trial057.stdout:(2/146): ceph-fuse-20.3.0-5343.gbcdd429b.el9.x8 825 kB/s | 914 kB 00:01 2026-02-20T23:39:59.005 INFO:teuthology.orchestra.run.trial057.stdout:(3/146): ceph-immutable-object-cache-20.3.0-534 1.1 MB/s | 149 kB 00:00 2026-02-20T23:40:00.019 INFO:teuthology.orchestra.run.trial057.stdout:(4/146): ceph-mds-20.3.0-5343.gbcdd429b.el9.x86 2.0 MB/s | 2.0 MB 00:01 2026-02-20T23:40:00.367 INFO:teuthology.orchestra.run.trial057.stdout:(5/146): ceph-mgr-20.3.0-5343.gbcdd429b.el9.x86 2.6 MB/s | 937 kB 00:00 2026-02-20T23:40:03.021 INFO:teuthology.orchestra.run.trial057.stdout:(6/146): ceph-base-20.3.0-5343.gbcdd429b.el9.x8 944 kB/s | 5.1 MB 00:05 2026-02-20T23:40:03.072 INFO:teuthology.orchestra.run.trial057.stdout:(7/146): ceph-mon-20.3.0-5343.gbcdd429b.el9.x86 1.8 MB/s | 4.9 MB 00:02 2026-02-20T23:40:03.150 INFO:teuthology.orchestra.run.trial057.stdout:(8/146): ceph-osd-20.3.0-5343.gbcdd429b.el9.x86 1.2 MB/s | 163 kB 00:00 2026-02-20T23:40:22.886 INFO:teuthology.orchestra.run.trial057.stdout:(9/146): ceph-common-20.3.0-5343.gbcdd429b.el9. 839 kB/s | 21 MB 00:25 2026-02-20T23:40:25.558 INFO:teuthology.orchestra.run.trial057.stdout:(10/146): ceph-osd-classic-20.3.0-5343.gbcdd429 672 kB/s | 15 MB 00:22 2026-02-20T23:40:41.383 INFO:teuthology.orchestra.run.trial057.stdout:(11/146): ceph-radosgw-20.3.0-5343.gbcdd429b.el 723 kB/s | 11 MB 00:15 2026-02-20T23:40:41.450 INFO:teuthology.orchestra.run.trial057.stdout:(12/146): ceph-selinux-20.3.0-5343.gbcdd429b.el 376 kB/s | 25 kB 00:00 2026-02-20T23:41:58.526 INFO:teuthology.orchestra.run.trial057.stdout:(13/146): ceph-osd-crimson-20.3.0-5343.gbcdd429 779 kB/s | 88 MB 01:55 2026-02-20T23:41:58.595 INFO:teuthology.orchestra.run.trial057.stdout:(14/146): libcephfs-devel-20.3.0-5343.gbcdd429b 542 kB/s | 37 kB 00:00 2026-02-20T23:41:58.664 INFO:teuthology.orchestra.run.trial057.stdout:(15/146): libcephfs-proxy2-20.3.0-5343.gbcdd429 374 kB/s | 26 kB 00:00 2026-02-20T23:41:59.694 INFO:teuthology.orchestra.run.trial057.stdout:(16/146): libcephfs2-20.3.0-5343.gbcdd429b.el9. 835 kB/s | 860 kB 00:01 2026-02-20T23:41:59.969 INFO:teuthology.orchestra.run.trial057.stdout:(17/146): libcephsqlite-20.3.0-5343.gbcdd429b.e 558 kB/s | 153 kB 00:00 2026-02-20T23:42:00.334 INFO:teuthology.orchestra.run.trial057.stdout:(18/146): librados-devel-20.3.0-5343.gbcdd429b. 338 kB/s | 124 kB 00:00 2026-02-20T23:42:00.882 INFO:teuthology.orchestra.run.trial057.stdout:(19/146): libradosstriper1-20.3.0-5343.gbcdd429 479 kB/s | 263 kB 00:00 2026-02-20T23:42:12.632 INFO:teuthology.orchestra.run.trial057.stdout:(20/146): librgw2-20.3.0-5343.gbcdd429b.el9.x86 491 kB/s | 5.6 MB 00:11 2026-02-20T23:42:12.771 INFO:teuthology.orchestra.run.trial057.stdout:(21/146): python3-ceph-argparse-20.3.0-5343.gbc 326 kB/s | 45 kB 00:00 2026-02-20T23:42:13.117 INFO:teuthology.orchestra.run.trial057.stdout:(22/146): python3-ceph-common-20.3.0-5343.gbcdd 535 kB/s | 185 kB 00:00 2026-02-20T23:42:13.461 INFO:teuthology.orchestra.run.trial057.stdout:(23/146): python3-cephfs-20.3.0-5343.gbcdd429b. 591 kB/s | 203 kB 00:00 2026-02-20T23:42:13.949 INFO:teuthology.orchestra.run.trial057.stdout:(24/146): python3-rados-20.3.0-5343.gbcdd429b.e 665 kB/s | 324 kB 00:00 2026-02-20T23:42:14.356 INFO:teuthology.orchestra.run.trial057.stdout:(25/146): python3-rbd-20.3.0-5343.gbcdd429b.el9 749 kB/s | 305 kB 00:00 2026-02-20T23:42:14.497 INFO:teuthology.orchestra.run.trial057.stdout:(26/146): python3-rgw-20.3.0-5343.gbcdd429b.el9 713 kB/s | 100 kB 00:00 2026-02-20T23:42:14.703 INFO:teuthology.orchestra.run.trial057.stdout:(27/146): rbd-fuse-20.3.0-5343.gbcdd429b.el9.x8 421 kB/s | 86 kB 00:00 2026-02-20T23:42:19.111 INFO:teuthology.orchestra.run.trial057.stdout:(28/146): rbd-mirror-20.3.0-5343.gbcdd429b.el9. 602 kB/s | 2.6 MB 00:04 2026-02-20T23:42:19.592 INFO:teuthology.orchestra.run.trial057.stdout:(29/146): rbd-nbd-20.3.0-5343.gbcdd429b.el9.x86 346 kB/s | 166 kB 00:00 2026-02-20T23:42:19.791 INFO:teuthology.orchestra.run.trial057.stdout:(30/146): ceph-grafana-dashboards-20.3.0-5343.g 287 kB/s | 57 kB 00:00 2026-02-20T23:42:20.211 INFO:teuthology.orchestra.run.trial057.stdout:(31/146): ceph-mgr-cephadm-20.3.0-5343.gbcdd429 437 kB/s | 184 kB 00:00 2026-02-20T23:43:19.363 INFO:teuthology.orchestra.run.trial057.stdout:(32/146): ceph-test-20.3.0-5343.gbcdd429b.el9.x 819 kB/s | 126 MB 02:37 2026-02-20T23:43:31.289 INFO:teuthology.orchestra.run.trial057.stdout:(33/146): ceph-mgr-diskprediction-local-20.3.0- 634 kB/s | 7.4 MB 00:11 2026-02-20T23:43:31.747 INFO:teuthology.orchestra.run.trial057.stdout:(34/146): ceph-mgr-modules-core-20.3.0-5343.gbc 648 kB/s | 297 kB 00:00 2026-02-20T23:43:32.195 INFO:teuthology.orchestra.run.trial057.stdout:(35/146): ceph-mgr-rook-20.3.0-5343.gbcdd429b.e 112 kB/s | 50 kB 00:00 2026-02-20T23:43:32.265 INFO:teuthology.orchestra.run.trial057.stdout:(36/146): ceph-prometheus-alerts-20.3.0-5343.gb 248 kB/s | 17 kB 00:00 2026-02-20T23:43:32.819 INFO:teuthology.orchestra.run.trial057.stdout:(37/146): ceph-volume-20.3.0-5343.gbcdd429b.el9 544 kB/s | 301 kB 00:00 2026-02-20T23:43:33.283 INFO:teuthology.orchestra.run.trial057.stdout:(38/146): ceph-mgr-dashboard-20.3.0-5343.gbcdd4 710 kB/s | 51 MB 01:13 2026-02-20T23:43:33.643 INFO:teuthology.orchestra.run.trial057.stdout:[MIRROR] cryptsetup-2.8.1-3.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/cryptsetup-2.8.1-3.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-20T23:43:34.058 INFO:teuthology.orchestra.run.trial057.stdout:[MIRROR] cryptsetup-2.8.1-3.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/cryptsetup-2.8.1-3.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-20T23:43:34.110 INFO:teuthology.orchestra.run.trial057.stdout:(39/146): cryptsetup-2.8.1-3.el9.x86_64.rpm 425 kB/s | 351 kB 00:00 2026-02-20T23:43:34.459 INFO:teuthology.orchestra.run.trial057.stdout:[MIRROR] hwloc-libs-2.4.1-6.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/hwloc-libs-2.4.1-6.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-20T23:43:34.642 INFO:teuthology.orchestra.run.trial057.stdout:(40/146): cephadm-20.3.0-5343.gbcdd429b.el9.noa 571 kB/s | 1.0 MB 00:01 2026-02-20T23:43:35.100 INFO:teuthology.orchestra.run.trial057.stdout:(41/146): ledmon-libs-1.1.0-3.el9.x86_64.rpm 88 kB/s | 40 kB 00:00 2026-02-20T23:43:35.199 INFO:teuthology.orchestra.run.trial057.stdout:(42/146): libatomic-11.5.0-14.el9.x86_64.rpm 231 kB/s | 23 kB 00:00 2026-02-20T23:43:35.375 INFO:teuthology.orchestra.run.trial057.stdout:(43/146): libconfig-1.7.2-9.el9.x86_64.rpm 409 kB/s | 72 kB 00:00 2026-02-20T23:43:36.332 INFO:teuthology.orchestra.run.trial057.stdout:(44/146): libgfortran-11.5.0-14.el9.x86_64.rpm 830 kB/s | 794 kB 00:00 2026-02-20T23:43:36.587 INFO:teuthology.orchestra.run.trial057.stdout:(45/146): libquadmath-11.5.0-14.el9.x86_64.rpm 724 kB/s | 184 kB 00:00 2026-02-20T23:43:36.686 INFO:teuthology.orchestra.run.trial057.stdout:[MIRROR] lksctp-tools-1.0.19-2.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/lksctp-tools-1.0.19-2.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-20T23:43:37.024 INFO:teuthology.orchestra.run.trial057.stdout:[MIRROR] lksctp-tools-1.0.19-2.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/lksctp-tools-1.0.19-2.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-20T23:43:37.052 INFO:teuthology.orchestra.run.trial057.stdout:(46/146): lksctp-tools-1.0.19-2.el9.x86_64.rpm 202 kB/s | 94 kB 00:00 2026-02-20T23:43:37.284 INFO:teuthology.orchestra.run.trial057.stdout:(47/146): hwloc-libs-2.4.1-6.el9.x86_64.rpm 677 kB/s | 2.1 MB 00:03 2026-02-20T23:43:37.459 INFO:teuthology.orchestra.run.trial057.stdout:(48/146): pciutils-3.7.0-7.el9.x86_64.rpm 531 kB/s | 93 kB 00:00 2026-02-20T23:43:37.474 INFO:teuthology.orchestra.run.trial057.stdout:(49/146): mailcap-2.1.49-5.el9.noarch.rpm 78 kB/s | 33 kB 00:00 2026-02-20T23:43:37.637 INFO:teuthology.orchestra.run.trial057.stdout:[MIRROR] python3-cryptography-36.0.1-5.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-cryptography-36.0.1-5.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-20T23:43:37.790 INFO:teuthology.orchestra.run.trial057.stdout:(50/146): python3-cffi-1.14.5-5.el9.x86_64.rpm 765 kB/s | 253 kB 00:00 2026-02-20T23:43:37.970 INFO:teuthology.orchestra.run.trial057.stdout:(51/146): python3-ply-3.11-14.el9.noarch.rpm 593 kB/s | 106 kB 00:00 2026-02-20T23:43:37.987 INFO:teuthology.orchestra.run.trial057.stdout:[MIRROR] python3-cryptography-36.0.1-5.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-cryptography-36.0.1-5.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-20T23:43:38.070 INFO:teuthology.orchestra.run.trial057.stdout:[MIRROR] python3-pycparser-2.20-6.el9.noarch.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/python3-pycparser-2.20-6.el9.noarch.rpm (IP: 140.211.166.134) 2026-02-20T23:43:38.072 INFO:teuthology.orchestra.run.trial057.stdout:(52/146): python3-cryptography-36.0.1-5.el9.x86 2.1 MB/s | 1.2 MB 00:00 2026-02-20T23:43:38.642 INFO:teuthology.orchestra.run.trial057.stdout:(53/146): python3-pycparser-2.20-6.el9.noarch.r 201 kB/s | 135 kB 00:00 2026-02-20T23:43:38.654 INFO:teuthology.orchestra.run.trial057.stdout:(54/146): python3-pyparsing-2.4.7-9.el9.noarch. 259 kB/s | 150 kB 00:00 2026-02-20T23:43:38.820 INFO:teuthology.orchestra.run.trial057.stdout:(55/146): python3-requests-2.25.1-10.el9.noarch 711 kB/s | 126 kB 00:00 2026-02-20T23:43:38.988 INFO:teuthology.orchestra.run.trial057.stdout:(56/146): python3-urllib3-1.26.5-7.el9.noarch.r 652 kB/s | 218 kB 00:00 2026-02-20T23:43:39.074 INFO:teuthology.orchestra.run.trial057.stdout:(57/146): unzip-6.0-59.el9.x86_64.rpm 716 kB/s | 182 kB 00:00 2026-02-20T23:43:39.140 INFO:teuthology.orchestra.run.trial057.stdout:(58/146): boost-program-options-1.75.0-13.el9.x 1.5 MB/s | 104 kB 00:00 2026-02-20T23:43:39.173 INFO:teuthology.orchestra.run.trial057.stdout:(59/146): flexiblas-3.0.4-9.el9.x86_64.rpm 910 kB/s | 30 kB 00:00 2026-02-20T23:43:39.271 INFO:teuthology.orchestra.run.trial057.stdout:(60/146): flexiblas-netlib-3.0.4-9.el9.x86_64.r 31 MB/s | 3.0 MB 00:00 2026-02-20T23:43:39.298 INFO:teuthology.orchestra.run.trial057.stdout:(61/146): flexiblas-openblas-openmp-3.0.4-9.el9 554 kB/s | 15 kB 00:00 2026-02-20T23:43:39.334 INFO:teuthology.orchestra.run.trial057.stdout:(62/146): libasan8-15.2.1-7.el9.x86_64.rpm 14 MB/s | 525 kB 00:00 2026-02-20T23:43:39.335 INFO:teuthology.orchestra.run.trial057.stdout:(63/146): zip-3.0-35.el9.x86_64.rpm 766 kB/s | 266 kB 00:00 2026-02-20T23:43:39.363 INFO:teuthology.orchestra.run.trial057.stdout:(64/146): libnbd-1.20.3-4.el9.x86_64.rpm 5.5 MB/s | 164 kB 00:00 2026-02-20T23:43:39.370 INFO:teuthology.orchestra.run.trial057.stdout:(65/146): libpmemobj-1.12.1-1.el9.x86_64.rpm 4.6 MB/s | 160 kB 00:00 2026-02-20T23:43:39.390 INFO:teuthology.orchestra.run.trial057.stdout:(66/146): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.7 MB/s | 45 kB 00:00 2026-02-20T23:43:39.414 INFO:teuthology.orchestra.run.trial057.stdout:(67/146): librdkafka-1.6.1-102.el9.x86_64.rpm 15 MB/s | 662 kB 00:00 2026-02-20T23:43:39.420 INFO:teuthology.orchestra.run.trial057.stdout:(68/146): libstoragemgmt-1.10.1-1.el9.x86_64.rp 8.0 MB/s | 246 kB 00:00 2026-02-20T23:43:39.437 INFO:teuthology.orchestra.run.trial057.stdout:(69/146): libubsan-11.5.0-14.el9.x86_64.rpm 7.7 MB/s | 176 kB 00:00 2026-02-20T23:43:39.444 INFO:teuthology.orchestra.run.trial057.stdout:(70/146): libxslt-1.1.34-12.el9.x86_64.rpm 9.6 MB/s | 233 kB 00:00 2026-02-20T23:43:39.462 INFO:teuthology.orchestra.run.trial057.stdout:(71/146): lttng-ust-2.12.0-6.el9.x86_64.rpm 11 MB/s | 292 kB 00:00 2026-02-20T23:43:39.471 INFO:teuthology.orchestra.run.trial057.stdout:(72/146): lua-5.4.4-4.el9.x86_64.rpm 7.0 MB/s | 188 kB 00:00 2026-02-20T23:43:39.486 INFO:teuthology.orchestra.run.trial057.stdout:(73/146): openblas-0.3.29-1.el9.x86_64.rpm 1.7 MB/s | 42 kB 00:00 2026-02-20T23:43:39.527 INFO:teuthology.orchestra.run.trial057.stdout:(74/146): protobuf-3.14.0-17.el9.x86_64.rpm 25 MB/s | 1.0 MB 00:00 2026-02-20T23:43:39.534 INFO:teuthology.orchestra.run.trial057.stdout:(75/146): openblas-openmp-0.3.29-1.el9.x86_64.r 84 MB/s | 5.3 MB 00:00 2026-02-20T23:43:39.559 INFO:teuthology.orchestra.run.trial057.stdout:(76/146): python3-devel-3.9.25-3.el9.x86_64.rpm 9.6 MB/s | 244 kB 00:00 2026-02-20T23:43:39.590 INFO:teuthology.orchestra.run.trial057.stdout:(77/146): python3-babel-2.9.1-2.el9.noarch.rpm 96 MB/s | 6.0 MB 00:00 2026-02-20T23:43:39.591 INFO:teuthology.orchestra.run.trial057.stdout:(78/146): python3-iniconfig-1.1.1-7.el9.noarch. 560 kB/s | 17 kB 00:00 2026-02-20T23:43:39.615 INFO:teuthology.orchestra.run.trial057.stdout:(79/146): python3-jinja2-2.11.3-8.el9.noarch.rp 9.7 MB/s | 249 kB 00:00 2026-02-20T23:43:39.619 INFO:teuthology.orchestra.run.trial057.stdout:(80/146): python3-jmespath-1.0.1-1.el9.noarch.r 1.7 MB/s | 48 kB 00:00 2026-02-20T23:43:39.642 INFO:teuthology.orchestra.run.trial057.stdout:(81/146): python3-libstoragemgmt-1.10.1-1.el9.x 6.6 MB/s | 177 kB 00:00 2026-02-20T23:43:39.642 INFO:teuthology.orchestra.run.trial057.stdout:(82/146): python3-markupsafe-1.1.1-12.el9.x86_6 1.5 MB/s | 35 kB 00:00 2026-02-20T23:43:39.669 INFO:teuthology.orchestra.run.trial057.stdout:(83/146): python3-numpy-f2py-1.23.5-2.el9.x86_6 16 MB/s | 442 kB 00:00 2026-02-20T23:43:39.702 INFO:teuthology.orchestra.run.trial057.stdout:(84/146): python3-numpy-1.23.5-2.el9.x86_64.rpm 103 MB/s | 6.1 MB 00:00 2026-02-20T23:43:39.702 INFO:teuthology.orchestra.run.trial057.stdout:(85/146): python3-packaging-20.9-5.el9.noarch.r 2.3 MB/s | 77 kB 00:00 2026-02-20T23:43:39.725 INFO:teuthology.orchestra.run.trial057.stdout:(86/146): python3-pluggy-0.13.1-7.el9.noarch.rp 1.8 MB/s | 41 kB 00:00 2026-02-20T23:43:39.730 INFO:teuthology.orchestra.run.trial057.stdout:(87/146): python3-protobuf-3.14.0-17.el9.noarch 9.4 MB/s | 267 kB 00:00 2026-02-20T23:43:39.750 INFO:teuthology.orchestra.run.trial057.stdout:(88/146): python3-py-1.10.0-6.el9.noarch.rpm 18 MB/s | 477 kB 00:00 2026-02-20T23:43:39.752 INFO:teuthology.orchestra.run.trial057.stdout:(89/146): python3-pyasn1-0.4.8-6.el9.noarch.rpm 7.1 MB/s | 159 kB 00:00 2026-02-20T23:43:39.777 INFO:teuthology.orchestra.run.trial057.stdout:(90/146): python3-pyasn1-modules-0.4.8-6.el9.no 10 MB/s | 279 kB 00:00 2026-02-20T23:43:39.780 INFO:teuthology.orchestra.run.trial057.stdout:(91/146): python3-pytest-6.2.2-7.el9.noarch.rpm 18 MB/s | 519 kB 00:00 2026-02-20T23:43:39.800 INFO:teuthology.orchestra.run.trial057.stdout:(92/146): python3-requests-oauthlib-1.3.0-12.el 2.2 MB/s | 54 kB 00:00 2026-02-20T23:43:39.835 INFO:teuthology.orchestra.run.trial057.stdout:(93/146): python3-toml-0.10.2-6.el9.noarch.rpm 1.2 MB/s | 42 kB 00:00 2026-02-20T23:43:39.860 INFO:teuthology.orchestra.run.trial057.stdout:(94/146): qatlib-25.08.0-2.el9.x86_64.rpm 9.6 MB/s | 240 kB 00:00 2026-02-20T23:43:39.908 INFO:teuthology.orchestra.run.trial057.stdout:(95/146): python3-scipy-1.9.3-2.el9.x86_64.rpm 151 MB/s | 19 MB 00:00 2026-02-20T23:43:39.909 INFO:teuthology.orchestra.run.trial057.stdout:(96/146): qatlib-service-25.08.0-2.el9.x86_64.r 772 kB/s | 37 kB 00:00 2026-02-20T23:43:39.934 INFO:teuthology.orchestra.run.trial057.stdout:(97/146): qatzip-libs-1.3.1-1.el9.x86_64.rpm 2.5 MB/s | 66 kB 00:00 2026-02-20T23:43:39.937 INFO:teuthology.orchestra.run.trial057.stdout:(98/146): socat-1.7.4.1-8.el9.x86_64.rpm 10 MB/s | 303 kB 00:00 2026-02-20T23:43:39.957 INFO:teuthology.orchestra.run.trial057.stdout:(99/146): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.7 MB/s | 64 kB 00:00 2026-02-20T23:43:39.959 INFO:teuthology.orchestra.run.trial057.stdout:(100/146): lua-devel-5.4.4-4.el9.x86_64.rpm 1.0 MB/s | 22 kB 00:00 2026-02-20T23:43:40.020 INFO:teuthology.orchestra.run.trial057.stdout:(101/146): protobuf-compiler-3.14.0-17.el9.x86_ 13 MB/s | 862 kB 00:00 2026-02-20T23:43:40.223 INFO:teuthology.orchestra.run.trial057.stdout:(102/146): abseil-cpp-20211102.0-4.el9.x86_64.r 2.0 MB/s | 551 kB 00:00 2026-02-20T23:43:40.243 INFO:teuthology.orchestra.run.trial057.stdout:(103/146): gperftools-libs-2.9.1-3.el9.x86_64.r 1.3 MB/s | 308 kB 00:00 2026-02-20T23:43:40.256 INFO:teuthology.orchestra.run.trial057.stdout:(104/146): grpc-data-1.46.7-10.el9.noarch.rpm 579 kB/s | 19 kB 00:00 2026-02-20T23:43:40.290 INFO:teuthology.orchestra.run.trial057.stdout:(105/146): libarrow-doc-9.0.0-15.el9.noarch.rpm 744 kB/s | 25 kB 00:00 2026-02-20T23:43:40.324 INFO:teuthology.orchestra.run.trial057.stdout:(106/146): liboath-2.6.12-1.el9.x86_64.rpm 1.4 MB/s | 49 kB 00:00 2026-02-20T23:43:40.359 INFO:teuthology.orchestra.run.trial057.stdout:(107/146): libunwind-1.6.2-1.el9.x86_64.rpm 1.9 MB/s | 67 kB 00:00 2026-02-20T23:43:40.398 INFO:teuthology.orchestra.run.trial057.stdout:(108/146): luarocks-3.9.2-5.el9.noarch.rpm 3.8 MB/s | 151 kB 00:00 2026-02-20T23:43:40.424 INFO:teuthology.orchestra.run.trial057.stdout:(109/146): libarrow-9.0.0-15.el9.x86_64.rpm 24 MB/s | 4.4 MB 00:00 2026-02-20T23:43:40.466 INFO:teuthology.orchestra.run.trial057.stdout:(110/146): parquet-libs-9.0.0-15.el9.x86_64.rpm 12 MB/s | 838 kB 00:00 2026-02-20T23:43:40.470 INFO:teuthology.orchestra.run.trial057.stdout:(111/146): python3-asyncssh-2.13.2-5.el9.noarch 11 MB/s | 548 kB 00:00 2026-02-20T23:43:40.501 INFO:teuthology.orchestra.run.trial057.stdout:(112/146): python3-autocommand-2.2.2-8.el9.noar 849 kB/s | 29 kB 00:00 2026-02-20T23:43:40.506 INFO:teuthology.orchestra.run.trial057.stdout:(113/146): python3-backports-tarfile-1.2.0-1.el 1.7 MB/s | 60 kB 00:00 2026-02-20T23:43:40.536 INFO:teuthology.orchestra.run.trial057.stdout:(114/146): python3-bcrypt-3.2.2-1.el9.x86_64.rp 1.2 MB/s | 43 kB 00:00 2026-02-20T23:43:40.546 INFO:teuthology.orchestra.run.trial057.stdout:(115/146): python3-cachetools-4.2.4-1.el9.noarc 803 kB/s | 32 kB 00:00 2026-02-20T23:43:40.570 INFO:teuthology.orchestra.run.trial057.stdout:(116/146): python3-certifi-2023.05.07-4.el9.noa 418 kB/s | 14 kB 00:00 2026-02-20T23:43:40.588 INFO:teuthology.orchestra.run.trial057.stdout:(117/146): python3-cheroot-10.0.1-4.el9.noarch. 4.0 MB/s | 173 kB 00:00 2026-02-20T23:43:40.608 INFO:teuthology.orchestra.run.trial057.stdout:(118/146): python3-cherrypy-18.6.1-2.el9.noarch 9.2 MB/s | 358 kB 00:00 2026-02-20T23:43:40.632 INFO:teuthology.orchestra.run.trial057.stdout:(119/146): python3-google-auth-2.45.0-1.el9.noa 5.7 MB/s | 254 kB 00:00 2026-02-20T23:43:40.668 INFO:teuthology.orchestra.run.trial057.stdout:(120/146): python3-grpcio-tools-1.46.7-10.el9.x 3.9 MB/s | 144 kB 00:00 2026-02-20T23:43:40.682 INFO:teuthology.orchestra.run.trial057.stdout:(121/146): python3-grpcio-1.46.7-10.el9.x86_64. 27 MB/s | 2.0 MB 00:00 2026-02-20T23:43:40.710 INFO:teuthology.orchestra.run.trial057.stdout:(122/146): python3-jaraco-8.2.1-3.el9.noarch.rp 253 kB/s | 11 kB 00:00 2026-02-20T23:43:40.716 INFO:teuthology.orchestra.run.trial057.stdout:(123/146): python3-jaraco-classes-3.2.1-5.el9.n 523 kB/s | 18 kB 00:00 2026-02-20T23:43:40.750 INFO:teuthology.orchestra.run.trial057.stdout:(124/146): python3-jaraco-context-6.0.1-3.el9.n 581 kB/s | 20 kB 00:00 2026-02-20T23:43:40.752 INFO:teuthology.orchestra.run.trial057.stdout:(125/146): python3-jaraco-collections-3.0.0-8.e 553 kB/s | 23 kB 00:00 2026-02-20T23:43:40.784 INFO:teuthology.orchestra.run.trial057.stdout:(126/146): python3-jaraco-functools-3.5.0-2.el9 574 kB/s | 19 kB 00:00 2026-02-20T23:43:40.790 INFO:teuthology.orchestra.run.trial057.stdout:(127/146): python3-jaraco-text-4.0.0-2.el9.noar 714 kB/s | 26 kB 00:00 2026-02-20T23:43:40.828 INFO:teuthology.orchestra.run.trial057.stdout:(128/146): python3-more-itertools-8.12.0-2.el9. 2.0 MB/s | 79 kB 00:00 2026-02-20T23:43:40.839 INFO:teuthology.orchestra.run.trial057.stdout:(129/146): python3-kubernetes-26.1.0-3.el9.noar 19 MB/s | 1.0 MB 00:00 2026-02-20T23:43:40.863 INFO:teuthology.orchestra.run.trial057.stdout:(130/146): python3-natsort-7.1.1-5.el9.noarch.r 1.6 MB/s | 58 kB 00:00 2026-02-20T23:43:40.873 INFO:teuthology.orchestra.run.trial057.stdout:(131/146): python3-portend-3.1.0-2.el9.noarch.r 484 kB/s | 16 kB 00:00 2026-02-20T23:43:40.900 INFO:teuthology.orchestra.run.trial057.stdout:(132/146): python3-pyOpenSSL-21.0.0-1.el9.noarc 2.4 MB/s | 90 kB 00:00 2026-02-20T23:43:40.907 INFO:teuthology.orchestra.run.trial057.stdout:(133/146): python3-repoze-lru-0.7-16.el9.noarch 905 kB/s | 31 kB 00:00 2026-02-20T23:43:40.939 INFO:teuthology.orchestra.run.trial057.stdout:(134/146): python3-routes-2.5.1-5.el9.noarch.rp 4.7 MB/s | 188 kB 00:00 2026-02-20T23:43:40.942 INFO:teuthology.orchestra.run.trial057.stdout:(135/146): python3-rsa-4.9-2.el9.noarch.rpm 1.6 MB/s | 59 kB 00:00 2026-02-20T23:43:40.973 INFO:teuthology.orchestra.run.trial057.stdout:(136/146): python3-tempora-5.0.0-2.el9.noarch.r 1.0 MB/s | 36 kB 00:00 2026-02-20T23:43:40.977 INFO:teuthology.orchestra.run.trial057.stdout:(137/146): python3-typing-extensions-4.15.0-1.e 2.4 MB/s | 86 kB 00:00 2026-02-20T23:43:41.011 INFO:teuthology.orchestra.run.trial057.stdout:(138/146): python3-websocket-client-1.2.3-2.el9 2.4 MB/s | 90 kB 00:00 2026-02-20T23:43:41.011 INFO:teuthology.orchestra.run.trial057.stdout:(139/146): python3-xmltodict-0.12.0-15.el9.noar 654 kB/s | 22 kB 00:00 2026-02-20T23:43:41.045 INFO:teuthology.orchestra.run.trial057.stdout:(140/146): python3-zc-lockfile-2.0-10.el9.noarc 585 kB/s | 20 kB 00:00 2026-02-20T23:43:41.048 INFO:teuthology.orchestra.run.trial057.stdout:(141/146): re2-20211101-20.el9.x86_64.rpm 5.1 MB/s | 191 kB 00:00 2026-02-20T23:43:41.085 INFO:teuthology.orchestra.run.trial057.stdout:(142/146): yaml-cpp-0.6.3-5.el9.x86_64.rpm 3.2 MB/s | 122 kB 00:00 2026-02-20T23:43:41.088 INFO:teuthology.orchestra.run.trial057.stdout:(143/146): thrift-0.15.0-4.el9.x86_64.rpm 37 MB/s | 1.6 MB 00:00 2026-02-20T23:43:43.127 INFO:teuthology.orchestra.run.trial057.stdout:(144/146): librbd1-20.3.0-5343.gbcdd429b.el9.x8 1.3 MB/s | 2.7 MB 00:02 2026-02-20T23:43:45.828 INFO:teuthology.orchestra.run.trial057.stdout:(145/146): librados2-20.3.0-5343.gbcdd429b.el9. 709 kB/s | 3.3 MB 00:04 2026-02-20T23:48:38.165 INFO:teuthology.orchestra.run.trial057.stdout:(146/146): ceph-osd-crimson-debuginfo-20.3.0-53 896 kB/s | 433 MB 08:15 2026-02-20T23:48:38.172 INFO:teuthology.orchestra.run.trial057.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:48:38.172 INFO:teuthology.orchestra.run.trial057.stdout:Total 1.6 MB/s | 853 MB 08:41 2026-02-20T23:48:38.687 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction check 2026-02-20T23:48:38.734 INFO:teuthology.orchestra.run.trial057.stdout:Transaction check succeeded. 2026-02-20T23:48:38.734 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction test 2026-02-20T23:48:41.420 INFO:teuthology.orchestra.run.trial057.stdout:Transaction test succeeded. 2026-02-20T23:48:41.420 INFO:teuthology.orchestra.run.trial057.stdout:Running transaction 2026-02-20T23:48:44.180 INFO:teuthology.orchestra.run.trial057.stdout: Preparing : 1/1 2026-02-20T23:48:44.183 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/148 2026-02-20T23:48:44.192 INFO:teuthology.orchestra.run.trial057.stdout: Installing : liboath-2.6.12-1.el9.x86_64 2/148 2026-02-20T23:48:44.204 INFO:teuthology.orchestra.run.trial057.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/148 2026-02-20T23:48:44.356 INFO:teuthology.orchestra.run.trial057.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 4/148 2026-02-20T23:48:44.358 INFO:teuthology.orchestra.run.trial057.stdout: Upgrading : librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 5/148 2026-02-20T23:48:44.384 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 5/148 2026-02-20T23:48:44.392 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 6/148 2026-02-20T23:48:44.395 INFO:teuthology.orchestra.run.trial057.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 7/148 2026-02-20T23:48:44.398 INFO:teuthology.orchestra.run.trial057.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 8/148 2026-02-20T23:48:44.399 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 9/148 2026-02-20T23:48:44.405 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 10/148 2026-02-20T23:48:44.526 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 11/148 2026-02-20T23:48:44.528 INFO:teuthology.orchestra.run.trial057.stdout: Upgrading : librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 12/148 2026-02-20T23:48:44.545 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 12/148 2026-02-20T23:48:44.546 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libcephsqlite-2:20.3.0-5343.gbcdd429b.el9.x86_64 13/148 2026-02-20T23:48:44.568 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5343.gbcdd429b.el9.x86_64 13/148 2026-02-20T23:48:44.569 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libradosstriper1-2:20.3.0-5343.gbcdd429b.el9.x86 14/148 2026-02-20T23:48:44.581 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5343.gbcdd429b.el9.x86 14/148 2026-02-20T23:48:44.585 INFO:teuthology.orchestra.run.trial057.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 15/148 2026-02-20T23:48:44.617 INFO:teuthology.orchestra.run.trial057.stdout: Installing : re2-1:20211101-20.el9.x86_64 16/148 2026-02-20T23:48:44.622 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 17/148 2026-02-20T23:48:44.627 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 18/148 2026-02-20T23:48:44.648 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 19/148 2026-02-20T23:48:44.654 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 20/148 2026-02-20T23:48:44.658 INFO:teuthology.orchestra.run.trial057.stdout: Installing : lua-5.4.4-4.el9.x86_64 21/148 2026-02-20T23:48:44.662 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 22/148 2026-02-20T23:48:44.663 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 23/148 2026-02-20T23:48:44.675 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 23/148 2026-02-20T23:48:44.680 INFO:teuthology.orchestra.run.trial057.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 24/148 2026-02-20T23:48:44.703 INFO:teuthology.orchestra.run.trial057.stdout: Installing : unzip-6.0-59.el9.x86_64 25/148 2026-02-20T23:48:44.715 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 26/148 2026-02-20T23:48:44.722 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 27/148 2026-02-20T23:48:44.725 INFO:teuthology.orchestra.run.trial057.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 28/148 2026-02-20T23:48:44.731 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 29/148 2026-02-20T23:48:44.733 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 30/148 2026-02-20T23:48:44.734 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 31/148 2026-02-20T23:48:44.745 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 32/148 2026-02-20T23:48:44.779 INFO:teuthology.orchestra.run.trial057.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 33/148 2026-02-20T23:48:44.783 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-ceph-common-2:20.3.0-5343.gbcdd429b.el9. 34/148 2026-02-20T23:48:44.785 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-ceph-argparse-2:20.3.0-5343.gbcdd429b.el 35/148 2026-02-20T23:48:44.786 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libcephfs-proxy2-2:20.3.0-5343.gbcdd429b.el9.x86 36/148 2026-02-20T23:48:44.831 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5343.gbcdd429b.el9.x86 36/148 2026-02-20T23:48:44.832 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 37/148 2026-02-20T23:48:44.853 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 37/148 2026-02-20T23:48:44.865 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_6 38/148 2026-02-20T23:48:44.871 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 39/148 2026-02-20T23:48:44.896 INFO:teuthology.orchestra.run.trial057.stdout: Installing : zip-3.0-35.el9.x86_64 40/148 2026-02-20T23:48:44.899 INFO:teuthology.orchestra.run.trial057.stdout: Installing : luarocks-3.9.2-5.el9.noarch 41/148 2026-02-20T23:48:44.950 INFO:teuthology.orchestra.run.trial057.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 42/148 2026-02-20T23:48:44.962 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 43/148 2026-02-20T23:48:44.969 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-rsa-4.9-2.el9.noarch 44/148 2026-02-20T23:48:44.973 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 45/148 2026-02-20T23:48:44.990 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 46/148 2026-02-20T23:48:44.995 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 47/148 2026-02-20T23:48:45.003 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 48/148 2026-02-20T23:48:45.011 INFO:teuthology.orchestra.run.trial057.stdout: Installing : librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_6 49/148 2026-02-20T23:48:45.023 INFO:teuthology.orchestra.run.trial057.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 50/148 2026-02-20T23:48:45.027 INFO:teuthology.orchestra.run.trial057.stdout: Installing : thrift-0.15.0-4.el9.x86_64 51/148 2026-02-20T23:48:45.031 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 52/148 2026-02-20T23:48:45.045 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 53/148 2026-02-20T23:48:45.049 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 54/148 2026-02-20T23:48:45.054 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 55/148 2026-02-20T23:48:45.066 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 56/148 2026-02-20T23:48:45.075 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 57/148 2026-02-20T23:48:45.079 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 58/148 2026-02-20T23:48:45.087 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 59/148 2026-02-20T23:48:45.127 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 60/148 2026-02-20T23:48:45.451 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 61/148 2026-02-20T23:48:45.461 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 62/148 2026-02-20T23:48:45.466 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 63/148 2026-02-20T23:48:45.471 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 64/148 2026-02-20T23:48:45.477 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 65/148 2026-02-20T23:48:45.480 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 66/148 2026-02-20T23:48:45.483 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 67/148 2026-02-20T23:48:45.491 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 68/148 2026-02-20T23:48:45.493 INFO:teuthology.orchestra.run.trial057.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 69/148 2026-02-20T23:48:45.494 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 70/148 2026-02-20T23:48:45.524 INFO:teuthology.orchestra.run.trial057.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 71/148 2026-02-20T23:48:45.568 INFO:teuthology.orchestra.run.trial057.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 72/148 2026-02-20T23:48:45.577 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 73/148 2026-02-20T23:48:45.624 INFO:teuthology.orchestra.run.trial057.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 74/148 2026-02-20T23:48:45.653 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-py-1.10.0-6.el9.noarch 75/148 2026-02-20T23:48:45.663 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 76/148 2026-02-20T23:48:45.670 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 77/148 2026-02-20T23:48:45.675 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-pluggy-0.13.1-7.el9.noarch 78/148 2026-02-20T23:48:45.682 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 79/148 2026-02-20T23:48:45.686 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 80/148 2026-02-20T23:48:45.720 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-iniconfig-1.1.1-7.el9.noarch 81/148 2026-02-20T23:48:45.945 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 82/148 2026-02-20T23:48:45.968 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 83/148 2026-02-20T23:48:45.972 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 84/148 2026-02-20T23:48:46.031 INFO:teuthology.orchestra.run.trial057.stdout: Installing : openblas-0.3.29-1.el9.x86_64 85/148 2026-02-20T23:48:46.033 INFO:teuthology.orchestra.run.trial057.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 86/148 2026-02-20T23:48:46.053 INFO:teuthology.orchestra.run.trial057.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 87/148 2026-02-20T23:48:46.380 INFO:teuthology.orchestra.run.trial057.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 88/148 2026-02-20T23:48:46.453 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 89/148 2026-02-20T23:48:47.128 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 90/148 2026-02-20T23:48:47.146 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 91/148 2026-02-20T23:48:47.149 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 92/148 2026-02-20T23:48:47.152 INFO:teuthology.orchestra.run.trial057.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 93/148 2026-02-20T23:48:47.158 INFO:teuthology.orchestra.run.trial057.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 94/148 2026-02-20T23:48:47.418 INFO:teuthology.orchestra.run.trial057.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 95/148 2026-02-20T23:48:47.419 INFO:teuthology.orchestra.run.trial057.stdout: Installing : librgw2-2:20.3.0-5343.gbcdd429b.el9.x86_64 96/148 2026-02-20T23:48:47.437 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: librgw2-2:20.3.0-5343.gbcdd429b.el9.x86_64 96/148 2026-02-20T23:48:47.438 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 97/148 2026-02-20T23:48:48.446 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 98/148 2026-02-20T23:48:48.448 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 98/148 2026-02-20T23:48:48.465 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 98/148 2026-02-20T23:48:48.475 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 99/148 2026-02-20T23:48:48.481 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-packaging-20.9-5.el9.noarch 100/148 2026-02-20T23:48:48.495 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-ply-3.11-14.el9.noarch 101/148 2026-02-20T23:48:48.511 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 102/148 2026-02-20T23:48:48.586 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 103/148 2026-02-20T23:48:48.595 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 104/148 2026-02-20T23:48:48.618 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 105/148 2026-02-20T23:48:48.648 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 106/148 2026-02-20T23:48:48.699 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 107/148 2026-02-20T23:48:48.705 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 108/148 2026-02-20T23:48:48.708 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 109/148 2026-02-20T23:48:48.714 INFO:teuthology.orchestra.run.trial057.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 110/148 2026-02-20T23:48:48.717 INFO:teuthology.orchestra.run.trial057.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 111/148 2026-02-20T23:48:48.718 INFO:teuthology.orchestra.run.trial057.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 112/148 2026-02-20T23:48:48.735 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 112/148 2026-02-20T23:48:48.991 INFO:teuthology.orchestra.run.trial057.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 113/148 2026-02-20T23:48:48.995 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 114/148 2026-02-20T23:48:49.035 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 114/148 2026-02-20T23:48:49.035 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-20T23:48:49.035 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-20T23:48:49.036 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:48:49.038 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-selinux-2:20.3.0-5343.gbcdd429b.el9.x86_64 115/148 2026-02-20T23:48:53.687 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5343.gbcdd429b.el9.x86_64 115/148 2026-02-20T23:48:53.688 INFO:teuthology.orchestra.run.trial057.stdout:skipping the directory /sys 2026-02-20T23:48:53.688 INFO:teuthology.orchestra.run.trial057.stdout:skipping the directory /proc 2026-02-20T23:48:53.688 INFO:teuthology.orchestra.run.trial057.stdout:skipping the directory /mnt 2026-02-20T23:48:53.688 INFO:teuthology.orchestra.run.trial057.stdout:skipping the directory /var/tmp 2026-02-20T23:48:53.688 INFO:teuthology.orchestra.run.trial057.stdout:skipping the directory /home 2026-02-20T23:48:53.688 INFO:teuthology.orchestra.run.trial057.stdout:skipping the directory /root 2026-02-20T23:48:53.688 INFO:teuthology.orchestra.run.trial057.stdout:skipping the directory /tmp 2026-02-20T23:48:53.688 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:48:53.781 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mds-2:20.3.0-5343.gbcdd429b.el9.x86_64 116/148 2026-02-20T23:48:53.804 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-mds-2:20.3.0-5343.gbcdd429b.el9.x86_64 116/148 2026-02-20T23:48:53.804 INFO:teuthology.orchestra.run.trial057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:48:53.804 INFO:teuthology.orchestra.run.trial057.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-20T23:48:53.804 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T23:48:53.804 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T23:48:53.804 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:48:54.023 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mon-2:20.3.0-5343.gbcdd429b.el9.x86_64 117/148 2026-02-20T23:48:54.043 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-mon-2:20.3.0-5343.gbcdd429b.el9.x86_64 117/148 2026-02-20T23:48:54.043 INFO:teuthology.orchestra.run.trial057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:48:54.043 INFO:teuthology.orchestra.run.trial057.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-20T23:48:54.043 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T23:48:54.044 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T23:48:54.044 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:48:54.050 INFO:teuthology.orchestra.run.trial057.stdout: Installing : mailcap-2.1.49-5.el9.noarch 118/148 2026-02-20T23:48:54.051 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 119/148 2026-02-20T23:48:54.068 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 120/148 2026-02-20T23:48:54.068 INFO:teuthology.orchestra.run.trial057.stdout:Creating group 'qat' with GID 992. 2026-02-20T23:48:54.068 INFO:teuthology.orchestra.run.trial057.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-20T23:48:54.068 INFO:teuthology.orchestra.run.trial057.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-20T23:48:54.068 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:48:54.075 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 120/148 2026-02-20T23:48:54.098 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 120/148 2026-02-20T23:48:54.098 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-20T23:48:54.098 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:48:54.131 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 121/148 2026-02-20T23:48:54.753 INFO:teuthology.orchestra.run.trial057.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 122/148 2026-02-20T23:49:00.362 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86 123/148 2026-02-20T23:49:00.426 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86 124/148 2026-02-20T23:49:00.428 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch 125/148 2026-02-20T23:49:00.441 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch 125/148 2026-02-20T23:49:00.441 INFO:teuthology.orchestra.run.trial057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:49:00.441 INFO:teuthology.orchestra.run.trial057.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-20T23:49:00.441 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:00.452 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 126/148 2026-02-20T23:49:00.474 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 126/148 2026-02-20T23:49:00.474 INFO:teuthology.orchestra.run.trial057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:49:00.474 INFO:teuthology.orchestra.run.trial057.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-20T23:49:00.474 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T23:49:00.475 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T23:49:00.475 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:00.529 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 127/148 2026-02-20T23:49:00.531 INFO:teuthology.orchestra.run.trial057.stdout: Installing : cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 127/148 2026-02-20T23:49:00.540 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5343.gbcdd429b.e 128/148 2026-02-20T23:49:00.564 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5343.gbcdd429b. 129/148 2026-02-20T23:49:00.565 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noa 130/148 2026-02-20T23:49:03.570 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noa 130/148 2026-02-20T23:49:03.577 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.n 131/148 2026-02-20T23:49:04.043 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.n 131/148 2026-02-20T23:49:04.044 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcd 132/148 2026-02-20T23:49:04.108 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcd 132/148 2026-02-20T23:49:04.148 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5343.gbcdd429b.el 133/148 2026-02-20T23:49:04.149 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 134/148 2026-02-20T23:49:04.170 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 134/148 2026-02-20T23:49:04.170 INFO:teuthology.orchestra.run.trial057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:49:04.170 INFO:teuthology.orchestra.run.trial057.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-20T23:49:04.170 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T23:49:04.170 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T23:49:04.171 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:04.182 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch 135/148 2026-02-20T23:49:04.191 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch 135/148 2026-02-20T23:49:04.705 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 136/148 2026-02-20T23:49:04.707 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 137/148 2026-02-20T23:49:04.729 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 137/148 2026-02-20T23:49:04.729 INFO:teuthology.orchestra.run.trial057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:49:04.729 INFO:teuthology.orchestra.run.trial057.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-20T23:49:04.729 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T23:49:04.730 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T23:49:04.730 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:04.738 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5343.gbcdd4 138/148 2026-02-20T23:49:04.757 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5343.gbcdd4 138/148 2026-02-20T23:49:04.758 INFO:teuthology.orchestra.run.trial057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:49:04.758 INFO:teuthology.orchestra.run.trial057.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-20T23:49:04.758 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:04.871 INFO:teuthology.orchestra.run.trial057.stdout: Installing : rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 139/148 2026-02-20T23:49:04.891 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 139/148 2026-02-20T23:49:04.892 INFO:teuthology.orchestra.run.trial057.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:49:04.892 INFO:teuthology.orchestra.run.trial057.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-20T23:49:04.892 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T23:49:04.892 INFO:teuthology.orchestra.run.trial057.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T23:49:04.892 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:12.108 INFO:teuthology.orchestra.run.trial057.stdout: Installing : python3-pytest-6.2.2-7.el9.noarch 140/148 2026-02-20T23:49:12.116 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 141/148 2026-02-20T23:49:12.121 INFO:teuthology.orchestra.run.trial057.stdout: Installing : libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_ 142/148 2026-02-20T23:49:12.129 INFO:teuthology.orchestra.run.trial057.stdout: Installing : rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 143/148 2026-02-20T23:49:12.172 INFO:teuthology.orchestra.run.trial057.stdout: Installing : rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 144/148 2026-02-20T23:49:35.468 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 145/148 2026-02-20T23:49:35.471 INFO:teuthology.orchestra.run.trial057.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5343.gbcdd42 146/148 2026-02-20T23:49:35.471 INFO:teuthology.orchestra.run.trial057.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 147/148 2026-02-20T23:49:35.483 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 147/148 2026-02-20T23:49:35.483 INFO:teuthology.orchestra.run.trial057.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 148/148 2026-02-20T23:49:35.494 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 148/148 2026-02-20T23:49:35.500 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86 148/148 2026-02-20T23:49:35.524 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86 148/148 2026-02-20T23:49:38.754 INFO:teuthology.orchestra.run.trial057.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 148/148 2026-02-20T23:49:38.754 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/148 2026-02-20T23:49:38.754 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/148 2026-02-20T23:49:38.754 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 3/148 2026-02-20T23:49:38.755 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 4/148 2026-02-20T23:49:38.755 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5343.gbcdd4 5/148 2026-02-20T23:49:38.755 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mds-2:20.3.0-5343.gbcdd429b.el9.x86_64 6/148 2026-02-20T23:49:38.755 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 7/148 2026-02-20T23:49:38.755 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mon-2:20.3.0-5343.gbcdd429b.el9.x86_64 8/148 2026-02-20T23:49:38.755 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 9/148 2026-02-20T23:49:38.755 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86 10/148 2026-02-20T23:49:38.755 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86 11/148 2026-02-20T23:49:38.755 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5343.gbcdd42 12/148 2026-02-20T23:49:38.755 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 13/148 2026-02-20T23:49:38.755 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-selinux-2:20.3.0-5343.gbcdd429b.el9.x86_64 14/148 2026-02-20T23:49:38.755 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 15/148 2026-02-20T23:49:38.755 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_ 16/148 2026-02-20T23:49:38.756 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5343.gbcdd429b.el9.x86 17/148 2026-02-20T23:49:38.756 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 18/148 2026-02-20T23:49:38.756 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libcephsqlite-2:20.3.0-5343.gbcdd429b.el9.x86_64 19/148 2026-02-20T23:49:38.756 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_6 20/148 2026-02-20T23:49:38.756 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libradosstriper1-2:20.3.0-5343.gbcdd429b.el9.x86 21/148 2026-02-20T23:49:38.756 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : librgw2-2:20.3.0-5343.gbcdd429b.el9.x86_64 22/148 2026-02-20T23:49:38.756 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5343.gbcdd429b.el 23/148 2026-02-20T23:49:38.756 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-ceph-common-2:20.3.0-5343.gbcdd429b.el9. 24/148 2026-02-20T23:49:38.756 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_6 25/148 2026-02-20T23:49:38.756 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 26/148 2026-02-20T23:49:38.756 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 27/148 2026-02-20T23:49:38.756 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 28/148 2026-02-20T23:49:38.756 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 29/148 2026-02-20T23:49:38.757 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 30/148 2026-02-20T23:49:38.757 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 31/148 2026-02-20T23:49:38.757 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5343.gbcdd429b. 32/148 2026-02-20T23:49:38.757 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noa 33/148 2026-02-20T23:49:38.757 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.n 34/148 2026-02-20T23:49:38.757 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcd 35/148 2026-02-20T23:49:38.757 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5343.gbcdd429b.el 36/148 2026-02-20T23:49:38.757 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch 37/148 2026-02-20T23:49:38.757 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5343.gbcdd429b.e 38/148 2026-02-20T23:49:38.757 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch 39/148 2026-02-20T23:49:38.757 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 40/148 2026-02-20T23:49:38.757 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 41/148 2026-02-20T23:49:38.757 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 42/148 2026-02-20T23:49:38.757 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 43/148 2026-02-20T23:49:38.758 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 44/148 2026-02-20T23:49:38.758 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 45/148 2026-02-20T23:49:38.758 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 46/148 2026-02-20T23:49:38.758 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 47/148 2026-02-20T23:49:38.758 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 48/148 2026-02-20T23:49:38.758 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 49/148 2026-02-20T23:49:38.758 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 50/148 2026-02-20T23:49:38.758 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 51/148 2026-02-20T23:49:38.758 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 52/148 2026-02-20T23:49:38.758 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-ply-3.11-14.el9.noarch 53/148 2026-02-20T23:49:38.758 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 54/148 2026-02-20T23:49:38.758 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 55/148 2026-02-20T23:49:38.758 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 56/148 2026-02-20T23:49:38.758 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 57/148 2026-02-20T23:49:38.758 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : unzip-6.0-59.el9.x86_64 58/148 2026-02-20T23:49:38.759 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : zip-3.0-35.el9.x86_64 59/148 2026-02-20T23:49:38.759 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 60/148 2026-02-20T23:49:38.759 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 61/148 2026-02-20T23:49:38.759 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 62/148 2026-02-20T23:49:38.759 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 63/148 2026-02-20T23:49:38.759 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 64/148 2026-02-20T23:49:38.759 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 65/148 2026-02-20T23:49:38.759 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 66/148 2026-02-20T23:49:38.759 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 67/148 2026-02-20T23:49:38.759 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 68/148 2026-02-20T23:49:38.759 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 69/148 2026-02-20T23:49:38.759 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 70/148 2026-02-20T23:49:38.759 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 71/148 2026-02-20T23:49:38.760 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 72/148 2026-02-20T23:49:38.760 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : lua-5.4.4-4.el9.x86_64 73/148 2026-02-20T23:49:38.760 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 74/148 2026-02-20T23:49:38.760 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 75/148 2026-02-20T23:49:38.760 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 76/148 2026-02-20T23:49:38.760 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 77/148 2026-02-20T23:49:38.760 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 78/148 2026-02-20T23:49:38.760 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-iniconfig-1.1.1-7.el9.noarch 79/148 2026-02-20T23:49:38.760 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 80/148 2026-02-20T23:49:38.760 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 81/148 2026-02-20T23:49:38.760 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 82/148 2026-02-20T23:49:38.760 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 83/148 2026-02-20T23:49:38.760 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 84/148 2026-02-20T23:49:38.761 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 85/148 2026-02-20T23:49:38.761 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 86/148 2026-02-20T23:49:38.761 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-pluggy-0.13.1-7.el9.noarch 87/148 2026-02-20T23:49:38.761 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 88/148 2026-02-20T23:49:38.761 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-py-1.10.0-6.el9.noarch 89/148 2026-02-20T23:49:38.761 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 90/148 2026-02-20T23:49:38.761 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 91/148 2026-02-20T23:49:38.761 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-pytest-6.2.2-7.el9.noarch 92/148 2026-02-20T23:49:38.761 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 93/148 2026-02-20T23:49:38.761 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 94/148 2026-02-20T23:49:38.761 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 95/148 2026-02-20T23:49:38.761 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 96/148 2026-02-20T23:49:38.761 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 97/148 2026-02-20T23:49:38.761 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 98/148 2026-02-20T23:49:38.762 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 99/148 2026-02-20T23:49:38.762 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 100/148 2026-02-20T23:49:38.762 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 101/148 2026-02-20T23:49:38.762 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 102/148 2026-02-20T23:49:38.762 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 103/148 2026-02-20T23:49:38.762 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 104/148 2026-02-20T23:49:38.762 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 105/148 2026-02-20T23:49:38.762 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 106/148 2026-02-20T23:49:38.762 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 107/148 2026-02-20T23:49:38.762 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 108/148 2026-02-20T23:49:38.762 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 109/148 2026-02-20T23:49:38.762 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 110/148 2026-02-20T23:49:38.762 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 111/148 2026-02-20T23:49:38.762 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 112/148 2026-02-20T23:49:38.762 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 113/148 2026-02-20T23:49:38.763 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 114/148 2026-02-20T23:49:38.763 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 115/148 2026-02-20T23:49:38.763 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 116/148 2026-02-20T23:49:38.763 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 117/148 2026-02-20T23:49:38.763 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 118/148 2026-02-20T23:49:38.763 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 119/148 2026-02-20T23:49:38.763 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 120/148 2026-02-20T23:49:38.763 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 121/148 2026-02-20T23:49:38.763 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 122/148 2026-02-20T23:49:38.763 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 123/148 2026-02-20T23:49:38.763 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 124/148 2026-02-20T23:49:38.763 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 125/148 2026-02-20T23:49:38.763 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 126/148 2026-02-20T23:49:38.763 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 127/148 2026-02-20T23:49:38.763 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 128/148 2026-02-20T23:49:38.763 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 129/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 130/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 131/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 132/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 133/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 134/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 135/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 136/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 137/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 138/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 139/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 140/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 141/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : re2-1:20211101-20.el9.x86_64 142/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 143/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 144/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 145/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 146/148 2026-02-20T23:49:38.764 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 147/148 2026-02-20T23:49:38.823 INFO:teuthology.orchestra.run.trial057.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 148/148 2026-02-20T23:49:38.823 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:38.823 INFO:teuthology.orchestra.run.trial057.stdout:Upgraded: 2026-02-20T23:49:38.823 INFO:teuthology.orchestra.run.trial057.stdout: librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.823 INFO:teuthology.orchestra.run.trial057.stdout: librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.823 INFO:teuthology.orchestra.run.trial057.stdout:Installed: 2026-02-20T23:49:38.823 INFO:teuthology.orchestra.run.trial057.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-20T23:49:38.823 INFO:teuthology.orchestra.run.trial057.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-20T23:49:38.823 INFO:teuthology.orchestra.run.trial057.stdout: ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.823 INFO:teuthology.orchestra.run.trial057.stdout: ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.823 INFO:teuthology.orchestra.run.trial057.stdout: ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.823 INFO:teuthology.orchestra.run.trial057.stdout: ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.823 INFO:teuthology.orchestra.run.trial057.stdout: ceph-grafana-dashboards-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mds-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-modules-core-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-mon-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-prometheus-alerts-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-selinux-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-20T23:49:38.824 INFO:teuthology.orchestra.run.trial057.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libcephfs-proxy2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libcephsqlite-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libradosstriper1-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: librgw2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: lua-5.4.4-4.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-20T23:49:38.825 INFO:teuthology.orchestra.run.trial057.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-ceph-argparse-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-iniconfig-1.1.1-7.el9.noarch 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-20T23:49:38.826 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-pluggy-0.13.1-7.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-ply-3.11-14.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-py-1.10.0-6.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-pytest-6.2.2-7.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-20T23:49:38.827 INFO:teuthology.orchestra.run.trial057.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: re2-1:20211101-20.el9.x86_64 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: unzip-6.0-59.el9.x86_64 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: zip-3.0-35.el9.x86_64 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:38.828 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-20T23:49:38.978 DEBUG:teuthology.parallel:result is None 2026-02-20T23:49:38.979 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-20T23:49:39.155 DEBUG:teuthology.orchestra.run.trial057:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-20T23:49:39.177 INFO:teuthology.orchestra.run.trial057.stdout:20.3.0-5343.gbcdd429b.el9 2026-02-20T23:49:39.178 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5343.gbcdd429b.el9 2026-02-20T23:49:39.178 INFO:teuthology.task.install:The correct ceph version 20.3.0-5343.gbcdd429b is installed. 2026-02-20T23:49:39.179 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-20T23:49:39.180 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:49:39.180 DEBUG:teuthology.orchestra.run.trial057:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-20T23:49:39.244 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-20T23:49:39.245 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:49:39.245 DEBUG:teuthology.orchestra.run.trial057:> sudo dd of=/usr/bin/daemon-helper 2026-02-20T23:49:39.311 DEBUG:teuthology.orchestra.run.trial057:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-20T23:49:39.377 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-20T23:49:39.377 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:49:39.378 DEBUG:teuthology.orchestra.run.trial057:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-20T23:49:39.440 DEBUG:teuthology.orchestra.run.trial057:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-20T23:49:39.505 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-20T23:49:39.506 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:49:39.506 DEBUG:teuthology.orchestra.run.trial057:> sudo dd of=/usr/bin/stdin-killer 2026-02-20T23:49:39.569 DEBUG:teuthology.orchestra.run.trial057:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-20T23:49:39.633 INFO:teuthology.run_tasks:Running task pexec... 2026-02-20T23:49:39.653 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-20T23:49:39.654 DEBUG:teuthology.orchestra.run.trial057:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-20T23:49:39.676 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial057.front.sepia.ceph.com 2026-02-20T23:49:39.676 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-20T23:49:39.727 DEBUG:teuthology.parallel:result is None 2026-02-20T23:49:39.727 INFO:teuthology.run_tasks:Running task install... 2026-02-20T23:49:39.740 DEBUG:teuthology.task.install:project ceph 2026-02-20T23:49:39.741 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}} 2026-02-20T23:49:39.741 DEBUG:teuthology.task.install:config {'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'} 2026-02-20T23:49:39.741 INFO:teuthology.task.install:Using flavor: debug 2026-02-20T23:49:39.750 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-20T23:49:39.750 INFO:teuthology.task.install:extra packages: [] 2026-02-20T23:49:39.751 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': ['python3-pytest'], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'tag': None, 'wait_for_package': False} 2026-02-20T23:49:39.751 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-20T23:49:39.911 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/ 2026-02-20T23:49:39.911 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5343.gbcdd429b 2026-02-20T23:49:40.298 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-20T23:49:40.298 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:49:40.298 DEBUG:teuthology.orchestra.run.trial057:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-20T23:49:40.323 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, python3-pytest on remote rpm x86_64 2026-02-20T23:49:40.324 DEBUG:teuthology.orchestra.run.trial057:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-20T23:49:40.387 DEBUG:teuthology.orchestra.run.trial057:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-20T23:49:40.452 DEBUG:teuthology.orchestra.run.trial057:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-20T23:49:40.509 INFO:teuthology.orchestra.run.trial057.stdout:check_obsoletes = 1 2026-02-20T23:49:40.519 DEBUG:teuthology.orchestra.run.trial057:> sudo yum clean all 2026-02-20T23:49:40.682 INFO:teuthology.orchestra.run.trial057.stdout:56 files removed 2026-02-20T23:49:40.697 DEBUG:teuthology.orchestra.run.trial057:> sudo yum -y install ceph-radosgw ceph-test ceph ceph-base cephadm ceph-immutable-object-cache ceph-mgr ceph-mgr-dashboard ceph-mgr-diskprediction-local ceph-mgr-rook ceph-mgr-cephadm ceph-osd ceph-osd-classic ceph-fuse ceph-volume librados-devel libcephfs2 libcephfs-devel librados2 librbd1 python3-rados python3-rgw python3-cephfs python3-rbd rbd-fuse rbd-mirror rbd-nbd python3-pytest 2026-02-20T23:49:41.561 INFO:teuthology.orchestra.run.trial057.stdout:ceph packages for x86_64 128 kB/s | 92 kB 00:00 2026-02-20T23:49:42.230 INFO:teuthology.orchestra.run.trial057.stdout:ceph noarch packages 37 kB/s | 24 kB 00:00 2026-02-20T23:49:42.829 INFO:teuthology.orchestra.run.trial057.stdout:ceph source packages 3.6 kB/s | 2.1 kB 00:00 2026-02-20T23:49:43.514 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.9 MB 00:00 2026-02-20T23:49:47.251 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - AppStream 8.4 MB/s | 27 MB 00:03 2026-02-20T23:49:49.961 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - CRB 26 MB/s | 7.8 MB 00:00 2026-02-20T23:49:50.814 INFO:teuthology.orchestra.run.trial057.stdout:CentOS Stream 9 - Extras packages 156 kB/s | 20 kB 00:00 2026-02-20T23:49:51.732 INFO:teuthology.orchestra.run.trial057.stdout:Extra Packages for Enterprise Linux 24 MB/s | 20 MB 00:00 2026-02-20T23:49:55.050 INFO:teuthology.orchestra.run.trial057.stdout:lab-extras 1.1 MB/s | 50 kB 00:00 2026-02-20T23:49:56.239 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.239 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.239 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.240 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.240 INFO:teuthology.orchestra.run.trial057.stdout:Package cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-20T23:49:56.241 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.241 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.242 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-20T23:49:56.242 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-20T23:49:56.242 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-20T23:49:56.243 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-20T23:49:56.243 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.244 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.244 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.245 INFO:teuthology.orchestra.run.trial057.stdout:Package ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-20T23:49:56.245 INFO:teuthology.orchestra.run.trial057.stdout:Package librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.246 INFO:teuthology.orchestra.run.trial057.stdout:Package libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.246 INFO:teuthology.orchestra.run.trial057.stdout:Package libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.246 INFO:teuthology.orchestra.run.trial057.stdout:Package librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.247 INFO:teuthology.orchestra.run.trial057.stdout:Package librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.247 INFO:teuthology.orchestra.run.trial057.stdout:Package python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.248 INFO:teuthology.orchestra.run.trial057.stdout:Package python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.248 INFO:teuthology.orchestra.run.trial057.stdout:Package python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.248 INFO:teuthology.orchestra.run.trial057.stdout:Package python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.249 INFO:teuthology.orchestra.run.trial057.stdout:Package rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.249 INFO:teuthology.orchestra.run.trial057.stdout:Package rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.250 INFO:teuthology.orchestra.run.trial057.stdout:Package rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:49:56.250 INFO:teuthology.orchestra.run.trial057.stdout:Package python3-pytest-6.2.2-7.el9.noarch is already installed. 2026-02-20T23:49:56.269 INFO:teuthology.orchestra.run.trial057.stdout:Dependencies resolved. 2026-02-20T23:49:56.270 INFO:teuthology.orchestra.run.trial057.stdout:Nothing to do. 2026-02-20T23:49:56.270 INFO:teuthology.orchestra.run.trial057.stdout:Complete! 2026-02-20T23:49:56.297 DEBUG:teuthology.parallel:result is None 2026-02-20T23:49:56.297 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-20T23:49:56.456 DEBUG:teuthology.orchestra.run.trial057:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-20T23:49:56.478 INFO:teuthology.orchestra.run.trial057.stdout:20.3.0-5343.gbcdd429b.el9 2026-02-20T23:49:56.478 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5343.gbcdd429b.el9 2026-02-20T23:49:56.478 INFO:teuthology.task.install:The correct ceph version 20.3.0-5343.gbcdd429b is installed. 2026-02-20T23:49:56.480 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-20T23:49:56.480 INFO:teuthology.run_tasks:Running task ceph... 2026-02-20T23:49:56.570 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-20T23:49:56.571 DEBUG:teuthology.orchestra.run.trial057:> sudo chmod 777 /var/log/ceph 2026-02-20T23:49:56.596 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-20T23:49:56.596 DEBUG:teuthology.orchestra.run.trial057:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-20T23:49:56.660 INFO:tasks.ceph:Creating extra log directories... 2026-02-20T23:49:56.660 DEBUG:teuthology.orchestra.run.trial057:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-20T23:49:56.732 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-20T23:49:56.732 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'slow request'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-20T23:49:56.732 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61870', 'branch': 'wip-liucm-osd-shard-redesign-crimson-only', 'description': 'crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} tasks/rbd_python_api_tests}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '61870', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-liucm-osd-shard-redesign-crimson-only'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'slow request'], 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}}, 'workunit': {'branch': 'wip-liucm-osd-shard-redesign-crimson-only', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}}, 'owner': 'scheduled_liucm@soko04.front.sepia.ceph.com', 'priority': 120, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 6283, 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'wip-liucm-osd-shard-redesign-crimson-only', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'targets': {'trial057.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBJSHHyhJtv/dDcwMHqjCODxpvw3rgBPcmUMWFhPBegk6A9G7XF5/Lc0g+rv+/H6xw+i3I0SzBFxHy4ybUa/TAYY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': ['python3-pytest'], 'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(SLOW_OPS\\)', 'slow request'], 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['rbd/test_librbd_python.sh']}, 'env': {'RBD_FEATURES': '61'}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-20_22:16:48', 'tube': 'trial', 'user': 'liucm', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699'} 2026-02-20T23:49:56.732 DEBUG:teuthology.orchestra.run.trial057:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-20T23:49:56.782 DEBUG:teuthology.orchestra.run.trial057:> sudo install -d -m0777 -- /var/run/ceph 2026-02-20T23:49:56.857 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:49:56.857 DEBUG:teuthology.orchestra.run.trial057:> dd if=/scratch_devs of=/dev/stdout 2026-02-20T23:49:56.917 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-20T23:49:56.917 DEBUG:teuthology.orchestra.run.trial057:> stat /dev/vg_nvme/lv_1 2026-02-20T23:49:56.973 INFO:teuthology.orchestra.run.trial057.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-20T23:49:56.973 INFO:teuthology.orchestra.run.trial057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T23:49:56.973 INFO:teuthology.orchestra.run.trial057.stdout:Device: 6h/6d Inode: 910 Links: 1 2026-02-20T23:49:56.973 INFO:teuthology.orchestra.run.trial057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T23:49:56.973 INFO:teuthology.orchestra.run.trial057.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T23:49:56.973 INFO:teuthology.orchestra.run.trial057.stdout:Access: 2026-02-20 23:49:35.737660125 +0000 2026-02-20T23:49:56.973 INFO:teuthology.orchestra.run.trial057.stdout:Modify: 2026-02-20 23:38:56.992842912 +0000 2026-02-20T23:49:56.973 INFO:teuthology.orchestra.run.trial057.stdout:Change: 2026-02-20 23:38:56.992842912 +0000 2026-02-20T23:49:56.974 INFO:teuthology.orchestra.run.trial057.stdout: Birth: 2026-02-20 23:38:56.992842912 +0000 2026-02-20T23:49:56.974 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-20T23:49:57.036 INFO:teuthology.orchestra.run.trial057.stderr:1+0 records in 2026-02-20T23:49:57.036 INFO:teuthology.orchestra.run.trial057.stderr:1+0 records out 2026-02-20T23:49:57.036 INFO:teuthology.orchestra.run.trial057.stderr:512 bytes copied, 0.000161761 s, 3.2 MB/s 2026-02-20T23:49:57.037 DEBUG:teuthology.orchestra.run.trial057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-20T23:49:57.095 DEBUG:teuthology.orchestra.run.trial057:> stat /dev/vg_nvme/lv_2 2026-02-20T23:49:57.151 INFO:teuthology.orchestra.run.trial057.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-20T23:49:57.151 INFO:teuthology.orchestra.run.trial057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T23:49:57.152 INFO:teuthology.orchestra.run.trial057.stdout:Device: 6h/6d Inode: 937 Links: 1 2026-02-20T23:49:57.152 INFO:teuthology.orchestra.run.trial057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T23:49:57.152 INFO:teuthology.orchestra.run.trial057.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T23:49:57.152 INFO:teuthology.orchestra.run.trial057.stdout:Access: 2026-02-20 23:49:35.737660125 +0000 2026-02-20T23:49:57.152 INFO:teuthology.orchestra.run.trial057.stdout:Modify: 2026-02-20 23:38:57.218836651 +0000 2026-02-20T23:49:57.152 INFO:teuthology.orchestra.run.trial057.stdout:Change: 2026-02-20 23:38:57.218836651 +0000 2026-02-20T23:49:57.152 INFO:teuthology.orchestra.run.trial057.stdout: Birth: 2026-02-20 23:38:57.218836651 +0000 2026-02-20T23:49:57.152 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-20T23:49:57.215 INFO:teuthology.orchestra.run.trial057.stderr:1+0 records in 2026-02-20T23:49:57.215 INFO:teuthology.orchestra.run.trial057.stderr:1+0 records out 2026-02-20T23:49:57.216 INFO:teuthology.orchestra.run.trial057.stderr:512 bytes copied, 0.000146823 s, 3.5 MB/s 2026-02-20T23:49:57.217 DEBUG:teuthology.orchestra.run.trial057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-20T23:49:57.274 DEBUG:teuthology.orchestra.run.trial057:> stat /dev/vg_nvme/lv_3 2026-02-20T23:49:57.330 INFO:teuthology.orchestra.run.trial057.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-20T23:49:57.330 INFO:teuthology.orchestra.run.trial057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T23:49:57.330 INFO:teuthology.orchestra.run.trial057.stdout:Device: 6h/6d Inode: 963 Links: 1 2026-02-20T23:49:57.330 INFO:teuthology.orchestra.run.trial057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T23:49:57.331 INFO:teuthology.orchestra.run.trial057.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T23:49:57.331 INFO:teuthology.orchestra.run.trial057.stdout:Access: 2026-02-20 23:49:35.738660106 +0000 2026-02-20T23:49:57.331 INFO:teuthology.orchestra.run.trial057.stdout:Modify: 2026-02-20 23:38:57.462829891 +0000 2026-02-20T23:49:57.331 INFO:teuthology.orchestra.run.trial057.stdout:Change: 2026-02-20 23:38:57.462829891 +0000 2026-02-20T23:49:57.331 INFO:teuthology.orchestra.run.trial057.stdout: Birth: 2026-02-20 23:38:57.462829891 +0000 2026-02-20T23:49:57.331 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-20T23:49:57.391 INFO:teuthology.orchestra.run.trial057.stderr:1+0 records in 2026-02-20T23:49:57.392 INFO:teuthology.orchestra.run.trial057.stderr:1+0 records out 2026-02-20T23:49:57.392 INFO:teuthology.orchestra.run.trial057.stderr:512 bytes copied, 0.000145701 s, 3.5 MB/s 2026-02-20T23:49:57.393 DEBUG:teuthology.orchestra.run.trial057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-20T23:49:57.450 DEBUG:teuthology.orchestra.run.trial057:> stat /dev/vg_nvme/lv_4 2026-02-20T23:49:57.506 INFO:teuthology.orchestra.run.trial057.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-20T23:49:57.506 INFO:teuthology.orchestra.run.trial057.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T23:49:57.506 INFO:teuthology.orchestra.run.trial057.stdout:Device: 6h/6d Inode: 995 Links: 1 2026-02-20T23:49:57.506 INFO:teuthology.orchestra.run.trial057.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T23:49:57.506 INFO:teuthology.orchestra.run.trial057.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T23:49:57.507 INFO:teuthology.orchestra.run.trial057.stdout:Access: 2026-02-20 23:49:35.738660106 +0000 2026-02-20T23:49:57.507 INFO:teuthology.orchestra.run.trial057.stdout:Modify: 2026-02-20 23:38:57.698823352 +0000 2026-02-20T23:49:57.507 INFO:teuthology.orchestra.run.trial057.stdout:Change: 2026-02-20 23:38:57.698823352 +0000 2026-02-20T23:49:57.507 INFO:teuthology.orchestra.run.trial057.stdout: Birth: 2026-02-20 23:38:57.698823352 +0000 2026-02-20T23:49:57.507 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-20T23:49:57.569 INFO:teuthology.orchestra.run.trial057.stderr:1+0 records in 2026-02-20T23:49:57.569 INFO:teuthology.orchestra.run.trial057.stderr:1+0 records out 2026-02-20T23:49:57.569 INFO:teuthology.orchestra.run.trial057.stderr:512 bytes copied, 0.000151231 s, 3.4 MB/s 2026-02-20T23:49:57.570 DEBUG:teuthology.orchestra.run.trial057:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-20T23:49:57.627 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T23:49:57.627 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial057.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-20T23:49:57.627 INFO:tasks.ceph:Generating config... 2026-02-20T23:49:57.630 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2026-02-20T23:49:57.630 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-20T23:49:57.630 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-20T23:49:57.630 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-20T23:49:57.630 INFO:tasks.ceph:[mon] paxos service trim min = 10 2026-02-20T23:49:57.630 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-20T23:49:57.630 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-20T23:49:57.630 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-20T23:49:57.631 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-20T23:49:57.631 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-20T23:49:57.631 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-20T23:49:57.631 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-20T23:49:57.631 INFO:tasks.ceph:[osd] debug ms = 20 2026-02-20T23:49:57.631 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-20T23:49:57.631 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-20T23:49:57.631 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-20T23:49:57.631 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-20T23:49:57.631 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-20T23:49:57.631 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-20T23:49:57.632 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-20T23:49:57.632 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-20T23:49:57.632 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-20T23:49:57.632 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-20T23:49:57.632 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-20T23:49:57.632 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-20T23:49:57.632 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-20T23:49:57.632 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-20T23:49:57.632 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-20T23:49:57.632 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-20T23:49:57.632 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-20T23:49:57.632 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-20T23:49:57.632 INFO:tasks.ceph:[osd] seastore main device type = RANDOM_BLOCK_SSD 2026-02-20T23:49:57.633 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-20T23:49:57.633 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-20T23:49:57.633 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-20T23:49:57.633 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-20T23:49:57.633 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-20T23:49:57.633 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-20T23:49:57.633 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-20T23:49:57.633 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-20T23:49:57.633 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-20T23:49:57.633 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-20T23:49:57.633 INFO:tasks.ceph:Setting up mon.a... 2026-02-20T23:49:57.634 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-20T23:49:57.709 INFO:teuthology.orchestra.run.trial057.stdout:creating /etc/ceph/ceph.keyring 2026-02-20T23:49:57.711 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-20T23:49:57.796 DEBUG:teuthology.orchestra.run.trial057:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-20T23:49:57.860 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.57')] 2026-02-20T23:49:57.861 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.57', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-20T23:49:57.863 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:49:57.863 DEBUG:teuthology.orchestra.run.trial057:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-20T23:49:57.917 DEBUG:teuthology.orchestra.run.trial057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.57 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-20T23:49:57.998 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:49:58.005+0000 7f7b6d68ef40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:49:57.998 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:49:58.005+0000 7f7b6d68ef40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:49:57.998 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:49:58.005+0000 7f7b6d68ef40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:49:57.999 INFO:teuthology.orchestra.run.trial057.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-20T23:49:57.999 INFO:teuthology.orchestra.run.trial057.stdout:monmaptool: generated fsid 797a1d6f-8149-4ca0-b8e3-a68b969d2bab 2026-02-20T23:49:57.999 INFO:teuthology.orchestra.run.trial057.stdout:setting min_mon_release = tentacle 2026-02-20T23:49:57.999 INFO:teuthology.orchestra.run.trial057.stdout:epoch 0 2026-02-20T23:49:57.999 INFO:teuthology.orchestra.run.trial057.stdout:fsid 797a1d6f-8149-4ca0-b8e3-a68b969d2bab 2026-02-20T23:49:58.000 INFO:teuthology.orchestra.run.trial057.stdout:last_changed 2026-02-20T23:49:58.006036+0000 2026-02-20T23:49:58.000 INFO:teuthology.orchestra.run.trial057.stdout:created 2026-02-20T23:49:58.006036+0000 2026-02-20T23:49:58.000 INFO:teuthology.orchestra.run.trial057.stdout:min_mon_release 20 (tentacle) 2026-02-20T23:49:58.000 INFO:teuthology.orchestra.run.trial057.stdout:election_strategy: 1 2026-02-20T23:49:58.000 INFO:teuthology.orchestra.run.trial057.stdout:0: [v2:10.20.193.57:3300/0,v1:10.20.193.57:6789/0] mon.a 2026-02-20T23:49:58.000 INFO:teuthology.orchestra.run.trial057.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-20T23:49:58.001 DEBUG:teuthology.orchestra.run.trial057:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-20T23:49:58.057 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 797a1d6f-8149-4ca0-b8e3-a68b969d2bab... 2026-02-20T23:49:58.058 DEBUG:teuthology.orchestra.run.trial057:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-20T23:49:58.139 INFO:teuthology.orchestra.run.trial057.stdout:[global] 2026-02-20T23:49:58.139 INFO:teuthology.orchestra.run.trial057.stdout: chdir = "" 2026-02-20T23:49:58.139 INFO:teuthology.orchestra.run.trial057.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-20T23:49:58.139 INFO:teuthology.orchestra.run.trial057.stdout: auth supported = cephx 2026-02-20T23:49:58.139 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.139 INFO:teuthology.orchestra.run.trial057.stdout: filestore xattr use omap = true 2026-02-20T23:49:58.139 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.139 INFO:teuthology.orchestra.run.trial057.stdout: mon clock drift allowed = 1.000 2026-02-20T23:49:58.140 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.140 INFO:teuthology.orchestra.run.trial057.stdout: osd crush chooseleaf type = 0 2026-02-20T23:49:58.140 INFO:teuthology.orchestra.run.trial057.stdout: auth debug = true 2026-02-20T23:49:58.140 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.140 INFO:teuthology.orchestra.run.trial057.stdout: ms die on old message = true 2026-02-20T23:49:58.140 INFO:teuthology.orchestra.run.trial057.stdout: ms die on bug = true 2026-02-20T23:49:58.140 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.140 INFO:teuthology.orchestra.run.trial057.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-20T23:49:58.140 INFO:teuthology.orchestra.run.trial057.stdout: mon pg warn max object skew = 0 2026-02-20T23:49:58.140 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.140 INFO:teuthology.orchestra.run.trial057.stdout: # disable pg_autoscaler by default for new pools 2026-02-20T23:49:58.140 INFO:teuthology.orchestra.run.trial057.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-20T23:49:58.141 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.141 INFO:teuthology.orchestra.run.trial057.stdout: osd pool default size = 2 2026-02-20T23:49:58.141 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.141 INFO:teuthology.orchestra.run.trial057.stdout: mon osd allow primary affinity = true 2026-02-20T23:49:58.141 INFO:teuthology.orchestra.run.trial057.stdout: mon osd allow pg remap = true 2026-02-20T23:49:58.141 INFO:teuthology.orchestra.run.trial057.stdout: mon warn on legacy crush tunables = false 2026-02-20T23:49:58.141 INFO:teuthology.orchestra.run.trial057.stdout: mon warn on crush straw calc version zero = false 2026-02-20T23:49:58.141 INFO:teuthology.orchestra.run.trial057.stdout: mon warn on no sortbitwise = false 2026-02-20T23:49:58.141 INFO:teuthology.orchestra.run.trial057.stdout: mon warn on osd down out interval zero = false 2026-02-20T23:49:58.141 INFO:teuthology.orchestra.run.trial057.stdout: mon warn on too few osds = false 2026-02-20T23:49:58.141 INFO:teuthology.orchestra.run.trial057.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-20T23:49:58.141 INFO:teuthology.orchestra.run.trial057.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-20T23:49:58.141 INFO:teuthology.orchestra.run.trial057.stdout: mon_allow_pool_size_one = true 2026-02-20T23:49:58.142 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.142 INFO:teuthology.orchestra.run.trial057.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-20T23:49:58.142 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.142 INFO:teuthology.orchestra.run.trial057.stdout: osd default data pool replay window = 5 2026-02-20T23:49:58.142 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.142 INFO:teuthology.orchestra.run.trial057.stdout: mon allow pool delete = true 2026-02-20T23:49:58.142 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.142 INFO:teuthology.orchestra.run.trial057.stdout: mon cluster log file level = debug 2026-02-20T23:49:58.142 INFO:teuthology.orchestra.run.trial057.stdout: debug asserts on shutdown = true 2026-02-20T23:49:58.142 INFO:teuthology.orchestra.run.trial057.stdout: mon health detail to clog = false 2026-02-20T23:49:58.142 INFO:teuthology.orchestra.run.trial057.stdout: mon host = 10.20.193.57 2026-02-20T23:49:58.142 INFO:teuthology.orchestra.run.trial057.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-20T23:49:58.142 INFO:teuthology.orchestra.run.trial057.stdout: ms client mode = crc 2026-02-20T23:49:58.143 INFO:teuthology.orchestra.run.trial057.stdout: ms cluster mode = crc 2026-02-20T23:49:58.143 INFO:teuthology.orchestra.run.trial057.stdout: ms mon client mode = crc 2026-02-20T23:49:58.143 INFO:teuthology.orchestra.run.trial057.stdout: ms mon cluster mode = crc 2026-02-20T23:49:58.143 INFO:teuthology.orchestra.run.trial057.stdout: ms mon service mode = crc 2026-02-20T23:49:58.143 INFO:teuthology.orchestra.run.trial057.stdout: ms service mode = crc 2026-02-20T23:49:58.143 INFO:teuthology.orchestra.run.trial057.stdout: fsid = 797a1d6f-8149-4ca0-b8e3-a68b969d2bab 2026-02-20T23:49:58.143 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.143 INFO:teuthology.orchestra.run.trial057.stdout:[osd] 2026-02-20T23:49:58.143 INFO:teuthology.orchestra.run.trial057.stdout: osd journal size = 100 2026-02-20T23:49:58.143 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.143 INFO:teuthology.orchestra.run.trial057.stdout: osd scrub load threshold = 5.0 2026-02-20T23:49:58.143 INFO:teuthology.orchestra.run.trial057.stdout: osd scrub max interval = 600 2026-02-20T23:49:58.144 INFO:teuthology.orchestra.run.trial057.stdout: osd mclock profile = high_recovery_ops 2026-02-20T23:49:58.144 INFO:teuthology.orchestra.run.trial057.stdout: osd mclock skip benchmark = true 2026-02-20T23:49:58.144 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.144 INFO:teuthology.orchestra.run.trial057.stdout: osd recover clone overlap = true 2026-02-20T23:49:58.144 INFO:teuthology.orchestra.run.trial057.stdout: osd recovery max chunk = 1048576 2026-02-20T23:49:58.144 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.144 INFO:teuthology.orchestra.run.trial057.stdout: osd debug shutdown = true 2026-02-20T23:49:58.144 INFO:teuthology.orchestra.run.trial057.stdout: osd debug op order = true 2026-02-20T23:49:58.144 INFO:teuthology.orchestra.run.trial057.stdout: osd debug verify stray on activate = true 2026-02-20T23:49:58.144 INFO:teuthology.orchestra.run.trial057.stdout: osd debug trim objects = true 2026-02-20T23:49:58.144 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.144 INFO:teuthology.orchestra.run.trial057.stdout: osd open classes on start = true 2026-02-20T23:49:58.144 INFO:teuthology.orchestra.run.trial057.stdout: osd debug pg log writeout = true 2026-02-20T23:49:58.145 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.145 INFO:teuthology.orchestra.run.trial057.stdout: osd deep scrub update digest min age = 30 2026-02-20T23:49:58.145 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.145 INFO:teuthology.orchestra.run.trial057.stdout: osd map max advance = 10 2026-02-20T23:49:58.145 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.145 INFO:teuthology.orchestra.run.trial057.stdout: journal zero on create = true 2026-02-20T23:49:58.145 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.145 INFO:teuthology.orchestra.run.trial057.stdout: filestore ondisk finisher threads = 3 2026-02-20T23:49:58.145 INFO:teuthology.orchestra.run.trial057.stdout: filestore apply finisher threads = 3 2026-02-20T23:49:58.145 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.145 INFO:teuthology.orchestra.run.trial057.stdout: bdev debug aio = true 2026-02-20T23:49:58.145 INFO:teuthology.orchestra.run.trial057.stdout: osd debug misdirected ops = true 2026-02-20T23:49:58.145 INFO:teuthology.orchestra.run.trial057.stdout: debug monc = 20 2026-02-20T23:49:58.146 INFO:teuthology.orchestra.run.trial057.stdout: crimson cpu num = 4 2026-02-20T23:49:58.146 INFO:teuthology.orchestra.run.trial057.stdout: crimson osd obc lru size = 10 2026-02-20T23:49:58.146 INFO:teuthology.orchestra.run.trial057.stdout: debug ms = 20 2026-02-20T23:49:58.146 INFO:teuthology.orchestra.run.trial057.stdout: debug osd = 20 2026-02-20T23:49:58.146 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore = 20 2026-02-20T23:49:58.146 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore backref = 20 2026-02-20T23:49:58.146 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore cache = 20 2026-02-20T23:49:58.146 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore cleaner = 20 2026-02-20T23:49:58.146 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore device = 20 2026-02-20T23:49:58.146 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore epm = 20 2026-02-20T23:49:58.146 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore fixedkv tree = 20 2026-02-20T23:49:58.146 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore journal = 20 2026-02-20T23:49:58.146 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore lba = 20 2026-02-20T23:49:58.147 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore odata = 20 2026-02-20T23:49:58.147 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore omap = 20 2026-02-20T23:49:58.147 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore onode = 20 2026-02-20T23:49:58.147 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore t = 20 2026-02-20T23:49:58.147 INFO:teuthology.orchestra.run.trial057.stdout: debug seastore tm = 20 2026-02-20T23:49:58.147 INFO:teuthology.orchestra.run.trial057.stdout: osd objectstore = seastore 2026-02-20T23:49:58.147 INFO:teuthology.orchestra.run.trial057.stdout: osd shutdown pgref assert = True 2026-02-20T23:49:58.147 INFO:teuthology.orchestra.run.trial057.stdout: seastore cachepin size pershard = 64M 2026-02-20T23:49:58.147 INFO:teuthology.orchestra.run.trial057.stdout: seastore main device type = RANDOM_BLOCK_SSD 2026-02-20T23:49:58.147 INFO:teuthology.orchestra.run.trial057.stdout: seastore max concurrent transactions = 8 2026-02-20T23:49:58.147 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.147 INFO:teuthology.orchestra.run.trial057.stdout:[mgr] 2026-02-20T23:49:58.147 INFO:teuthology.orchestra.run.trial057.stdout: debug ms = 1 2026-02-20T23:49:58.148 INFO:teuthology.orchestra.run.trial057.stdout: debug mgr = 20 2026-02-20T23:49:58.148 INFO:teuthology.orchestra.run.trial057.stdout: debug mon = 20 2026-02-20T23:49:58.148 INFO:teuthology.orchestra.run.trial057.stdout: debug auth = 20 2026-02-20T23:49:58.148 INFO:teuthology.orchestra.run.trial057.stdout: mon reweight min pgs per osd = 4 2026-02-20T23:49:58.148 INFO:teuthology.orchestra.run.trial057.stdout: mon reweight min bytes per osd = 10 2026-02-20T23:49:58.148 INFO:teuthology.orchestra.run.trial057.stdout: mgr/telemetry/nag = false 2026-02-20T23:49:58.148 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.148 INFO:teuthology.orchestra.run.trial057.stdout:[mon] 2026-02-20T23:49:58.148 INFO:teuthology.orchestra.run.trial057.stdout: debug ms = 1 2026-02-20T23:49:58.148 INFO:teuthology.orchestra.run.trial057.stdout: debug mon = 20 2026-02-20T23:49:58.148 INFO:teuthology.orchestra.run.trial057.stdout: debug paxos = 20 2026-02-20T23:49:58.148 INFO:teuthology.orchestra.run.trial057.stdout: debug auth = 20 2026-02-20T23:49:58.148 INFO:teuthology.orchestra.run.trial057.stdout: mon data avail warn = 5 2026-02-20T23:49:58.148 INFO:teuthology.orchestra.run.trial057.stdout: mon mgr mkfs grace = 240 2026-02-20T23:49:58.149 INFO:teuthology.orchestra.run.trial057.stdout: mon reweight min pgs per osd = 4 2026-02-20T23:49:58.149 INFO:teuthology.orchestra.run.trial057.stdout: mon osd reporter subtree level = osd 2026-02-20T23:49:58.149 INFO:teuthology.orchestra.run.trial057.stdout: mon osd prime pg temp = true 2026-02-20T23:49:58.149 INFO:teuthology.orchestra.run.trial057.stdout: mon reweight min bytes per osd = 10 2026-02-20T23:49:58.149 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.149 INFO:teuthology.orchestra.run.trial057.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-20T23:49:58.149 INFO:teuthology.orchestra.run.trial057.stdout: auth mon ticket ttl = 660 # 11m 2026-02-20T23:49:58.149 INFO:teuthology.orchestra.run.trial057.stdout: auth service ticket ttl = 240 # 4m 2026-02-20T23:49:58.149 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.149 INFO:teuthology.orchestra.run.trial057.stdout: # don't complain about insecure global_id in the test suite 2026-02-20T23:49:58.149 INFO:teuthology.orchestra.run.trial057.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-20T23:49:58.149 INFO:teuthology.orchestra.run.trial057.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-20T23:49:58.149 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.150 INFO:teuthology.orchestra.run.trial057.stdout: # 1m isn't quite enough 2026-02-20T23:49:58.150 INFO:teuthology.orchestra.run.trial057.stdout: mon_down_mkfs_grace = 2m 2026-02-20T23:49:58.150 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.150 INFO:teuthology.orchestra.run.trial057.stdout: mon_warn_on_filestore_osds = false 2026-02-20T23:49:58.150 INFO:teuthology.orchestra.run.trial057.stdout: mon min osdmap epochs = 50 2026-02-20T23:49:58.150 INFO:teuthology.orchestra.run.trial057.stdout: mon osdmap full prune interval = 2 2026-02-20T23:49:58.150 INFO:teuthology.orchestra.run.trial057.stdout: mon osdmap full prune min = 15 2026-02-20T23:49:58.150 INFO:teuthology.orchestra.run.trial057.stdout: mon osdmap full prune txsize = 2 2026-02-20T23:49:58.150 INFO:teuthology.orchestra.run.trial057.stdout: paxos service trim min = 10 2026-02-20T23:49:58.150 INFO:teuthology.orchestra.run.trial057.stdout: osd pool default crimson = True 2026-02-20T23:49:58.150 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:49:58.150 INFO:teuthology.orchestra.run.trial057.stdout:[client] 2026-02-20T23:49:58.150 INFO:teuthology.orchestra.run.trial057.stdout: rgw cache enabled = true 2026-02-20T23:49:58.151 INFO:teuthology.orchestra.run.trial057.stdout: rgw enable ops log = true 2026-02-20T23:49:58.151 INFO:teuthology.orchestra.run.trial057.stdout: rgw enable usage log = true 2026-02-20T23:49:58.151 INFO:teuthology.orchestra.run.trial057.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-20T23:49:58.151 INFO:teuthology.orchestra.run.trial057.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-20T23:49:58.151 INFO:teuthology.orchestra.run.trial057.stdout:[mon.a] 2026-02-20T23:49:58.152 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-20T23:49:58.152 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-20T23:49:58.244 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-20T23:49:58.244 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:49:58.244 DEBUG:teuthology.orchestra.run.trial057:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-20T23:49:58.261 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:49:58.261 DEBUG:teuthology.orchestra.run.trial057:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-20T23:49:58.317 INFO:tasks.ceph:Sending monmap to node ubuntu@trial057.front.sepia.ceph.com 2026-02-20T23:49:58.317 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:49:58.318 DEBUG:teuthology.orchestra.run.trial057:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-20T23:49:58.318 DEBUG:teuthology.orchestra.run.trial057:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-20T23:49:58.392 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:49:58.392 DEBUG:teuthology.orchestra.run.trial057:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-20T23:49:58.447 INFO:tasks.ceph:Setting up mon nodes... 2026-02-20T23:49:58.448 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-20T23:49:58.448 DEBUG:teuthology.orchestra.run.trial057:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-20T23:49:58.538 INFO:teuthology.orchestra.run.trial057.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-20T23:49:58.541 INFO:tasks.ceph:Setting up mds nodes... 2026-02-20T23:49:58.542 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-20T23:49:58.542 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-20T23:49:58.584 INFO:teuthology.orchestra.run.trial057.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-20T23:49:58.595 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-20T23:49:58.596 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial057.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-20T23:49:58.596 DEBUG:teuthology.orchestra.run.trial057:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-20T23:49:58.660 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T23:49:58.660 INFO:tasks.ceph:role: osd.0 2026-02-20T23:49:58.660 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial057.front.sepia.ceph.com 2026-02-20T23:49:58.661 DEBUG:teuthology.orchestra.run.trial057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-20T23:49:58.727 INFO:teuthology.orchestra.run.trial057.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T23:49:58.727 INFO:teuthology.orchestra.run.trial057.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T23:49:58.727 INFO:teuthology.orchestra.run.trial057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T23:49:58.727 INFO:teuthology.orchestra.run.trial057.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T23:49:58.727 INFO:teuthology.orchestra.run.trial057.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T23:49:58.728 INFO:teuthology.orchestra.run.trial057.stdout: = sunit=0 swidth=0 blks 2026-02-20T23:49:58.728 INFO:teuthology.orchestra.run.trial057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T23:49:58.728 INFO:teuthology.orchestra.run.trial057.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T23:49:58.728 INFO:teuthology.orchestra.run.trial057.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T23:49:58.728 INFO:teuthology.orchestra.run.trial057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T23:49:59.299 INFO:teuthology.orchestra.run.trial057.stdout:Discarding blocks...Done. 2026-02-20T23:49:59.302 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial057.front.sepia.ceph.com -o noatime 2026-02-20T23:49:59.302 DEBUG:teuthology.orchestra.run.trial057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-20T23:49:59.372 DEBUG:teuthology.orchestra.run.trial057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-20T23:49:59.440 DEBUG:teuthology.orchestra.run.trial057:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-20T23:49:59.506 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T23:49:59.506 INFO:tasks.ceph:role: osd.1 2026-02-20T23:49:59.506 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial057.front.sepia.ceph.com 2026-02-20T23:49:59.506 DEBUG:teuthology.orchestra.run.trial057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-20T23:49:59.572 INFO:teuthology.orchestra.run.trial057.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T23:49:59.572 INFO:teuthology.orchestra.run.trial057.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T23:49:59.572 INFO:teuthology.orchestra.run.trial057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T23:49:59.572 INFO:teuthology.orchestra.run.trial057.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T23:49:59.572 INFO:teuthology.orchestra.run.trial057.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T23:49:59.572 INFO:teuthology.orchestra.run.trial057.stdout: = sunit=0 swidth=0 blks 2026-02-20T23:49:59.572 INFO:teuthology.orchestra.run.trial057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T23:49:59.572 INFO:teuthology.orchestra.run.trial057.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T23:49:59.572 INFO:teuthology.orchestra.run.trial057.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T23:49:59.573 INFO:teuthology.orchestra.run.trial057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T23:50:00.131 INFO:teuthology.orchestra.run.trial057.stdout:Discarding blocks...Done. 2026-02-20T23:50:00.133 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial057.front.sepia.ceph.com -o noatime 2026-02-20T23:50:00.133 DEBUG:teuthology.orchestra.run.trial057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-20T23:50:00.202 DEBUG:teuthology.orchestra.run.trial057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-20T23:50:00.270 DEBUG:teuthology.orchestra.run.trial057:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-20T23:50:00.334 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T23:50:00.334 INFO:tasks.ceph:role: osd.2 2026-02-20T23:50:00.334 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial057.front.sepia.ceph.com 2026-02-20T23:50:00.334 DEBUG:teuthology.orchestra.run.trial057:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-20T23:50:00.399 INFO:teuthology.orchestra.run.trial057.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T23:50:00.399 INFO:teuthology.orchestra.run.trial057.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T23:50:00.399 INFO:teuthology.orchestra.run.trial057.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T23:50:00.399 INFO:teuthology.orchestra.run.trial057.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T23:50:00.399 INFO:teuthology.orchestra.run.trial057.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T23:50:00.400 INFO:teuthology.orchestra.run.trial057.stdout: = sunit=0 swidth=0 blks 2026-02-20T23:50:00.400 INFO:teuthology.orchestra.run.trial057.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T23:50:00.400 INFO:teuthology.orchestra.run.trial057.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T23:50:00.400 INFO:teuthology.orchestra.run.trial057.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T23:50:00.400 INFO:teuthology.orchestra.run.trial057.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T23:50:00.970 INFO:teuthology.orchestra.run.trial057.stdout:Discarding blocks...Done. 2026-02-20T23:50:00.972 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial057.front.sepia.ceph.com -o noatime 2026-02-20T23:50:00.972 DEBUG:teuthology.orchestra.run.trial057:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-20T23:50:01.041 DEBUG:teuthology.orchestra.run.trial057:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-20T23:50:01.108 DEBUG:teuthology.orchestra.run.trial057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T23:50:01.262 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T23:50:01.323 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:01,331 seastar - Reactor backend: linux-aio 2026-02-20T23:50:01.332 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:01,340 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T23:50:01.333 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:01,340 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T23:50:01.335 INFO:teuthology.orchestra.run.trial057.stderr:==35325==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T23:50:01.390 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:01,397 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-20T23:50:01.391 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:01,397 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-20T23:50:01.391 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:01,397 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-20T23:50:01.391 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:01,397 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-20T23:50:01.391 INFO:teuthology.orchestra.run.trial057.stderr:DEBUG 2026-02-20 23:50:01,398 [shard 0:main] seastar - reactor::drain 2026-02-20T23:50:03.449 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-20T23:50:03.449 INFO:teuthology.orchestra.run.trial057.stderr:Suppressions used: 2026-02-20T23:50:03.449 INFO:teuthology.orchestra.run.trial057.stderr: count bytes template 2026-02-20T23:50:03.449 INFO:teuthology.orchestra.run.trial057.stderr: 1 8 MallocExtension::Register 2026-02-20T23:50:03.449 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-20T23:50:03.449 INFO:teuthology.orchestra.run.trial057.stderr: 2026-02-20T23:50:03.468 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T23:50:03.468 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T23:50:03.509 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T23:50:03.509 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T23:50:03.516 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:03,523 seastar - Reactor backend: linux-aio 2026-02-20T23:50:03.529 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:03,536 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T23:50:03.529 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:03,536 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T23:50:03.532 INFO:teuthology.orchestra.run.trial057.stderr:==35323==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T23:50:03.532 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:03,539 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T23:50:03.590 INFO:teuthology.orchestra.run.trial057.stderr:DEBUG 2026-02-20 23:50:03,598 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-20T23:50:03.687 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1294d2 /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x341fad3 0x9fe518f 0x9fe6ec7 0x9fe7414 0x9fe7613 0xa0c7985 0xa0c80e0 0xa0d236e 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:03.687 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xa31f /lib64/libasan.so.8+0xc478 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe5abd5 0xe5aef8 0xa0b489c 0xa0d1aff 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:03.687 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x157c7 0x10d1d95 0xf741d0 0x9fe4a64 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5a18 0x9fec227 0x9fec907 0xa0d213c 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:03.720 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191c1b /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9ec99fb 0x9f2f1f6 0x9f2f30c 0x9f2f3e8 0x9f2f664 0x9f2f967 0x9f2fc0b 0x9f3000f 0x9f302cc 0xa0d49b5 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-20T23:50:03.720 INFO:teuthology.orchestra.run.trial057.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff94c912e1 0xffffffff9594d451 0xffffffff95a00ba2 2026-02-20T23:50:03.779 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x11681cc 0xf741c5 0xa0c2b65 0xa0c2eaa 0xa0c322d 0xa0d3384 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:03.780 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xedc86 0xe6ca27 0xe6cd08 0xa0d87e9 0x7bc71f4 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:03.780 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb04e6 /lib64/libasan.so.8+0xb1437 0xba66a7 0x10d1d1e 0xf741d0 0x9fe4a64 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5f05 0x9fe6403 0x9fe6a33 0x9fe6b9d 0xa0c1f5d 0xa0d4d60 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:03.893 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x9fe5f05 0x9fe6403 0x9fe6a33 0x9fe6b9d 0xa0c1f5d 0xa0d4d60 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:03.893 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x341e135 0x9fe4eb7 0x9fe51a1 0x9fe5457 0x9fe5f05 0x9fe6403 0x9fe6a33 0x9fe6b9d 0xa0c1f5d 0xa0d4d60 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:03.894 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x1168051 0xf741c5 0x9fe4a64 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5a18 0x9fec227 0x9fec907 0xa0d213c 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:06.098 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-20T23:50:06.099 INFO:teuthology.orchestra.run.trial057.stderr:Suppressions used: 2026-02-20T23:50:06.099 INFO:teuthology.orchestra.run.trial057.stderr: count bytes template 2026-02-20T23:50:06.099 INFO:teuthology.orchestra.run.trial057.stderr: 1 8 MallocExtension::Register 2026-02-20T23:50:06.099 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-20T23:50:06.099 INFO:teuthology.orchestra.run.trial057.stderr: 2026-02-20T23:50:06.132 DEBUG:teuthology.orchestra.run.trial057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-20T23:50:06.158 DEBUG:teuthology.orchestra.run.trial057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T23:50:06.310 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T23:50:06.363 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:06,370 seastar - Reactor backend: linux-aio 2026-02-20T23:50:06.372 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:06,380 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T23:50:06.372 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:06,380 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T23:50:06.375 INFO:teuthology.orchestra.run.trial057.stderr:==35392==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T23:50:06.430 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:06,437 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-20T23:50:06.431 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:06,437 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-20T23:50:06.431 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:06,437 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-20T23:50:06.431 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:06,437 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-20T23:50:06.431 INFO:teuthology.orchestra.run.trial057.stderr:DEBUG 2026-02-20 23:50:06,437 [shard 0:main] seastar - reactor::drain 2026-02-20T23:50:08.477 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-20T23:50:08.477 INFO:teuthology.orchestra.run.trial057.stderr:Suppressions used: 2026-02-20T23:50:08.477 INFO:teuthology.orchestra.run.trial057.stderr: count bytes template 2026-02-20T23:50:08.477 INFO:teuthology.orchestra.run.trial057.stderr: 1 8 MallocExtension::Register 2026-02-20T23:50:08.477 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-20T23:50:08.477 INFO:teuthology.orchestra.run.trial057.stderr: 2026-02-20T23:50:08.496 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T23:50:08.497 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T23:50:08.540 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T23:50:08.540 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T23:50:08.547 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:08,555 seastar - Reactor backend: linux-aio 2026-02-20T23:50:08.558 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:08,565 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T23:50:08.558 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:08,565 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T23:50:08.562 INFO:teuthology.orchestra.run.trial057.stderr:==35390==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T23:50:08.562 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:08,569 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T23:50:08.620 INFO:teuthology.orchestra.run.trial057.stderr:DEBUG 2026-02-20 23:50:08,627 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-20T23:50:08.717 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0xbddba7 0xa0c2bb5 0xa0c2eaa 0xa0c322d 0xa0d3384 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:08.717 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x116816a 0xf741c5 0x9fe874e 0x9fe908d 0xa0ce025 0xa0d870b 0x7bc71f4 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:08.717 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc3ef /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe6ca27 0xe6cd08 0xa0d87e9 0x7bc71f4 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:08.749 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/ld-linux-x86-64.so.2+0x16ec6 /lib64/libasan.so.8+0xc65b1 /lib64/libasan.so.8+0x15197 0x9fe4acd 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5f05 0x9fe6403 0x9fe6a33 0x9fe6b9d 0xa0c1f5d 0xa0d4d60 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-20T23:50:08.808 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119155 /lib64/libasan.so.8+0xc5f4 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe6ca27 0xe6cd7e 0xa0d87e9 0x7bc71f4 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:08.809 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb04f2 /lib64/libasan.so.8+0xb1437 0xba66a7 0x10d1d1e 0xf741d0 0x9fe874e 0x9fe908d 0xa0ce025 0xa0d870b 0x7bc71f4 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:08.809 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0xcee060 0xa0d8780 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:08.921 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x3422fc0 0xa0d4d7d 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:08.921 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0xa0b2c41 0xa0b2f70 0xa0b31da 0xa0b4464 0xa0b4884 0xa0d1aff 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:08.922 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x100697 /lib64/libasan.so.8+0xfd19a /lib64/libasan.so.8+0xee859 0x9ec99fb 0x9f2f1f6 0x9f2f30c 0x9f2f3e8 0x9f2faa4 0x9f2fc0b 0x9f3000f 0x9f302cc 0x9f31114 0xa0a9a5b 0xa0a9b55 0x7eff032 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:11.095 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-20T23:50:11.095 INFO:teuthology.orchestra.run.trial057.stderr:Suppressions used: 2026-02-20T23:50:11.095 INFO:teuthology.orchestra.run.trial057.stderr: count bytes template 2026-02-20T23:50:11.095 INFO:teuthology.orchestra.run.trial057.stderr: 1 8 MallocExtension::Register 2026-02-20T23:50:11.096 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-20T23:50:11.096 INFO:teuthology.orchestra.run.trial057.stderr: 2026-02-20T23:50:11.129 DEBUG:teuthology.orchestra.run.trial057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-20T23:50:11.155 DEBUG:teuthology.orchestra.run.trial057:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T23:50:11.308 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T23:50:11.365 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:11,372 seastar - Reactor backend: linux-aio 2026-02-20T23:50:11.374 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:11,381 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T23:50:11.375 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:11,382 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T23:50:11.376 INFO:teuthology.orchestra.run.trial057.stderr:==35457==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T23:50:11.432 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:11,439 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-20T23:50:11.432 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:11,439 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-20T23:50:11.433 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:11,439 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-20T23:50:11.433 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:11,439 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-20T23:50:11.433 INFO:teuthology.orchestra.run.trial057.stderr:DEBUG 2026-02-20 23:50:11,439 [shard 0:main] seastar - reactor::drain 2026-02-20T23:50:13.490 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-20T23:50:13.490 INFO:teuthology.orchestra.run.trial057.stderr:Suppressions used: 2026-02-20T23:50:13.490 INFO:teuthology.orchestra.run.trial057.stderr: count bytes template 2026-02-20T23:50:13.490 INFO:teuthology.orchestra.run.trial057.stderr: 1 8 MallocExtension::Register 2026-02-20T23:50:13.490 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-20T23:50:13.490 INFO:teuthology.orchestra.run.trial057.stderr: 2026-02-20T23:50:13.509 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T23:50:13.510 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T23:50:13.555 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T23:50:13.555 INFO:teuthology.orchestra.run.trial057.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T23:50:13.562 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:13,569 seastar - Reactor backend: linux-aio 2026-02-20T23:50:13.570 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:13,578 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T23:50:13.571 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:13,578 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T23:50:13.574 INFO:teuthology.orchestra.run.trial057.stderr:==35455==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T23:50:13.574 INFO:teuthology.orchestra.run.trial057.stderr:INFO 2026-02-20 23:50:13,582 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T23:50:13.632 INFO:teuthology.orchestra.run.trial057.stderr:DEBUG 2026-02-20 23:50:13,640 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-20T23:50:13.729 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xfd191 /lib64/libasan.so.8+0xef979 0x3422ab8 0x3422c92 0x3422f49 0x342303e 0xa0d4d7d 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:13.729 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15277 0x341fad3 0x9fe50e2 0x9fe5457 0x9fe5a18 0x9fec227 0x9fec907 0xa0d213c 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:13.729 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297db /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x157c7 0x10d1d95 0xf741d0 0x9fe4a64 0x9fe4f2e 0x9fe51a1 0x9fe6ec7 0xa0c10e2 0xa0c185b 0xa0c1d46 0xa0c20d9 0xa0d4d60 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:13.761 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119830 /lib64/libasan.so.8+0xfd239 /lib64/libasan.so.8+0xedcb5 0xe6ca27 0x7bc7042 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-20T23:50:13.820 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x15197 0x116816a 0xf741c5 0x9fe4c31 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5a18 0x9fec227 0x9fec907 0xa0d213c 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:13.822 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x9fe6d13 0xa0c58ee 0xa0c7540 0xa0c7b9a 0xa0c80e0 0xa0d236e 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:13.823 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1583b 0x10d1d95 0xf741d0 0x9fe4a64 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5f05 0x9fe6403 0x9fe6a33 0x9fe6b9d 0xa0c1f5d 0xa0d4d60 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:13.936 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x9fe8710 0x9fe908d 0xa0ce025 0xa0d870b 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:13.936 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x10b6c3 0xe6c75a 0xe6cb15 0xa0d86d8 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:13.936 INFO:teuthology.orchestra.run.trial057.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0xcee22b 0x10d1c72 0xf741d0 0xa0c28bd 0xa0c2eaa 0xa0c322d 0xa0d3384 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T23:50:16.130 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-20T23:50:16.130 INFO:teuthology.orchestra.run.trial057.stderr:Suppressions used: 2026-02-20T23:50:16.130 INFO:teuthology.orchestra.run.trial057.stderr: count bytes template 2026-02-20T23:50:16.130 INFO:teuthology.orchestra.run.trial057.stderr: 1 8 MallocExtension::Register 2026-02-20T23:50:16.131 INFO:teuthology.orchestra.run.trial057.stderr:----------------------------------------------------- 2026-02-20T23:50:16.131 INFO:teuthology.orchestra.run.trial057.stderr: 2026-02-20T23:50:16.165 DEBUG:teuthology.orchestra.run.trial057:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-20T23:50:16.190 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-20T23:50:16.191 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:50:16.191 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-20T23:50:16.254 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:50:16.254 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-20T23:50:16.318 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:50:16.318 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-20T23:50:16.381 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:50:16.382 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-20T23:50:16.446 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:50:16.446 DEBUG:teuthology.orchestra.run.trial057:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-20T23:50:16.502 INFO:tasks.ceph:Adding keys to all mons... 2026-02-20T23:50:16.503 DEBUG:teuthology.orchestra.run.trial057:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-20T23:50:16.565 INFO:teuthology.orchestra.run.trial057.stdout:[mgr.x] 2026-02-20T23:50:16.565 INFO:teuthology.orchestra.run.trial057.stdout: key = AQAm85hpkpqKIBAA4v4VG6PDhIunST1bOJARMw== 2026-02-20T23:50:16.565 INFO:teuthology.orchestra.run.trial057.stdout:[osd.0] 2026-02-20T23:50:16.565 INFO:teuthology.orchestra.run.trial057.stdout: key = AQAr85hpI4fkIxAAb1GIejW05/INxcgr/BnXRw== 2026-02-20T23:50:16.565 INFO:teuthology.orchestra.run.trial057.stdout:[osd.1] 2026-02-20T23:50:16.565 INFO:teuthology.orchestra.run.trial057.stdout: key = AQAw85hpyimlJRAAGgMD7V1lFx69jZ9y/fWkug== 2026-02-20T23:50:16.566 INFO:teuthology.orchestra.run.trial057.stdout:[osd.2] 2026-02-20T23:50:16.566 INFO:teuthology.orchestra.run.trial057.stdout: key = AQA185hpgxdiJhAAtZ3OmWzFXGkeYe5LQSK20Q== 2026-02-20T23:50:16.566 INFO:teuthology.orchestra.run.trial057.stdout:[client.0] 2026-02-20T23:50:16.566 INFO:teuthology.orchestra.run.trial057.stdout: key = AQAm85hpp8BCIxAA/8GLDU7tZLd+3rffOD7GAg== 2026-02-20T23:50:16.567 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-20T23:50:16.655 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T23:50:16.704 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T23:50:16.753 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T23:50:16.801 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-20T23:50:16.851 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-20T23:50:16.851 DEBUG:teuthology.orchestra.run.trial057:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-20T23:50:16.875 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-20T23:50:16.960 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:16.967+0000 7fa588497c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:16.961 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:16.967+0000 7fa588497c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:16.972 DEBUG:teuthology.orchestra.run.trial057:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-20T23:50:16.998 DEBUG:teuthology.orchestra.run.trial057:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-20T23:50:17.055 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-20T23:50:17.056 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-20T23:50:17.056 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-20T23:50:17.098 INFO:tasks.ceph.mon.a:Started 2026-02-20T23:50:17.099 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-20T23:50:17.099 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-20T23:50:17.099 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-20T23:50:17.102 INFO:tasks.ceph.mgr.x:Started 2026-02-20T23:50:17.102 DEBUG:tasks.ceph:set 0 configs 2026-02-20T23:50:17.103 DEBUG:teuthology.orchestra.run.trial057:> sudo ceph --cluster ceph config dump 2026-02-20T23:50:17.163 INFO:tasks.ceph.mon.a.trial057.stderr:2026-02-20T23:50:17.169+0000 7f1d2a13bc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:17.163 INFO:tasks.ceph.mon.a.trial057.stderr:2026-02-20T23:50:17.169+0000 7f1d2a13bc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:17.163 INFO:tasks.ceph.mon.a.trial057.stderr:2026-02-20T23:50:17.169+0000 7f1d2a13bc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:17.166 INFO:tasks.ceph.mgr.x.trial057.stderr:2026-02-20T23:50:17.172+0000 7fcd2d62cfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:17.179 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:17.184+0000 7f2c309f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:17.182 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:17.189+0000 7f2c309f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:17.215 INFO:teuthology.orchestra.run.trial057.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-20T23:50:17.223 INFO:tasks.ceph:Setting crush tunables to default 2026-02-20T23:50:17.223 DEBUG:teuthology.orchestra.run.trial057:> sudo ceph --cluster ceph osd crush tunables default 2026-02-20T23:50:17.297 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:17.300+0000 7f4ac6dcc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:17.300 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:17.306+0000 7f4ac6dcc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:17.335 INFO:teuthology.orchestra.run.trial057.stderr:adjusted tunables profile to default 2026-02-20T23:50:17.343 INFO:tasks.ceph:check_enable_crimson: True 2026-02-20T23:50:17.344 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-20T23:50:17.344 DEBUG:teuthology.orchestra.run.trial057:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-20T23:50:17.370 INFO:tasks.ceph.mgr.x.trial057.stderr:2026-02-20T23:50:17.376+0000 7fcd2d62cfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:17.370 INFO:tasks.ceph.mgr.x.trial057.stderr:2026-02-20T23:50:17.376+0000 7fcd2d62cfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:17.416 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:17.420+0000 7ff5ef9dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:17.418 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:17.425+0000 7ff5ef9dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:17.458 INFO:tasks.ceph.mon.a.trial057.stderr:2026-02-20T23:50:17.464+0000 7f1d223f1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-20T23:50:17.458 INFO:tasks.ceph.mon.a.trial057.stderr:Please be aware that this feature is experimental, untested, 2026-02-20T23:50:17.458 INFO:tasks.ceph.mon.a.trial057.stderr:unsupported, and may result in data corruption, data loss, 2026-02-20T23:50:17.458 INFO:tasks.ceph.mon.a.trial057.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-20T23:50:17.458 INFO:tasks.ceph.mon.a.trial057.stderr:feature with important data. 2026-02-20T23:50:17.458 INFO:tasks.ceph.mon.a.trial057.stderr: 2026-02-20T23:50:18.204 INFO:tasks.ceph.mgr.x.trial057.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-20T23:50:18.204 INFO:tasks.ceph.mgr.x.trial057.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-20T23:50:18.204 INFO:tasks.ceph.mgr.x.trial057.stderr: from numpy import show_config as show_numpy_config 2026-02-20T23:50:18.346 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-20T23:50:18.346 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:50:18.346 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-20T23:50:18.411 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:50:18.411 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-20T23:50:18.474 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:50:18.475 DEBUG:teuthology.orchestra.run.trial057:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-20T23:50:18.539 DEBUG:teuthology.orchestra.run.trial057:> sudo ceph --cluster ceph osd new 90bb8d89-1532-46ee-93ba-f158f3b5246e 0 2026-02-20T23:50:18.656 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:18.658+0000 7f0dbdc11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:18.658 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:18.664+0000 7f0dbdc11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:18.695 INFO:teuthology.orchestra.run.trial057.stdout:0 2026-02-20T23:50:18.704 DEBUG:teuthology.orchestra.run.trial057:> sudo ceph --cluster ceph osd new 6b55a8c6-e150-426a-b559-97bda041f886 1 2026-02-20T23:50:18.776 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:18.778+0000 7f916295c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:18.778 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:18.784+0000 7f916295c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:18.815 INFO:teuthology.orchestra.run.trial057.stdout:1 2026-02-20T23:50:18.824 DEBUG:teuthology.orchestra.run.trial057:> sudo ceph --cluster ceph osd new 859c2643-1ef2-4990-80b2-9f98600684ce 2 2026-02-20T23:50:18.897 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:18.900+0000 7f67cfacb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:18.900 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:18.906+0000 7f67cfacb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:18.937 INFO:teuthology.orchestra.run.trial057.stdout:2 2026-02-20T23:50:18.946 DEBUG:teuthology.orchestra.run.trial057:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-20T23:50:18.975 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-20T23:50:18.975 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-20T23:50:19.017 INFO:tasks.ceph.osd.0:Started 2026-02-20T23:50:19.017 DEBUG:teuthology.orchestra.run.trial057:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-20T23:50:19.046 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-20T23:50:19.046 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-20T23:50:19.089 INFO:tasks.ceph.osd.1:Started 2026-02-20T23:50:19.089 DEBUG:teuthology.orchestra.run.trial057:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-20T23:50:19.115 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-20T23:50:19.116 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-20T23:50:19.158 INFO:tasks.ceph.osd.2:Started 2026-02-20T23:50:19.159 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T23:50:19.233 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:19.237+0000 7f42c405a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:19.235 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:19.242+0000 7f42c405a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:19.268 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:50:19.268 INFO:teuthology.orchestra.run.trial057.stdout:{"epoch":6,"fsid":"797a1d6f-8149-4ca0-b8e3-a68b969d2bab","created":"2026-02-20T23:50:17.182210+0000","modified":"2026-02-20T23:50:18.944022+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T23:50:18.944022+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"90bb8d89-1532-46ee-93ba-f158f3b5246e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6b55a8c6-e150-426a-b559-97bda041f886","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"859c2643-1ef2-4990-80b2-9f98600684ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T23:50:19.277 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-20T23:50:19.277 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-20T23:50:19.579 DEBUG:teuthology.orchestra.run.trial057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T23:50:19.646 INFO:teuthology.misc.health.trial057.stderr:2026-02-20T23:50:19.649+0000 7f98907fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:19.648 INFO:teuthology.misc.health.trial057.stderr:2026-02-20T23:50:19.654+0000 7f98907fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:19.681 INFO:teuthology.misc.health.trial057.stdout: 2026-02-20T23:50:19.681 INFO:teuthology.misc.health.trial057.stdout:{"epoch":6,"fsid":"797a1d6f-8149-4ca0-b8e3-a68b969d2bab","created":"2026-02-20T23:50:17.182210+0000","modified":"2026-02-20T23:50:18.944022+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T23:50:18.944022+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"90bb8d89-1532-46ee-93ba-f158f3b5246e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6b55a8c6-e150-426a-b559-97bda041f886","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"859c2643-1ef2-4990-80b2-9f98600684ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T23:50:19.689 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-20T23:50:20.443 INFO:tasks.ceph.mgr.x.trial057.stderr:2026-02-20T23:50:20.449+0000 7fcc4a17f640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-20T23:50:25.991 DEBUG:teuthology.orchestra.run.trial057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T23:50:26.060 INFO:teuthology.misc.health.trial057.stderr:2026-02-20T23:50:26.064+0000 7f02087dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:26.063 INFO:teuthology.misc.health.trial057.stderr:2026-02-20T23:50:26.069+0000 7f02087dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:26.173 INFO:teuthology.misc.health.trial057.stdout: 2026-02-20T23:50:26.173 INFO:teuthology.misc.health.trial057.stdout:{"epoch":7,"fsid":"797a1d6f-8149-4ca0-b8e3-a68b969d2bab","created":"2026-02-20T23:50:17.182210+0000","modified":"2026-02-20T23:50:25.606811+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T23:50:18.944022+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"90bb8d89-1532-46ee-93ba-f158f3b5246e","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"6b55a8c6-e150-426a-b559-97bda041f886","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"859c2643-1ef2-4990-80b2-9f98600684ce","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T23:50:26.181 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-20T23:50:32.483 DEBUG:teuthology.orchestra.run.trial057:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T23:50:32.553 INFO:teuthology.misc.health.trial057.stderr:2026-02-20T23:50:32.557+0000 7f509776f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:32.555 INFO:teuthology.misc.health.trial057.stderr:2026-02-20T23:50:32.561+0000 7f509776f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:32.669 INFO:teuthology.misc.health.trial057.stdout: 2026-02-20T23:50:32.669 INFO:teuthology.misc.health.trial057.stdout:{"epoch":13,"fsid":"797a1d6f-8149-4ca0-b8e3-a68b969d2bab","created":"2026-02-20T23:50:17.182210+0000","modified":"2026-02-20T23:50:31.633119+0000","last_up_change":"2026-02-20T23:50:28.619857+0000","last_in_change":"2026-02-20T23:50:18.944022+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T23:50:29.465865+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90bb8d89-1532-46ee-93ba-f158f3b5246e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6807","nonce":1825992705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6806","nonce":1825992705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6817","nonce":1825992705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6818","nonce":1825992705}]},"public_addr":"10.20.193.57:6807/1825992705","cluster_addr":"10.20.193.57:6806/1825992705","heartbeat_back_addr":"10.20.193.57:6817/1825992705","heartbeat_front_addr":"10.20.193.57:6818/1825992705","state":["exists","up"]},{"osd":1,"uuid":"6b55a8c6-e150-426a-b559-97bda041f886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6803","nonce":1979508686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6802","nonce":1979508686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6810","nonce":1979508686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6809","nonce":1979508686}]},"public_addr":"10.20.193.57:6803/1979508686","cluster_addr":"10.20.193.57:6802/1979508686","heartbeat_back_addr":"10.20.193.57:6810/1979508686","heartbeat_front_addr":"10.20.193.57:6809/1979508686","state":["exists","up"]},{"osd":2,"uuid":"859c2643-1ef2-4990-80b2-9f98600684ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6804","nonce":1165223019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6805","nonce":1165223019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6814","nonce":1165223019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6815","nonce":1165223019}]},"public_addr":"10.20.193.57:6804/1165223019","cluster_addr":"10.20.193.57:6805/1165223019","heartbeat_back_addr":"10.20.193.57:6814/1165223019","heartbeat_front_addr":"10.20.193.57:6815/1165223019","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T23:50:32.679 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-20T23:50:32.679 INFO:tasks.ceph:Creating RBD pool 2026-02-20T23:50:32.679 DEBUG:teuthology.orchestra.run.trial057:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-20T23:50:32.755 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:32.758+0000 7ff62b85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:32.757 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:32.763+0000 7ff62b85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:33.634 INFO:teuthology.orchestra.run.trial057.stderr:pool 'rbd' created 2026-02-20T23:50:33.646 DEBUG:teuthology.orchestra.run.trial057:> rbd --cluster ceph pool init rbd 2026-02-20T23:50:33.689 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:33.695+0000 7fd4cb4fb240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:33.689 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:33.695+0000 7fd4cb4fb240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:33.692 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:33.698+0000 7fd4cb4fb240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:36.673 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-20T23:50:36.674 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-20T23:50:36.674 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-20T23:50:36.788 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:36.792+0000 7fcb1d25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:36.791 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:36.797+0000 7fcb1d25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:36.900 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:50:36.909 INFO:teuthology.orchestra.run.trial057.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-20T23:50:36.909 INFO:tasks.ceph_manager:config epoch is 1 2026-02-20T23:50:36.909 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-20T23:50:36.909 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-20T23:50:36.910 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-20T23:50:36.986 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:36.989+0000 7f9513bdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:36.988 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:36.994+0000 7f9513bdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.132 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:50:37.142 INFO:teuthology.orchestra.run.trial057.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6800","nonce":1827675630},{"type":"v1","addr":"10.20.193.57:6801","nonce":1827675630}]},"active_addr":"10.20.193.57:6801/1827675630","active_change":"2026-02-20T23:50:19.445222+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.57:0","nonce":3582621081}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.57:0","nonce":3760354051}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.57:0","nonce":3016566392}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.57:0","nonce":1397052556}]}]} 2026-02-20T23:50:37.146 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-20T23:50:37.146 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-20T23:50:37.146 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T23:50:37.223 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:37.227+0000 7f4c0759c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.225 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:37.232+0000 7f4c0759c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.338 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:50:37.338 INFO:teuthology.orchestra.run.trial057.stdout:{"epoch":17,"fsid":"797a1d6f-8149-4ca0-b8e3-a68b969d2bab","created":"2026-02-20T23:50:17.182210+0000","modified":"2026-02-20T23:50:36.648744+0000","last_up_change":"2026-02-20T23:50:28.619857+0000","last_in_change":"2026-02-20T23:50:18.944022+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T23:50:29.465865+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-20T23:50:32.879033+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90bb8d89-1532-46ee-93ba-f158f3b5246e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6807","nonce":1825992705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6806","nonce":1825992705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6817","nonce":1825992705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6818","nonce":1825992705}]},"public_addr":"10.20.193.57:6807/1825992705","cluster_addr":"10.20.193.57:6806/1825992705","heartbeat_back_addr":"10.20.193.57:6817/1825992705","heartbeat_front_addr":"10.20.193.57:6818/1825992705","state":["exists","up"]},{"osd":1,"uuid":"6b55a8c6-e150-426a-b559-97bda041f886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6803","nonce":1979508686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6802","nonce":1979508686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6810","nonce":1979508686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6809","nonce":1979508686}]},"public_addr":"10.20.193.57:6803/1979508686","cluster_addr":"10.20.193.57:6802/1979508686","heartbeat_back_addr":"10.20.193.57:6810/1979508686","heartbeat_front_addr":"10.20.193.57:6809/1979508686","state":["exists","up"]},{"osd":2,"uuid":"859c2643-1ef2-4990-80b2-9f98600684ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6804","nonce":1165223019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6805","nonce":1165223019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6814","nonce":1165223019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6815","nonce":1165223019}]},"public_addr":"10.20.193.57:6804/1165223019","cluster_addr":"10.20.193.57:6805/1165223019","heartbeat_back_addr":"10.20.193.57:6814/1165223019","heartbeat_front_addr":"10.20.193.57:6815/1165223019","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T23:50:37.347 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-20T23:50:37.348 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T23:50:37.424 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:37.428+0000 7fea066c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.426 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:37.432+0000 7fea066c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.537 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:50:37.537 INFO:teuthology.orchestra.run.trial057.stdout:{"epoch":17,"fsid":"797a1d6f-8149-4ca0-b8e3-a68b969d2bab","created":"2026-02-20T23:50:17.182210+0000","modified":"2026-02-20T23:50:36.648744+0000","last_up_change":"2026-02-20T23:50:28.619857+0000","last_in_change":"2026-02-20T23:50:18.944022+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T23:50:29.465865+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-20T23:50:32.879033+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90bb8d89-1532-46ee-93ba-f158f3b5246e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6807","nonce":1825992705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6806","nonce":1825992705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6817","nonce":1825992705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6818","nonce":1825992705}]},"public_addr":"10.20.193.57:6807/1825992705","cluster_addr":"10.20.193.57:6806/1825992705","heartbeat_back_addr":"10.20.193.57:6817/1825992705","heartbeat_front_addr":"10.20.193.57:6818/1825992705","state":["exists","up"]},{"osd":1,"uuid":"6b55a8c6-e150-426a-b559-97bda041f886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6803","nonce":1979508686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6802","nonce":1979508686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6810","nonce":1979508686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6809","nonce":1979508686}]},"public_addr":"10.20.193.57:6803/1979508686","cluster_addr":"10.20.193.57:6802/1979508686","heartbeat_back_addr":"10.20.193.57:6810/1979508686","heartbeat_front_addr":"10.20.193.57:6809/1979508686","state":["exists","up"]},{"osd":2,"uuid":"859c2643-1ef2-4990-80b2-9f98600684ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6804","nonce":1165223019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6805","nonce":1165223019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6814","nonce":1165223019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6815","nonce":1165223019}]},"public_addr":"10.20.193.57:6804/1165223019","cluster_addr":"10.20.193.57:6805/1165223019","heartbeat_back_addr":"10.20.193.57:6814/1165223019","heartbeat_front_addr":"10.20.193.57:6815/1165223019","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T23:50:37.546 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-20T23:50:37.547 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-20T23:50:37.548 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-20T23:50:37.629 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:37.632+0000 7f6c0ec51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.629 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:37.632+0000 7f2332251640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.630 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:37.632+0000 7f4a1916f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.632 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:37.638+0000 7f2332251640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.632 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:37.638+0000 7f6c0ec51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.633 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:37.638+0000 7f4a1916f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.648 INFO:teuthology.orchestra.run.trial057.stdout:38654705674 2026-02-20T23:50:37.649 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-20T23:50:37.650 INFO:teuthology.orchestra.run.trial057.stdout:42949672969 2026-02-20T23:50:37.651 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-20T23:50:37.652 INFO:teuthology.orchestra.run.trial057.stdout:42949672969 2026-02-20T23:50:37.652 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-20T23:50:37.727 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:37.729+0000 7fab61a46640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.729 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:37.735+0000 7fab61a46640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.771 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:37.775+0000 7ff5fbb0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.772 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:37.775+0000 7f89cbacb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.774 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:37.780+0000 7ff5fbb0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.775 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:37.781+0000 7f89cbacb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:37.843 INFO:teuthology.orchestra.run.trial057.stdout:38654705673 2026-02-20T23:50:37.853 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705673 for osd.1 2026-02-20T23:50:37.886 INFO:teuthology.orchestra.run.trial057.stdout:42949672964 2026-02-20T23:50:37.892 INFO:teuthology.orchestra.run.trial057.stdout:42949672965 2026-02-20T23:50:37.897 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672964 for osd.2 2026-02-20T23:50:37.903 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672965 for osd.0 2026-02-20T23:50:38.855 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-20T23:50:38.898 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-20T23:50:38.904 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-20T23:50:38.934 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:38.937+0000 7f8d203b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:38.936 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:38.942+0000 7f8d203b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:38.975 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:38.979+0000 7fc5005e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:38.978 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:38.984+0000 7fc5005e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:39.010 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:39.013+0000 7fadcd446640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:39.012 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:39.018+0000 7fadcd446640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:39.054 INFO:teuthology.orchestra.run.trial057.stdout:38654705673 2026-02-20T23:50:39.065 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705673 for osd.1 2026-02-20T23:50:39.096 INFO:teuthology.orchestra.run.trial057.stdout:42949672964 2026-02-20T23:50:39.108 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672964 for osd.2 2026-02-20T23:50:39.123 INFO:teuthology.orchestra.run.trial057.stdout:42949672965 2026-02-20T23:50:39.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672965 for osd.0 2026-02-20T23:50:40.066 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-20T23:50:40.108 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-20T23:50:40.133 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-20T23:50:40.144 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:40.147+0000 7f9f7e59c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:40.147 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:40.153+0000 7f9f7e59c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:40.188 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:40.192+0000 7fa71cccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:40.191 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:40.197+0000 7fa71cccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:40.220 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:40.223+0000 7f72f030b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:40.222 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:40.228+0000 7f72f030b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:40.261 INFO:teuthology.orchestra.run.trial057.stdout:38654705674 2026-02-20T23:50:40.272 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705674 for osd.1 2026-02-20T23:50:40.273 DEBUG:teuthology.parallel:result is None 2026-02-20T23:50:40.304 INFO:teuthology.orchestra.run.trial057.stdout:42949672969 2026-02-20T23:50:40.316 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672969 for osd.2 2026-02-20T23:50:40.317 DEBUG:teuthology.parallel:result is None 2026-02-20T23:50:40.333 INFO:teuthology.orchestra.run.trial057.stdout:42949672970 2026-02-20T23:50:40.343 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672970 for osd.0 2026-02-20T23:50:40.343 DEBUG:teuthology.parallel:result is None 2026-02-20T23:50:40.343 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-20T23:50:40.343 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T23:50:40.459 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:40.462+0000 7f391501a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:40.462 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:40.467+0000 7f391501a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:40.570 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:50:40.570 INFO:teuthology.orchestra.run.trial057.stderr:dumped all 2026-02-20T23:50:40.579 INFO:teuthology.orchestra.run.trial057.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-20T23:50:39.449585+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62917052,"kb_used_data":62917052,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94369300,"statfs":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.813032"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.662601+0000","last_change":"2026-02-20T23:50:36.667747+0000","last_active":"2026-02-20T23:50:36.662601+0000","last_peered":"2026-02-20T23:50:36.662601+0000","last_clean":"2026-02-20T23:50:36.662601+0000","last_became_active":"2026-02-20T23:50:34.683215+0000","last_became_peered":"2026-02-20T23:50:34.683215+0000","last_unstale":"2026-02-20T23:50:36.662601+0000","last_undegraded":"2026-02-20T23:50:36.662601+0000","last_fullsized":"2026-02-20T23:50:36.662601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.664389+0000","last_change":"2026-02-20T23:50:36.668336+0000","last_active":"2026-02-20T23:50:36.664389+0000","last_peered":"2026-02-20T23:50:36.664389+0000","last_clean":"2026-02-20T23:50:36.664389+0000","last_became_active":"2026-02-20T23:50:34.689053+0000","last_became_peered":"2026-02-20T23:50:34.689053+0000","last_unstale":"2026-02-20T23:50:36.664389+0000","last_undegraded":"2026-02-20T23:50:36.664389+0000","last_fullsized":"2026-02-20T23:50:36.664389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.665810+0000","last_change":"2026-02-20T23:50:36.670710+0000","last_active":"2026-02-20T23:50:36.665810+0000","last_peered":"2026-02-20T23:50:36.665810+0000","last_clean":"2026-02-20T23:50:36.665810+0000","last_became_active":"2026-02-20T23:50:34.695736+0000","last_became_peered":"2026-02-20T23:50:34.695736+0000","last_unstale":"2026-02-20T23:50:36.665810+0000","last_undegraded":"2026-02-20T23:50:36.665810+0000","last_fullsized":"2026-02-20T23:50:36.665810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.664670+0000","last_change":"2026-02-20T23:50:36.667973+0000","last_active":"2026-02-20T23:50:36.664670+0000","last_peered":"2026-02-20T23:50:36.664670+0000","last_clean":"2026-02-20T23:50:36.664670+0000","last_became_active":"2026-02-20T23:50:34.693081+0000","last_became_peered":"2026-02-20T23:50:34.693081+0000","last_unstale":"2026-02-20T23:50:36.664670+0000","last_undegraded":"2026-02-20T23:50:36.664670+0000","last_fullsized":"2026-02-20T23:50:36.664670+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.666311+0000","last_change":"2026-02-20T23:50:36.666311+0000","last_active":"2026-02-20T23:50:36.666311+0000","last_peered":"2026-02-20T23:50:36.666311+0000","last_clean":"2026-02-20T23:50:36.666311+0000","last_became_active":"2026-02-20T23:50:34.689002+0000","last_became_peered":"2026-02-20T23:50:34.689002+0000","last_unstale":"2026-02-20T23:50:36.666311+0000","last_undegraded":"2026-02-20T23:50:36.666311+0000","last_fullsized":"2026-02-20T23:50:36.666311+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.663308+0000","last_change":"2026-02-20T23:50:36.666714+0000","last_active":"2026-02-20T23:50:36.663308+0000","last_peered":"2026-02-20T23:50:36.663308+0000","last_clean":"2026-02-20T23:50:36.663308+0000","last_became_active":"2026-02-20T23:50:34.690728+0000","last_became_peered":"2026-02-20T23:50:34.690728+0000","last_unstale":"2026-02-20T23:50:36.663308+0000","last_undegraded":"2026-02-20T23:50:36.663308+0000","last_fullsized":"2026-02-20T23:50:36.663308+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.660605+0000","last_change":"2026-02-20T23:50:36.664790+0000","last_active":"2026-02-20T23:50:36.660605+0000","last_peered":"2026-02-20T23:50:36.660605+0000","last_clean":"2026-02-20T23:50:36.660605+0000","last_became_active":"2026-02-20T23:50:34.699338+0000","last_became_peered":"2026-02-20T23:50:34.699338+0000","last_unstale":"2026-02-20T23:50:36.660605+0000","last_undegraded":"2026-02-20T23:50:36.660605+0000","last_fullsized":"2026-02-20T23:50:36.660605+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.664436+0000","last_change":"2026-02-20T23:50:36.670298+0000","last_active":"2026-02-20T23:50:36.664436+0000","last_peered":"2026-02-20T23:50:36.664436+0000","last_clean":"2026-02-20T23:50:36.664436+0000","last_became_active":"2026-02-20T23:50:34.682318+0000","last_became_peered":"2026-02-20T23:50:34.682318+0000","last_unstale":"2026-02-20T23:50:36.664436+0000","last_undegraded":"2026-02-20T23:50:36.664436+0000","last_fullsized":"2026-02-20T23:50:36.664436+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.662931+0000","last_change":"2026-02-20T23:50:30.669934+0000","last_active":"2026-02-20T23:50:36.662931+0000","last_peered":"2026-02-20T23:50:36.662931+0000","last_clean":"2026-02-20T23:50:36.662931+0000","last_became_active":"2026-02-20T23:50:30.660319+0000","last_became_peered":"2026-02-20T23:50:30.660319+0000","last_unstale":"2026-02-20T23:50:36.662931+0000","last_undegraded":"2026-02-20T23:50:36.662931+0000","last_fullsized":"2026-02-20T23:50:36.662931+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:29.624739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:29.624739+0000","last_clean_scrub_stamp":"2026-02-20T23:50:29.624739+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972652,"kb_used_data":20972652,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456132,"statfs":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972580,"kb_used_data":20972580,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456204,"statfs":{"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971820,"kb_used_data":20971820,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456964,"statfs":{"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T23:50:40.580 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T23:50:40.658 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:40.662+0000 7f287585a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:40.660 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:40.666+0000 7f287585a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:40.770 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:50:40.771 INFO:teuthology.orchestra.run.trial057.stderr:dumped all 2026-02-20T23:50:40.780 INFO:teuthology.orchestra.run.trial057.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-20T23:50:39.449585+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62917052,"kb_used_data":62917052,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94369300,"statfs":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.813032"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.662601+0000","last_change":"2026-02-20T23:50:36.667747+0000","last_active":"2026-02-20T23:50:36.662601+0000","last_peered":"2026-02-20T23:50:36.662601+0000","last_clean":"2026-02-20T23:50:36.662601+0000","last_became_active":"2026-02-20T23:50:34.683215+0000","last_became_peered":"2026-02-20T23:50:34.683215+0000","last_unstale":"2026-02-20T23:50:36.662601+0000","last_undegraded":"2026-02-20T23:50:36.662601+0000","last_fullsized":"2026-02-20T23:50:36.662601+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.664389+0000","last_change":"2026-02-20T23:50:36.668336+0000","last_active":"2026-02-20T23:50:36.664389+0000","last_peered":"2026-02-20T23:50:36.664389+0000","last_clean":"2026-02-20T23:50:36.664389+0000","last_became_active":"2026-02-20T23:50:34.689053+0000","last_became_peered":"2026-02-20T23:50:34.689053+0000","last_unstale":"2026-02-20T23:50:36.664389+0000","last_undegraded":"2026-02-20T23:50:36.664389+0000","last_fullsized":"2026-02-20T23:50:36.664389+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.665810+0000","last_change":"2026-02-20T23:50:36.670710+0000","last_active":"2026-02-20T23:50:36.665810+0000","last_peered":"2026-02-20T23:50:36.665810+0000","last_clean":"2026-02-20T23:50:36.665810+0000","last_became_active":"2026-02-20T23:50:34.695736+0000","last_became_peered":"2026-02-20T23:50:34.695736+0000","last_unstale":"2026-02-20T23:50:36.665810+0000","last_undegraded":"2026-02-20T23:50:36.665810+0000","last_fullsized":"2026-02-20T23:50:36.665810+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.664670+0000","last_change":"2026-02-20T23:50:36.667973+0000","last_active":"2026-02-20T23:50:36.664670+0000","last_peered":"2026-02-20T23:50:36.664670+0000","last_clean":"2026-02-20T23:50:36.664670+0000","last_became_active":"2026-02-20T23:50:34.693081+0000","last_became_peered":"2026-02-20T23:50:34.693081+0000","last_unstale":"2026-02-20T23:50:36.664670+0000","last_undegraded":"2026-02-20T23:50:36.664670+0000","last_fullsized":"2026-02-20T23:50:36.664670+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.666311+0000","last_change":"2026-02-20T23:50:36.666311+0000","last_active":"2026-02-20T23:50:36.666311+0000","last_peered":"2026-02-20T23:50:36.666311+0000","last_clean":"2026-02-20T23:50:36.666311+0000","last_became_active":"2026-02-20T23:50:34.689002+0000","last_became_peered":"2026-02-20T23:50:34.689002+0000","last_unstale":"2026-02-20T23:50:36.666311+0000","last_undegraded":"2026-02-20T23:50:36.666311+0000","last_fullsized":"2026-02-20T23:50:36.666311+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.663308+0000","last_change":"2026-02-20T23:50:36.666714+0000","last_active":"2026-02-20T23:50:36.663308+0000","last_peered":"2026-02-20T23:50:36.663308+0000","last_clean":"2026-02-20T23:50:36.663308+0000","last_became_active":"2026-02-20T23:50:34.690728+0000","last_became_peered":"2026-02-20T23:50:34.690728+0000","last_unstale":"2026-02-20T23:50:36.663308+0000","last_undegraded":"2026-02-20T23:50:36.663308+0000","last_fullsized":"2026-02-20T23:50:36.663308+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.660605+0000","last_change":"2026-02-20T23:50:36.664790+0000","last_active":"2026-02-20T23:50:36.660605+0000","last_peered":"2026-02-20T23:50:36.660605+0000","last_clean":"2026-02-20T23:50:36.660605+0000","last_became_active":"2026-02-20T23:50:34.699338+0000","last_became_peered":"2026-02-20T23:50:34.699338+0000","last_unstale":"2026-02-20T23:50:36.660605+0000","last_undegraded":"2026-02-20T23:50:36.660605+0000","last_fullsized":"2026-02-20T23:50:36.660605+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.664436+0000","last_change":"2026-02-20T23:50:36.670298+0000","last_active":"2026-02-20T23:50:36.664436+0000","last_peered":"2026-02-20T23:50:36.664436+0000","last_clean":"2026-02-20T23:50:36.664436+0000","last_became_active":"2026-02-20T23:50:34.682318+0000","last_became_peered":"2026-02-20T23:50:34.682318+0000","last_unstale":"2026-02-20T23:50:36.664436+0000","last_undegraded":"2026-02-20T23:50:36.664436+0000","last_fullsized":"2026-02-20T23:50:36.664436+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T23:50:36.662931+0000","last_change":"2026-02-20T23:50:30.669934+0000","last_active":"2026-02-20T23:50:36.662931+0000","last_peered":"2026-02-20T23:50:36.662931+0000","last_clean":"2026-02-20T23:50:36.662931+0000","last_became_active":"2026-02-20T23:50:30.660319+0000","last_became_peered":"2026-02-20T23:50:30.660319+0000","last_unstale":"2026-02-20T23:50:36.662931+0000","last_undegraded":"2026-02-20T23:50:36.662931+0000","last_fullsized":"2026-02-20T23:50:36.662931+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:29.624739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:29.624739+0000","last_clean_scrub_stamp":"2026-02-20T23:50:29.624739+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972652,"kb_used_data":20972652,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456132,"statfs":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672970,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972580,"kb_used_data":20972580,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456204,"statfs":{"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971820,"kb_used_data":20971820,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456964,"statfs":{"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T23:50:40.781 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-20T23:50:40.782 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-20T23:50:40.782 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-20T23:50:40.782 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-20T23:50:40.856 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:40.861+0000 7f1fbc7e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:40.858 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-20T23:50:40.864+0000 7f1fbc7e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T23:50:40.996 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-20T23:50:40.996 INFO:teuthology.orchestra.run.trial057.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-20T23:50:41.005 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-20T23:50:41.005 INFO:teuthology.run_tasks:Running task workunit... 2026-02-20T23:50:41.026 INFO:tasks.workunit:Pulling workunits from ref bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-20T23:50:41.027 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-20T23:50:41.027 DEBUG:teuthology.orchestra.run.trial057:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-20T23:50:41.044 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T23:50:41.045 INFO:teuthology.orchestra.run.trial057.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-20T23:50:41.045 DEBUG:teuthology.orchestra.run.trial057:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-20T23:50:41.101 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-20T23:50:41.102 DEBUG:teuthology.orchestra.run.trial057:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-20T23:50:41.158 INFO:tasks.workunit:timeout=3h 2026-02-20T23:50:41.158 INFO:tasks.workunit:cleanup=True 2026-02-20T23:50:41.159 DEBUG:teuthology.orchestra.run.trial057:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://git.ceph.com/ceph-ci.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-20T23:50:41.221 INFO:tasks.workunit.client.0.trial057.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-20T23:51:08.359 INFO:tasks.workunit.client.0.trial057.stderr:Note: switching to 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'. 2026-02-20T23:51:08.359 INFO:tasks.workunit.client.0.trial057.stderr: 2026-02-20T23:51:08.359 INFO:tasks.workunit.client.0.trial057.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-20T23:51:08.359 INFO:tasks.workunit.client.0.trial057.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-20T23:51:08.359 INFO:tasks.workunit.client.0.trial057.stderr:state without impacting any branches by switching back to a branch. 2026-02-20T23:51:08.359 INFO:tasks.workunit.client.0.trial057.stderr: 2026-02-20T23:51:08.359 INFO:tasks.workunit.client.0.trial057.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-20T23:51:08.360 INFO:tasks.workunit.client.0.trial057.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-20T23:51:08.360 INFO:tasks.workunit.client.0.trial057.stderr: 2026-02-20T23:51:08.360 INFO:tasks.workunit.client.0.trial057.stderr: git switch -c 2026-02-20T23:51:08.360 INFO:tasks.workunit.client.0.trial057.stderr: 2026-02-20T23:51:08.360 INFO:tasks.workunit.client.0.trial057.stderr:Or undo this operation with: 2026-02-20T23:51:08.360 INFO:tasks.workunit.client.0.trial057.stderr: 2026-02-20T23:51:08.360 INFO:tasks.workunit.client.0.trial057.stderr: git switch - 2026-02-20T23:51:08.360 INFO:tasks.workunit.client.0.trial057.stderr: 2026-02-20T23:51:08.360 INFO:tasks.workunit.client.0.trial057.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-20T23:51:08.360 INFO:tasks.workunit.client.0.trial057.stderr: 2026-02-20T23:51:08.360 INFO:tasks.workunit.client.0.trial057.stderr:HEAD is now at bcdd429b2cc qa/suites/crimson-rados: add fio test case for osd shard number changes upon restart for 3 osd 2026-02-20T23:51:08.367 DEBUG:teuthology.orchestra.run.trial057:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-20T23:51:08.425 INFO:tasks.workunit.client.0.trial057.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-20T23:51:08.426 INFO:tasks.workunit.client.0.trial057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-20T23:51:08.427 INFO:tasks.workunit.client.0.trial057.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-20T23:51:08.516 INFO:tasks.workunit.client.0.trial057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-20T23:51:08.546 INFO:tasks.workunit.client.0.trial057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-20T23:51:08.573 INFO:tasks.workunit.client.0.trial057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-20T23:51:08.575 INFO:tasks.workunit.client.0.trial057.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-20T23:51:08.575 INFO:tasks.workunit.client.0.trial057.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-20T23:51:08.599 INFO:tasks.workunit.client.0.trial057.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-20T23:51:08.603 DEBUG:teuthology.orchestra.run.trial057:> set -ex 2026-02-20T23:51:08.603 DEBUG:teuthology.orchestra.run.trial057:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-20T23:51:08.660 INFO:tasks.workunit:Running workunits matching rbd/test_librbd_python.sh on client.0... 2026-02-20T23:51:08.662 INFO:tasks.workunit:Running workunit rbd/test_librbd_python.sh... 2026-02-20T23:51:08.663 DEBUG:teuthology.orchestra.run.trial057:workunit test rbd/test_librbd_python.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 RBD_FEATURES=61 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd_python.sh 2026-02-20T23:51:08.720 INFO:tasks.workunit.client.0.trial057.stderr:++ dirname /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/test_librbd_python.sh 2026-02-20T23:51:08.721 INFO:tasks.workunit.client.0.trial057.stderr:+ relpath=/home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../../src/test/pybind 2026-02-20T23:51:08.721 INFO:tasks.workunit.client.0.trial057.stderr:+ '[' -n '' ']' 2026-02-20T23:51:08.721 INFO:tasks.workunit.client.0.trial057.stderr:+ python3 -m pytest -v /home/ubuntu/cephtest/clone.client.0/qa/workunits/rbd/../../../src/test/pybind/test_rbd.py 2026-02-20T23:51:08.795 INFO:tasks.workunit.client.0.trial057.stdout:============================= test session starts ============================== 2026-02-20T23:51:08.795 INFO:tasks.workunit.client.0.trial057.stdout:platform linux -- Python 3.9.25, pytest-6.2.2, py-1.10.0, pluggy-0.13.1 -- /usr/bin/python3 2026-02-20T23:51:08.795 INFO:tasks.workunit.client.0.trial057.stdout:cachedir: .pytest_cache 2026-02-20T23:51:08.795 INFO:tasks.workunit.client.0.trial057.stdout:rootdir: /home/ubuntu/cephtest/clone.client.0/src/test/pybind, configfile: pytest.ini 2026-02-20T23:51:09.093 INFO:tasks.workunit.client.0.trial057.stdout:collecting ... collected 196 items 2026-02-20T23:51:09.093 INFO:tasks.workunit.client.0.trial057.stdout: 2026-02-20T23:51:12.237 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_version PASSED [ 0%] 2026-02-20T23:51:12.730 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_create PASSED [ 1%] 2026-02-20T23:51:17.124 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_create_defaults PASSED [ 1%] 2026-02-20T23:51:17.812 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_context_manager PASSED [ 2%] 2026-02-20T23:51:19.762 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_open_read_only PASSED [ 2%] 2026-02-20T23:51:20.909 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_open_dne PASSED [ 3%] 2026-02-20T23:51:22.106 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_open_readonly_dne PASSED [ 3%] 2026-02-20T23:51:22.741 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_open_by_id PASSED [ 4%] 2026-02-20T23:51:23.320 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_aio_open PASSED [ 4%] 2026-02-20T23:51:23.378 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_remove_dne PASSED [ 5%] 2026-02-20T23:51:23.387 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list_empty PASSED [ 5%] 2026-02-20T23:51:23.958 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list PASSED [ 6%] 2026-02-20T23:51:24.471 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_remove_with_progress PASSED [ 6%] 2026-02-20T23:51:25.107 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_remove_canceled PASSED [ 7%] 2026-02-20T23:51:25.595 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_remove_with_progress_except PASSED [ 7%] 2026-02-20T23:51:26.426 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_rename PASSED [ 8%] 2026-02-20T23:51:28.200 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_pool_metadata PASSED [ 8%] 2026-02-20T23:51:28.245 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_config_list PASSED [ 9%] 2026-02-20T23:51:28.280 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_pool_config_set_and_get_and_remove PASSED [ 9%] 2026-02-20T23:51:28.469 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_namespaces PASSED [ 10%] 2026-02-20T23:51:31.748 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_pool_stats PASSED [ 10%] 2026-02-20T23:51:31.748 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_features_to_string PASSED [ 11%] 2026-02-20T23:51:31.749 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_features_from_string PASSED [ 11%] 2026-02-20T23:51:32.324 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_update_features SKIPPED [ 12%] 2026-02-20T23:51:32.877 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_with_params SKIPPED [ 12%] 2026-02-20T23:51:33.456 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_id PASSED [ 13%] 2026-02-20T23:51:34.025 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_block_name_prefix PASSED [ 13%] 2026-02-20T23:51:34.638 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_data_pool_id PASSED [ 14%] 2026-02-20T23:51:35.228 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_timestamp PASSED [ 14%] 2026-02-20T23:51:35.788 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_access_timestamp PASSED [ 15%] 2026-02-20T23:51:36.361 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_modify_timestamp PASSED [ 15%] 2026-02-20T23:51:37.057 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_invalidate_cache PASSED [ 16%] 2026-02-20T23:51:37.621 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_stat PASSED [ 16%] 2026-02-20T23:51:38.202 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_flags PASSED [ 17%] 2026-02-20T23:51:38.848 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_image_auto_close PASSED [ 17%] 2026-02-20T23:51:39.428 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_use_after_close PASSED [ 18%] 2026-02-20T23:51:40.137 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write PASSED [ 18%] 2026-02-20T23:51:40.836 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write_with_fadvise_flags PASSED [ 19%] 2026-02-20T23:51:41.555 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write_zeroes PASSED [ 19%] 2026-02-20T23:51:42.258 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write_zeroes_thick_provision PASSED [ 20%] 2026-02-20T23:51:42.856 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_read PASSED [ 20%] 2026-02-20T23:51:43.457 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_read_with_fadvise_flags PASSED [ 21%] 2026-02-20T23:51:44.200 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_large_write PASSED [ 21%] 2026-02-20T23:51:44.788 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_large_read PASSED [ 22%] 2026-02-20T23:51:45.495 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_write_read PASSED [ 22%] 2026-02-20T23:51:46.073 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_read_bad_offset PASSED [ 23%] 2026-02-20T23:51:46.778 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_resize PASSED [ 23%] 2026-02-20T23:51:47.441 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_resize_allow_shrink_False PASSED [ 24%] 2026-02-20T23:51:51.793 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_size PASSED [ 25%] 2026-02-20T23:51:52.579 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_resize_down PASSED [ 25%] 2026-02-20T23:51:53.451 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_resize_bytes PASSED [ 26%] 2026-02-20T23:51:54.931 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_copy PASSED [ 26%] 2026-02-20T23:51:56.444 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_copy2 PASSED [ 27%] 2026-02-20T23:51:57.052 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_copy3 SKIPPED [ 27%] 2026-02-20T23:52:01.401 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy PASSED [ 28%] 2026-02-20T23:52:08.883 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v1_to_v1 PASSED [ 28%] 2026-02-20T23:52:15.941 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v1_to_v2 PASSED [ 29%] 2026-02-20T23:52:22.952 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v2_to_v1 PASSED [ 29%] 2026-02-20T23:52:29.999 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v2_to_v2 PASSED [ 30%] 2026-02-20T23:52:36.997 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v1_flatten PASSED [ 30%] 2026-02-20T23:52:44.018 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_deep_copy_clone_v2_flatten PASSED [ 31%] 2026-02-20T23:52:46.015 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_snap PASSED [ 31%] 2026-02-20T23:52:47.769 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_snap_exists PASSED [ 32%] 2026-02-20T23:52:54.024 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_create_snap_flags PASSED [ 32%] 2026-02-20T23:52:57.782 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_list_snaps PASSED [ 33%] 2026-02-20T23:53:00.039 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_list_snaps_iterator_auto_close PASSED [ 33%] 2026-02-20T23:53:02.070 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_snap PASSED [ 34%] 2026-02-20T23:53:02.706 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_snap_not_found PASSED [ 34%] 2026-02-20T23:53:05.067 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_snap2 PASSED [ 35%] 2026-02-20T23:53:07.097 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_snap_by_id PASSED [ 35%] 2026-02-20T23:53:09.785 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_rename_snap PASSED [ 36%] 2026-02-20T23:53:11.828 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_protect_snap PASSED [ 36%] 2026-02-20T23:53:13.793 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_snap_exists PASSED [ 37%] 2026-02-20T23:53:15.802 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_snap_timestamp PASSED [ 37%] 2026-02-20T23:53:23.809 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_limit_snaps PASSED [ 38%] 2026-02-20T23:53:24.536 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_with_exclusive_lock PASSED [ 38%] 2026-02-20T23:53:25.139 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_with_snap SKIPPED [ 39%] 2026-02-20T23:53:25.748 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_remove_with_watcher SKIPPED [ 39%] 2026-02-20T23:53:27.814 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_rollback_to_snap PASSED [ 40%] 2026-02-20T23:53:29.823 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_rollback_to_snap_sparse PASSED [ 40%] 2026-02-20T23:53:33.890 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_rollback_with_resize PASSED [ 41%] 2026-02-20T23:53:35.883 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap PASSED [ 41%] 2026-02-20T23:53:37.835 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_no_snap PASSED [ 42%] 2026-02-20T23:53:39.905 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap_by_id PASSED [ 42%] 2026-02-20T23:53:45.884 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_snap_get_name PASSED [ 43%] 2026-02-20T23:53:51.916 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_snap_get_id PASSED [ 43%] 2026-02-20T23:53:53.946 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap_sparse PASSED [ 44%] 2026-02-21T00:02:42.955 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_many_snaps PASSED [ 44%] 2026-02-21T00:02:45.027 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap_deleted PASSED [ 45%] 2026-02-21T00:02:49.030 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_set_snap_recreated PASSED [ 45%] 2026-02-21T00:02:49.739 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_lock_unlock PASSED [ 46%] 2026-02-21T00:02:51.176 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_list_lockers PASSED [ 46%] 2026-02-21T00:02:58.008 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_diff_iterate PASSED [ 47%] 2026-02-21T00:03:00.127 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_diff_iterate_from_trash_snap PASSED [ 47%] 2026-02-21T00:03:03.065 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_diff_iterate_exclude_parent PASSED [ 48%] 2026-02-21T00:03:03.718 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_read PASSED [ 48%] 2026-02-21T00:03:04.455 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_write PASSED [ 49%] 2026-02-21T00:03:05.183 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_discard PASSED [ 50%] 2026-02-21T00:03:05.916 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_write_zeroes PASSED [ 50%] 2026-02-21T00:03:06.586 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_aio_flush PASSED [ 51%] 2026-02-21T00:03:13.623 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_metadata PASSED [ 51%] 2026-02-21T00:03:14.263 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_watchers_list PASSED [ 52%] 2026-02-21T00:03:15.101 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_config_list PASSED [ 52%] 2026-02-21T00:03:15.943 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_image_config_set_and_get_and_remove PASSED [ 53%] 2026-02-21T00:03:16.627 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_sparsify PASSED [ 53%] 2026-02-21T00:03:25.917 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_encryption_luks1 PASSED [ 54%] 2026-02-21T00:03:38.511 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImage::test_encryption_luks2 PASSED [ 54%] 2026-02-21T00:03:39.194 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImageId::test_read PASSED [ 55%] 2026-02-21T00:03:40.002 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImageId::test_write PASSED [ 55%] 2026-02-21T00:03:40.776 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestImageId::test_resize PASSED [ 56%] 2026-02-21T00:03:45.097 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_with_params PASSED [ 56%] 2026-02-21T00:03:50.114 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_with_params2 PASSED [ 57%] 2026-02-21T00:03:52.911 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_with_params3 SKIPPED [ 57%] 2026-02-21T00:03:59.126 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_stripe_unit_and_count PASSED [ 58%] 2026-02-21T00:04:03.948 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_clone_format PASSED [ 58%] 2026-02-21T00:04:07.974 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_unprotected PASSED [ 59%] 2026-02-21T00:04:15.003 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_unprotect_with_children PASSED [ 59%] 2026-02-21T00:04:22.981 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_clone_by_snap_id PASSED [ 60%] 2026-02-21T00:04:26.005 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_stat PASSED [ 60%] 2026-02-21T00:04:29.021 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_resize_stat PASSED [ 61%] 2026-02-21T00:04:32.004 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_resize_io PASSED [ 61%] 2026-02-21T00:04:35.033 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_read PASSED [ 62%] 2026-02-21T00:04:38.032 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_write PASSED [ 62%] 2026-02-21T00:05:08.935 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_list_children PASSED [ 63%] 2026-02-21T00:05:13.077 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_errors PASSED [ 63%] 2026-02-21T00:05:19.104 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_basic PASSED [ 64%] 2026-02-21T00:05:25.129 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_smaller_order PASSED [ 64%] 2026-02-21T00:05:31.156 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_larger_order PASSED [ 65%] 2026-02-21T00:05:35.147 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_drops_cache PASSED [ 65%] 2026-02-21T00:05:40.158 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_multi_level PASSED [ 66%] 2026-02-21T00:05:44.179 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_flatten_with_progress PASSED [ 66%] 2026-02-21T00:05:49.193 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_resize_flatten_multi_level PASSED [ 67%] 2026-02-21T00:05:54.011 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_trash_snapshot PASSED [ 67%] 2026-02-21T00:06:02.983 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_encryption_luks1 PASSED [ 68%] 2026-02-21T00:06:15.968 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestClone::test_encryption_luks2 PASSED [ 68%] 2026-02-21T00:06:16.835 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_ownership PASSED [ 69%] 2026-02-21T00:06:19.022 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_snapshot_leadership PASSED [ 69%] 2026-02-21T00:06:19.649 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_read_only_leadership PASSED [ 70%] 2026-02-21T00:06:22.985 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_flatten PASSED [ 70%] 2026-02-21T00:06:24.028 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_resize PASSED [ 71%] 2026-02-21T00:06:25.981 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_snap_create PASSED [ 71%] 2026-02-21T00:06:28.003 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_snap_rollback PASSED [ 72%] 2026-02-21T00:06:28.955 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_discard PASSED [ 72%] 2026-02-21T00:06:29.940 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_follower_write PASSED [ 73%] 2026-02-21T00:06:30.731 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_acquire_release_lock PASSED [ 73%] 2026-02-21T00:06:32.009 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestExclusiveLock::test_break_lock PASSED [ 74%] 2026-02-21T00:06:32.743 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_uuid PASSED [ 75%] 2026-02-21T00:06:33.469 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_site_name PASSED [ 75%] 2026-02-21T00:06:34.888 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_remote_namespace PASSED [ 76%] 2026-02-21T00:06:35.714 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_peer_bootstrap PASSED [ 76%] 2026-02-21T00:06:36.534 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_peer PASSED [ 77%] 2026-02-21T00:06:37.260 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_image SKIPPED [ 77%] 2026-02-21T00:06:37.972 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_image_status SKIPPED [ 78%] 2026-02-21T00:06:46.320 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_mirror_image_create_snapshot PASSED [ 78%] 2026-02-21T00:06:50.458 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMirroring::test_aio_mirror_image_create_snapshot PASSED [ 79%] 2026-02-21T00:06:51.020 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_move PASSED [ 79%] 2026-02-21T00:06:52.079 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_purge PASSED [ 80%] 2026-02-21T00:06:52.645 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_remove_denied PASSED [ 80%] 2026-02-21T00:06:53.188 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_remove PASSED [ 81%] 2026-02-21T00:06:53.741 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_remove_with_progress PASSED [ 81%] 2026-02-21T00:06:54.277 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_get PASSED [ 82%] 2026-02-21T00:06:55.381 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_list PASSED [ 82%] 2026-02-21T00:06:56.297 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestTrash::test_restore PASSED [ 83%] 2026-02-21T00:06:56.354 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_create_group PASSED [ 83%] 2026-02-21T00:06:56.442 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_rename_group PASSED [ 84%] 2026-02-21T00:06:56.445 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list_groups_empty PASSED [ 84%] 2026-02-21T00:06:56.505 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list_groups PASSED [ 85%] 2026-02-21T00:06:56.566 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::test_list_groups_after_removed PASSED [ 85%] 2026-02-21T00:06:57.355 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_add PASSED [ 86%] 2026-02-21T00:06:58.082 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_list_empty PASSED [ 86%] 2026-02-21T00:06:58.881 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_list PASSED [ 87%] 2026-02-21T00:06:59.801 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_list_move_to_trash PASSED [ 87%] 2026-02-21T00:07:01.342 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_list_remove PASSED [ 88%] 2026-02-21T00:07:02.019 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_get_id PASSED [ 88%] 2026-02-21T00:07:16.329 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_many_images PASSED [ 89%] 2026-02-21T00:07:17.274 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_image_remove PASSED [ 89%] 2026-02-21T00:07:20.668 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_get_info PASSED [ 90%] 2026-02-21T00:07:21.445 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_get_info_no_member_images PASSED [ 90%] 2026-02-21T00:07:22.212 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap PASSED [ 91%] 2026-02-21T00:07:23.115 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_flags PASSED [ 91%] 2026-02-21T00:08:04.801 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_list_many PASSED [ 92%] 2026-02-21T00:08:07.187 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_namespace PASSED [ 92%] 2026-02-21T00:08:08.022 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_rename PASSED [ 93%] 2026-02-21T00:08:11.428 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_clone PASSED [ 93%] 2026-02-21T00:08:15.029 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_clone_flatten PASSED [ 94%] 2026-02-21T00:08:17.311 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_diff_iterate PASSED [ 94%] 2026-02-21T00:08:30.342 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestGroups::test_group_snap_rollback PASSED [ 95%] 2026-02-21T00:08:32.742 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration PASSED [ 95%] 2026-02-21T00:08:37.970 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_clone_v1_to_v1 PASSED [ 96%] 2026-02-21T00:08:42.996 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_clone_v1_to_v2 PASSED [ 96%] 2026-02-21T00:08:47.965 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_clone_v2_to_v1 PASSED [ 97%] 2026-02-21T00:08:52.999 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_clone_v2_to_v2 PASSED [ 97%] 2026-02-21T00:08:58.512 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_import PASSED [ 98%] 2026-02-21T00:09:00.440 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migration_with_progress PASSED [ 98%] 2026-02-21T00:09:02.105 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migrate_abort PASSED [ 99%] 2026-02-21T00:09:04.405 INFO:tasks.workunit.client.0.trial057.stdout:../../../clone.client.0/src/test/pybind/test_rbd.py::TestMigration::test_migrate_abort_with_progress PASSED [100%] 2026-02-21T00:09:04.405 INFO:tasks.workunit.client.0.trial057.stdout: 2026-02-21T00:09:04.405 INFO:tasks.workunit.client.0.trial057.stdout:=============================== warnings summary =============================== 2026-02-21T00:09:04.405 INFO:tasks.workunit.client.0.trial057.stdout:test_rbd.py::test_remove_with_progress_except 2026-02-21T00:09:04.405 INFO:tasks.workunit.client.0.trial057.stdout: /usr/lib/python3.9/site-packages/_pytest/unraisableexception.py:78: PytestUnraisableExceptionWarning: Exception ignored in: 'rbd.progress_callback' 2026-02-21T00:09:04.405 INFO:tasks.workunit.client.0.trial057.stdout: 2026-02-21T00:09:04.406 INFO:tasks.workunit.client.0.trial057.stdout: Traceback (most recent call last): 2026-02-21T00:09:04.406 INFO:tasks.workunit.client.0.trial057.stdout: File "/home/ubuntu/cephtest/clone.client.0/src/test/pybind/test_rbd.py", line 430, in progress_cb 2026-02-21T00:09:04.406 INFO:tasks.workunit.client.0.trial057.stdout: raise Exception() 2026-02-21T00:09:04.406 INFO:tasks.workunit.client.0.trial057.stdout: Exception 2026-02-21T00:09:04.406 INFO:tasks.workunit.client.0.trial057.stdout: 2026-02-21T00:09:04.406 INFO:tasks.workunit.client.0.trial057.stdout: warnings.warn(pytest.PytestUnraisableExceptionWarning(msg)) 2026-02-21T00:09:04.406 INFO:tasks.workunit.client.0.trial057.stdout: 2026-02-21T00:09:04.406 INFO:tasks.workunit.client.0.trial057.stdout:-- Docs: https://docs.pytest.org/en/stable/warnings.html 2026-02-21T00:09:04.406 INFO:tasks.workunit.client.0.trial057.stdout:============ 188 passed, 8 skipped, 1 warning in 1075.61s (0:17:55) ============ 2026-02-21T00:09:04.449 INFO:tasks.workunit.client.0.trial057.stderr:+ exit 0 2026-02-21T00:09:04.450 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-21T00:09:04.451 DEBUG:teuthology.orchestra.run.trial057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-21T00:09:04.521 INFO:tasks.workunit:Stopping ['rbd/test_librbd_python.sh'] on client.0... 2026-02-21T00:09:04.522 DEBUG:teuthology.orchestra.run.trial057:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-21T00:09:04.893 DEBUG:teuthology.parallel:result is None 2026-02-21T00:09:04.894 DEBUG:teuthology.orchestra.run.trial057:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-21T00:09:04.922 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-21T00:09:04.922 DEBUG:teuthology.orchestra.run.trial057:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-21T00:09:04.978 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-21T00:09:04.979 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-21T00:09:04.993 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-21T00:09:04.993 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T00:09:05.096 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:05.099+0000 7f93bc97e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:05.098 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:05.103+0000 7f93bc97e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:05.210 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-21T00:09:05.211 INFO:teuthology.orchestra.run.trial057.stderr:dumped all 2026-02-21T00:09:05.219 INFO:teuthology.orchestra.run.trial057.stdout:{"pg_ready":true,"pg_map":{"version":1296,"stamp":"2026-02-21T00:09:04.405041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":64,"num_write":113,"num_write_kb":1762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62940952,"kb_used_data":62940952,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94345400,"statfs":{"total":161061224448,"available":96609689600,"internally_reserved":0,"allocated":64451534848,"data_stored":64451534848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-86,"num_objects":-12,"num_object_clones":0,"num_object_copies":-24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":-390288,"num_read_kb":-49993,"num_write":-13172,"num_write_kb":-80776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.007426"},"pg_stats":[{"pgid":"1.0","version":"432'80","reported_seq":2308,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.376806+0000","last_change":"2026-02-20T23:50:30.669934+0000","last_active":"2026-02-21T00:08:57.376806+0000","last_peered":"2026-02-21T00:08:57.376806+0000","last_clean":"2026-02-21T00:08:57.376806+0000","last_became_active":"2026-02-20T23:50:30.660319+0000","last_became_peered":"2026-02-20T23:50:30.660319+0000","last_unstale":"2026-02-21T00:08:57.376806+0000","last_undegraded":"2026-02-21T00:08:57.376806+0000","last_fullsized":"2026-02-21T00:08:57.376806+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:29.624739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:29.624739+0000","last_clean_scrub_stamp":"2026-02-20T23:50:29.624739+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":64,"num_write":111,"num_write_kb":1760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":2200,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.375439+0000","last_change":"2026-02-20T23:50:36.670298+0000","last_active":"2026-02-21T00:08:57.375439+0000","last_peered":"2026-02-21T00:08:57.375439+0000","last_clean":"2026-02-21T00:08:57.375439+0000","last_became_active":"2026-02-20T23:50:34.682318+0000","last_became_peered":"2026-02-20T23:50:34.682318+0000","last_unstale":"2026-02-21T00:08:57.375439+0000","last_undegraded":"2026-02-21T00:08:57.375439+0000","last_fullsized":"2026-02-21T00:08:57.375439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.373684+0000","last_change":"2026-02-20T23:50:36.664790+0000","last_active":"2026-02-21T00:08:57.373684+0000","last_peered":"2026-02-21T00:08:57.373684+0000","last_clean":"2026-02-21T00:08:57.373684+0000","last_became_active":"2026-02-20T23:50:34.699338+0000","last_became_peered":"2026-02-20T23:50:34.699338+0000","last_unstale":"2026-02-21T00:08:57.373684+0000","last_undegraded":"2026-02-21T00:08:57.373684+0000","last_fullsized":"2026-02-21T00:08:57.373684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.376883+0000","last_change":"2026-02-20T23:50:36.666714+0000","last_active":"2026-02-21T00:08:57.376883+0000","last_peered":"2026-02-21T00:08:57.376883+0000","last_clean":"2026-02-21T00:08:57.376883+0000","last_became_active":"2026-02-20T23:50:34.690728+0000","last_became_peered":"2026-02-20T23:50:34.690728+0000","last_unstale":"2026-02-21T00:08:57.376883+0000","last_undegraded":"2026-02-21T00:08:57.376883+0000","last_fullsized":"2026-02-21T00:08:57.376883+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":2201,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.374935+0000","last_change":"2026-02-20T23:50:36.666311+0000","last_active":"2026-02-21T00:08:57.374935+0000","last_peered":"2026-02-21T00:08:57.374935+0000","last_clean":"2026-02-21T00:08:57.374935+0000","last_became_active":"2026-02-20T23:50:34.689002+0000","last_became_peered":"2026-02-20T23:50:34.689002+0000","last_unstale":"2026-02-21T00:08:57.374935+0000","last_undegraded":"2026-02-21T00:08:57.374935+0000","last_fullsized":"2026-02-21T00:08:57.374935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.376839+0000","last_change":"2026-02-20T23:50:36.667973+0000","last_active":"2026-02-21T00:08:57.376839+0000","last_peered":"2026-02-21T00:08:57.376839+0000","last_clean":"2026-02-21T00:08:57.376839+0000","last_became_active":"2026-02-20T23:50:34.693081+0000","last_became_peered":"2026-02-20T23:50:34.693081+0000","last_unstale":"2026-02-21T00:08:57.376839+0000","last_undegraded":"2026-02-21T00:08:57.376839+0000","last_fullsized":"2026-02-21T00:08:57.376839+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.380450+0000","last_change":"2026-02-20T23:50:36.670710+0000","last_active":"2026-02-21T00:08:57.380450+0000","last_peered":"2026-02-21T00:08:57.380450+0000","last_clean":"2026-02-21T00:08:57.380450+0000","last_became_active":"2026-02-20T23:50:34.695736+0000","last_became_peered":"2026-02-20T23:50:34.695736+0000","last_unstale":"2026-02-21T00:08:57.380450+0000","last_undegraded":"2026-02-21T00:08:57.380450+0000","last_fullsized":"2026-02-21T00:08:57.380450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.379092+0000","last_change":"2026-02-20T23:50:36.668336+0000","last_active":"2026-02-21T00:08:57.379092+0000","last_peered":"2026-02-21T00:08:57.379092+0000","last_clean":"2026-02-21T00:08:57.379092+0000","last_became_active":"2026-02-20T23:50:34.689053+0000","last_became_peered":"2026-02-20T23:50:34.689053+0000","last_unstale":"2026-02-21T00:08:57.379092+0000","last_undegraded":"2026-02-21T00:08:57.379092+0000","last_fullsized":"2026-02-21T00:08:57.379092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.373928+0000","last_change":"2026-02-20T23:50:36.667747+0000","last_active":"2026-02-21T00:08:57.373928+0000","last_peered":"2026-02-21T00:08:57.373928+0000","last_clean":"2026-02-21T00:08:57.373928+0000","last_became_active":"2026-02-20T23:50:34.683215+0000","last_became_peered":"2026-02-20T23:50:34.683215+0000","last_unstale":"2026-02-21T00:08:57.373928+0000","last_undegraded":"2026-02-21T00:08:57.373928+0000","last_fullsized":"2026-02-21T00:08:57.373928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96609902592,"internally_reserved":0,"allocated":64451321856,"data_stored":64451321856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":64,"num_write":111,"num_write_kb":1760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32201609216,"internally_reserved":0,"allocated":21485465600,"data_stored":21485465600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":9,"seq":38654706778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20981812,"kb_used_data":20981812,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31446972,"statfs":{"total":53687074816,"available":32201699328,"internally_reserved":0,"allocated":21485375488,"data_stored":21485375488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949674074,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20980728,"kb_used_data":20980728,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31448056,"statfs":{"total":53687074816,"available":32202809344,"internally_reserved":0,"allocated":21484265472,"data_stored":21484265472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949674074,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20978412,"kb_used_data":20978412,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31450372,"statfs":{"total":53687074816,"available":32205180928,"internally_reserved":0,"allocated":21481893888,"data_stored":21481893888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32201609216,"internally_reserved":0,"allocated":21485465600,"data_stored":21485465600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32202924032,"internally_reserved":0,"allocated":21484150784,"data_stored":21484150784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32201609216,"internally_reserved":0,"allocated":21485465600,"data_stored":21485465600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32205369344,"internally_reserved":0,"allocated":21481705472,"data_stored":21481705472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T00:09:05.221 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T00:09:05.298 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:05.301+0000 7f28e070b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:05.300 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:05.305+0000 7f28e070b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:05.410 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-21T00:09:05.411 INFO:teuthology.orchestra.run.trial057.stderr:dumped all 2026-02-21T00:09:05.420 INFO:teuthology.orchestra.run.trial057.stdout:{"pg_ready":true,"pg_map":{"version":1296,"stamp":"2026-02-21T00:09:04.405041+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":64,"num_write":113,"num_write_kb":1762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62940952,"kb_used_data":62940952,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94345400,"statfs":{"total":161061224448,"available":96609689600,"internally_reserved":0,"allocated":64451534848,"data_stored":64451534848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-86,"num_objects":-12,"num_object_clones":0,"num_object_copies":-24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":-390288,"num_read_kb":-49993,"num_write":-13172,"num_write_kb":-80776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.007426"},"pg_stats":[{"pgid":"1.0","version":"432'80","reported_seq":2308,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.376806+0000","last_change":"2026-02-20T23:50:30.669934+0000","last_active":"2026-02-21T00:08:57.376806+0000","last_peered":"2026-02-21T00:08:57.376806+0000","last_clean":"2026-02-21T00:08:57.376806+0000","last_became_active":"2026-02-20T23:50:30.660319+0000","last_became_peered":"2026-02-20T23:50:30.660319+0000","last_unstale":"2026-02-21T00:08:57.376806+0000","last_undegraded":"2026-02-21T00:08:57.376806+0000","last_fullsized":"2026-02-21T00:08:57.376806+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:29.624739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:29.624739+0000","last_clean_scrub_stamp":"2026-02-20T23:50:29.624739+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":64,"num_write":111,"num_write_kb":1760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":2200,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.375439+0000","last_change":"2026-02-20T23:50:36.670298+0000","last_active":"2026-02-21T00:08:57.375439+0000","last_peered":"2026-02-21T00:08:57.375439+0000","last_clean":"2026-02-21T00:08:57.375439+0000","last_became_active":"2026-02-20T23:50:34.682318+0000","last_became_peered":"2026-02-20T23:50:34.682318+0000","last_unstale":"2026-02-21T00:08:57.375439+0000","last_undegraded":"2026-02-21T00:08:57.375439+0000","last_fullsized":"2026-02-21T00:08:57.375439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.373684+0000","last_change":"2026-02-20T23:50:36.664790+0000","last_active":"2026-02-21T00:08:57.373684+0000","last_peered":"2026-02-21T00:08:57.373684+0000","last_clean":"2026-02-21T00:08:57.373684+0000","last_became_active":"2026-02-20T23:50:34.699338+0000","last_became_peered":"2026-02-20T23:50:34.699338+0000","last_unstale":"2026-02-21T00:08:57.373684+0000","last_undegraded":"2026-02-21T00:08:57.373684+0000","last_fullsized":"2026-02-21T00:08:57.373684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.376883+0000","last_change":"2026-02-20T23:50:36.666714+0000","last_active":"2026-02-21T00:08:57.376883+0000","last_peered":"2026-02-21T00:08:57.376883+0000","last_clean":"2026-02-21T00:08:57.376883+0000","last_became_active":"2026-02-20T23:50:34.690728+0000","last_became_peered":"2026-02-20T23:50:34.690728+0000","last_unstale":"2026-02-21T00:08:57.376883+0000","last_undegraded":"2026-02-21T00:08:57.376883+0000","last_fullsized":"2026-02-21T00:08:57.376883+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":2201,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.374935+0000","last_change":"2026-02-20T23:50:36.666311+0000","last_active":"2026-02-21T00:08:57.374935+0000","last_peered":"2026-02-21T00:08:57.374935+0000","last_clean":"2026-02-21T00:08:57.374935+0000","last_became_active":"2026-02-20T23:50:34.689002+0000","last_became_peered":"2026-02-20T23:50:34.689002+0000","last_unstale":"2026-02-21T00:08:57.374935+0000","last_undegraded":"2026-02-21T00:08:57.374935+0000","last_fullsized":"2026-02-21T00:08:57.374935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.376839+0000","last_change":"2026-02-20T23:50:36.667973+0000","last_active":"2026-02-21T00:08:57.376839+0000","last_peered":"2026-02-21T00:08:57.376839+0000","last_clean":"2026-02-21T00:08:57.376839+0000","last_became_active":"2026-02-20T23:50:34.693081+0000","last_became_peered":"2026-02-20T23:50:34.693081+0000","last_unstale":"2026-02-21T00:08:57.376839+0000","last_undegraded":"2026-02-21T00:08:57.376839+0000","last_fullsized":"2026-02-21T00:08:57.376839+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.380450+0000","last_change":"2026-02-20T23:50:36.670710+0000","last_active":"2026-02-21T00:08:57.380450+0000","last_peered":"2026-02-21T00:08:57.380450+0000","last_clean":"2026-02-21T00:08:57.380450+0000","last_became_active":"2026-02-20T23:50:34.695736+0000","last_became_peered":"2026-02-20T23:50:34.695736+0000","last_unstale":"2026-02-21T00:08:57.380450+0000","last_undegraded":"2026-02-21T00:08:57.380450+0000","last_fullsized":"2026-02-21T00:08:57.380450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.379092+0000","last_change":"2026-02-20T23:50:36.668336+0000","last_active":"2026-02-21T00:08:57.379092+0000","last_peered":"2026-02-21T00:08:57.379092+0000","last_clean":"2026-02-21T00:08:57.379092+0000","last_became_active":"2026-02-20T23:50:34.689053+0000","last_became_peered":"2026-02-20T23:50:34.689053+0000","last_unstale":"2026-02-21T00:08:57.379092+0000","last_undegraded":"2026-02-21T00:08:57.379092+0000","last_fullsized":"2026-02-21T00:08:57.379092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.373928+0000","last_change":"2026-02-20T23:50:36.667747+0000","last_active":"2026-02-21T00:08:57.373928+0000","last_peered":"2026-02-21T00:08:57.373928+0000","last_clean":"2026-02-21T00:08:57.373928+0000","last_became_active":"2026-02-20T23:50:34.683215+0000","last_became_peered":"2026-02-20T23:50:34.683215+0000","last_unstale":"2026-02-21T00:08:57.373928+0000","last_undegraded":"2026-02-21T00:08:57.373928+0000","last_fullsized":"2026-02-21T00:08:57.373928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96609902592,"internally_reserved":0,"allocated":64451321856,"data_stored":64451321856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":64,"num_write":111,"num_write_kb":1760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32201609216,"internally_reserved":0,"allocated":21485465600,"data_stored":21485465600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":9,"seq":38654706778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20981812,"kb_used_data":20981812,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31446972,"statfs":{"total":53687074816,"available":32201699328,"internally_reserved":0,"allocated":21485375488,"data_stored":21485375488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949674074,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20980728,"kb_used_data":20980728,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31448056,"statfs":{"total":53687074816,"available":32202809344,"internally_reserved":0,"allocated":21484265472,"data_stored":21484265472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949674074,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20978412,"kb_used_data":20978412,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31450372,"statfs":{"total":53687074816,"available":32205180928,"internally_reserved":0,"allocated":21481893888,"data_stored":21481893888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32201609216,"internally_reserved":0,"allocated":21485465600,"data_stored":21485465600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32202924032,"internally_reserved":0,"allocated":21484150784,"data_stored":21484150784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32201609216,"internally_reserved":0,"allocated":21485465600,"data_stored":21485465600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32205369344,"internally_reserved":0,"allocated":21481705472,"data_stored":21481705472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T00:09:05.421 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-21T00:09:05.421 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T00:09:05.499 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:05.502+0000 7fdc1e4c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:05.501 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:05.506+0000 7fdc1e4c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:05.613 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-21T00:09:05.614 INFO:teuthology.orchestra.run.trial057.stderr:dumped all 2026-02-21T00:09:05.624 INFO:teuthology.orchestra.run.trial057.stdout:{"pg_ready":true,"pg_map":{"version":1297,"stamp":"2026-02-21T00:09:05.619170+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":64,"num_write":113,"num_write_kb":1762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62940952,"kb_used_data":62940952,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94345400,"statfs":{"total":161061224448,"available":96609689600,"internally_reserved":0,"allocated":64451534848,"data_stored":64451534848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-86,"num_objects":-12,"num_object_clones":0,"num_object_copies":-24,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":-390288,"num_read_kb":-49993,"num_write":-13172,"num_write_kb":-80776,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-9,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.257671"},"pg_stats":[{"pgid":"1.0","version":"432'80","reported_seq":2308,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.376806+0000","last_change":"2026-02-20T23:50:30.669934+0000","last_active":"2026-02-21T00:08:57.376806+0000","last_peered":"2026-02-21T00:08:57.376806+0000","last_clean":"2026-02-21T00:08:57.376806+0000","last_became_active":"2026-02-20T23:50:30.660319+0000","last_became_peered":"2026-02-20T23:50:30.660319+0000","last_unstale":"2026-02-21T00:08:57.376806+0000","last_undegraded":"2026-02-21T00:08:57.376806+0000","last_fullsized":"2026-02-21T00:08:57.376806+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:29.624739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:29.624739+0000","last_clean_scrub_stamp":"2026-02-20T23:50:29.624739+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":64,"num_write":111,"num_write_kb":1760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":2200,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.375439+0000","last_change":"2026-02-20T23:50:36.670298+0000","last_active":"2026-02-21T00:08:57.375439+0000","last_peered":"2026-02-21T00:08:57.375439+0000","last_clean":"2026-02-21T00:08:57.375439+0000","last_became_active":"2026-02-20T23:50:34.682318+0000","last_became_peered":"2026-02-20T23:50:34.682318+0000","last_unstale":"2026-02-21T00:08:57.375439+0000","last_undegraded":"2026-02-21T00:08:57.375439+0000","last_fullsized":"2026-02-21T00:08:57.375439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.373684+0000","last_change":"2026-02-20T23:50:36.664790+0000","last_active":"2026-02-21T00:08:57.373684+0000","last_peered":"2026-02-21T00:08:57.373684+0000","last_clean":"2026-02-21T00:08:57.373684+0000","last_became_active":"2026-02-20T23:50:34.699338+0000","last_became_peered":"2026-02-20T23:50:34.699338+0000","last_unstale":"2026-02-21T00:08:57.373684+0000","last_undegraded":"2026-02-21T00:08:57.373684+0000","last_fullsized":"2026-02-21T00:08:57.373684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.376883+0000","last_change":"2026-02-20T23:50:36.666714+0000","last_active":"2026-02-21T00:08:57.376883+0000","last_peered":"2026-02-21T00:08:57.376883+0000","last_clean":"2026-02-21T00:08:57.376883+0000","last_became_active":"2026-02-20T23:50:34.690728+0000","last_became_peered":"2026-02-20T23:50:34.690728+0000","last_unstale":"2026-02-21T00:08:57.376883+0000","last_undegraded":"2026-02-21T00:08:57.376883+0000","last_fullsized":"2026-02-21T00:08:57.376883+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":2201,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.374935+0000","last_change":"2026-02-20T23:50:36.666311+0000","last_active":"2026-02-21T00:08:57.374935+0000","last_peered":"2026-02-21T00:08:57.374935+0000","last_clean":"2026-02-21T00:08:57.374935+0000","last_became_active":"2026-02-20T23:50:34.689002+0000","last_became_peered":"2026-02-20T23:50:34.689002+0000","last_unstale":"2026-02-21T00:08:57.374935+0000","last_undegraded":"2026-02-21T00:08:57.374935+0000","last_fullsized":"2026-02-21T00:08:57.374935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.376839+0000","last_change":"2026-02-20T23:50:36.667973+0000","last_active":"2026-02-21T00:08:57.376839+0000","last_peered":"2026-02-21T00:08:57.376839+0000","last_clean":"2026-02-21T00:08:57.376839+0000","last_became_active":"2026-02-20T23:50:34.693081+0000","last_became_peered":"2026-02-20T23:50:34.693081+0000","last_unstale":"2026-02-21T00:08:57.376839+0000","last_undegraded":"2026-02-21T00:08:57.376839+0000","last_fullsized":"2026-02-21T00:08:57.376839+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.380450+0000","last_change":"2026-02-20T23:50:36.670710+0000","last_active":"2026-02-21T00:08:57.380450+0000","last_peered":"2026-02-21T00:08:57.380450+0000","last_clean":"2026-02-21T00:08:57.380450+0000","last_became_active":"2026-02-20T23:50:34.695736+0000","last_became_peered":"2026-02-20T23:50:34.695736+0000","last_unstale":"2026-02-21T00:08:57.380450+0000","last_undegraded":"2026-02-21T00:08:57.380450+0000","last_fullsized":"2026-02-21T00:08:57.380450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.379092+0000","last_change":"2026-02-20T23:50:36.668336+0000","last_active":"2026-02-21T00:08:57.379092+0000","last_peered":"2026-02-21T00:08:57.379092+0000","last_clean":"2026-02-21T00:08:57.379092+0000","last_became_active":"2026-02-20T23:50:34.689053+0000","last_became_peered":"2026-02-20T23:50:34.689053+0000","last_unstale":"2026-02-21T00:08:57.379092+0000","last_undegraded":"2026-02-21T00:08:57.379092+0000","last_fullsized":"2026-02-21T00:08:57.379092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.373928+0000","last_change":"2026-02-20T23:50:36.667747+0000","last_active":"2026-02-21T00:08:57.373928+0000","last_peered":"2026-02-21T00:08:57.373928+0000","last_clean":"2026-02-21T00:08:57.373928+0000","last_became_active":"2026-02-20T23:50:34.683215+0000","last_became_peered":"2026-02-20T23:50:34.683215+0000","last_unstale":"2026-02-21T00:08:57.373928+0000","last_undegraded":"2026-02-21T00:08:57.373928+0000","last_fullsized":"2026-02-21T00:08:57.373928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96609902592,"internally_reserved":0,"allocated":64451321856,"data_stored":64451321856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":64,"num_write":111,"num_write_kb":1760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32201609216,"internally_reserved":0,"allocated":21485465600,"data_stored":21485465600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":9,"seq":38654706778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20981812,"kb_used_data":20981812,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31446972,"statfs":{"total":53687074816,"available":32201699328,"internally_reserved":0,"allocated":21485375488,"data_stored":21485375488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949674074,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20980728,"kb_used_data":20980728,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31448056,"statfs":{"total":53687074816,"available":32202809344,"internally_reserved":0,"allocated":21484265472,"data_stored":21484265472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949674074,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20978412,"kb_used_data":20978412,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31450372,"statfs":{"total":53687074816,"available":32205180928,"internally_reserved":0,"allocated":21481893888,"data_stored":21481893888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32201609216,"internally_reserved":0,"allocated":21485465600,"data_stored":21485465600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32202924032,"internally_reserved":0,"allocated":21484150784,"data_stored":21484150784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32201609216,"internally_reserved":0,"allocated":21485465600,"data_stored":21485465600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32205369344,"internally_reserved":0,"allocated":21481705472,"data_stored":21481705472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T00:09:05.625 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T00:09:05.704 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:05.707+0000 7fb986b6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:05.706 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:05.712+0000 7fb986b6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:05.818 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-21T00:09:05.818 INFO:teuthology.orchestra.run.trial057.stdout:{"epoch":746,"fsid":"797a1d6f-8149-4ca0-b8e3-a68b969d2bab","created":"2026-02-20T23:50:17.182210+0000","modified":"2026-02-21T00:09:04.401607+0000","last_up_change":"2026-02-20T23:50:28.619857+0000","last_in_change":"2026-02-20T23:50:18.944022+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":4,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T23:50:29.465865+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-20T23:50:32.879033+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"90bb8d89-1532-46ee-93ba-f158f3b5246e","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":600,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6807","nonce":1825992705}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6806","nonce":1825992705}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6817","nonce":1825992705}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6818","nonce":1825992705}]},"public_addr":"10.20.193.57:6807/1825992705","cluster_addr":"10.20.193.57:6806/1825992705","heartbeat_back_addr":"10.20.193.57:6817/1825992705","heartbeat_front_addr":"10.20.193.57:6818/1825992705","state":["exists","up"]},{"osd":1,"uuid":"6b55a8c6-e150-426a-b559-97bda041f886","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":600,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6803","nonce":1979508686}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6802","nonce":1979508686}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6810","nonce":1979508686}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6809","nonce":1979508686}]},"public_addr":"10.20.193.57:6803/1979508686","cluster_addr":"10.20.193.57:6802/1979508686","heartbeat_back_addr":"10.20.193.57:6810/1979508686","heartbeat_front_addr":"10.20.193.57:6809/1979508686","state":["exists","up"]},{"osd":2,"uuid":"859c2643-1ef2-4990-80b2-9f98600684ce","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":600,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6804","nonce":1165223019}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6805","nonce":1165223019}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6814","nonce":1165223019}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.57:6815","nonce":1165223019}]},"public_addr":"10.20.193.57:6804/1165223019","cluster_addr":"10.20.193.57:6805/1165223019","heartbeat_back_addr":"10.20.193.57:6814/1165223019","heartbeat_front_addr":"10.20.193.57:6815/1165223019","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{"10.20.193.57:0/3699007558":"2026-02-21T01:06:31.014125+0000"},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]},{"pool":3,"snaps":[{"begin":2,"length":369}]},{"pool":4,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:09:06.829 INFO:tasks.ceph:Scrubbing osd.0 2026-02-21T00:09:06.829 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-21T00:09:06.908 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:06.909+0000 7f870edec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:06.910 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:06.916+0000 7f870edec640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:06.920 INFO:teuthology.orchestra.run.trial057.stdout:{ 2026-02-21T00:09:06.920 INFO:teuthology.orchestra.run.trial057.stdout: "success": "" 2026-02-21T00:09:06.920 INFO:teuthology.orchestra.run.trial057.stdout:} 2026-02-21T00:09:06.928 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-21T00:09:07.007 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:07.009+0000 7f47ab05a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:07.009 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:07.014+0000 7f47ab05a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:07.122 INFO:teuthology.orchestra.run.trial057.stderr:instructed osd(s) 0 to deep-scrub 2026-02-21T00:09:07.133 INFO:tasks.ceph:Scrubbing osd.1 2026-02-21T00:09:07.133 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-21T00:09:07.212 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:07.213+0000 7eff3345a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:07.214 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:07.220+0000 7eff3345a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:07.224 INFO:teuthology.orchestra.run.trial057.stdout:{ 2026-02-21T00:09:07.224 INFO:teuthology.orchestra.run.trial057.stdout: "success": "" 2026-02-21T00:09:07.224 INFO:teuthology.orchestra.run.trial057.stdout:} 2026-02-21T00:09:07.233 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-21T00:09:07.311 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:07.314+0000 7fcef20c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:07.313 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:07.318+0000 7fcef20c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:07.433 INFO:teuthology.orchestra.run.trial057.stderr:instructed osd(s) 1 to deep-scrub 2026-02-21T00:09:07.449 INFO:tasks.ceph:Scrubbing osd.2 2026-02-21T00:09:07.449 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-21T00:09:07.528 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:07.529+0000 7fc1b8b6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:07.531 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:07.536+0000 7fc1b8b6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:07.541 INFO:teuthology.orchestra.run.trial057.stdout:{ 2026-02-21T00:09:07.541 INFO:teuthology.orchestra.run.trial057.stdout: "success": "" 2026-02-21T00:09:07.541 INFO:teuthology.orchestra.run.trial057.stdout:} 2026-02-21T00:09:07.548 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-21T00:09:07.626 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:07.628+0000 7f2e5e88b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:07.628 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:07.633+0000 7f2e5e88b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:07.739 INFO:teuthology.orchestra.run.trial057.stderr:instructed osd(s) 2 to deep-scrub 2026-02-21T00:09:07.751 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T00:09:07.828 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:07.831+0000 7fe444ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:07.830 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:07.836+0000 7fe444ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:07.943 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-21T00:09:07.943 INFO:teuthology.orchestra.run.trial057.stderr:dumped all 2026-02-21T00:09:07.952 INFO:teuthology.orchestra.run.trial057.stdout:{"pg_ready":true,"pg_map":{"version":1299,"stamp":"2026-02-21T00:09:07.619440+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":64,"num_write":113,"num_write_kb":1762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62940952,"kb_used_data":62940952,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94345400,"statfs":{"total":161061224448,"available":96609689600,"internally_reserved":0,"allocated":64451534848,"data_stored":64451534848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":-89,"num_objects":-10,"num_object_clones":0,"num_object_copies":-20,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":-1,"num_read":-391146,"num_read_kb":-50189,"num_write":-13245,"num_write_kb":-80806,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":-8,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"8.001229"},"pg_stats":[{"pgid":"1.0","version":"432'80","reported_seq":2308,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.376806+0000","last_change":"2026-02-20T23:50:30.669934+0000","last_active":"2026-02-21T00:08:57.376806+0000","last_peered":"2026-02-21T00:08:57.376806+0000","last_clean":"2026-02-21T00:08:57.376806+0000","last_became_active":"2026-02-20T23:50:30.660319+0000","last_became_peered":"2026-02-20T23:50:30.660319+0000","last_unstale":"2026-02-21T00:08:57.376806+0000","last_undegraded":"2026-02-21T00:08:57.376806+0000","last_fullsized":"2026-02-21T00:08:57.376806+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:29.624739+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:29.624739+0000","last_clean_scrub_stamp":"2026-02-20T23:50:29.624739+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":64,"num_write":111,"num_write_kb":1760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":2200,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.375439+0000","last_change":"2026-02-20T23:50:36.670298+0000","last_active":"2026-02-21T00:08:57.375439+0000","last_peered":"2026-02-21T00:08:57.375439+0000","last_clean":"2026-02-21T00:08:57.375439+0000","last_became_active":"2026-02-20T23:50:34.682318+0000","last_became_peered":"2026-02-20T23:50:34.682318+0000","last_unstale":"2026-02-21T00:08:57.375439+0000","last_undegraded":"2026-02-21T00:08:57.375439+0000","last_fullsized":"2026-02-21T00:08:57.375439+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.373684+0000","last_change":"2026-02-20T23:50:36.664790+0000","last_active":"2026-02-21T00:08:57.373684+0000","last_peered":"2026-02-21T00:08:57.373684+0000","last_clean":"2026-02-21T00:08:57.373684+0000","last_became_active":"2026-02-20T23:50:34.699338+0000","last_became_peered":"2026-02-20T23:50:34.699338+0000","last_unstale":"2026-02-21T00:08:57.373684+0000","last_undegraded":"2026-02-21T00:08:57.373684+0000","last_fullsized":"2026-02-21T00:08:57.373684+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.376883+0000","last_change":"2026-02-20T23:50:36.666714+0000","last_active":"2026-02-21T00:08:57.376883+0000","last_peered":"2026-02-21T00:08:57.376883+0000","last_clean":"2026-02-21T00:08:57.376883+0000","last_became_active":"2026-02-20T23:50:34.690728+0000","last_became_peered":"2026-02-20T23:50:34.690728+0000","last_unstale":"2026-02-21T00:08:57.376883+0000","last_undegraded":"2026-02-21T00:08:57.376883+0000","last_fullsized":"2026-02-21T00:08:57.376883+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":2201,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.374935+0000","last_change":"2026-02-20T23:50:36.666311+0000","last_active":"2026-02-21T00:08:57.374935+0000","last_peered":"2026-02-21T00:08:57.374935+0000","last_clean":"2026-02-21T00:08:57.374935+0000","last_became_active":"2026-02-20T23:50:34.689002+0000","last_became_peered":"2026-02-20T23:50:34.689002+0000","last_unstale":"2026-02-21T00:08:57.374935+0000","last_undegraded":"2026-02-21T00:08:57.374935+0000","last_fullsized":"2026-02-21T00:08:57.374935+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.376839+0000","last_change":"2026-02-20T23:50:36.667973+0000","last_active":"2026-02-21T00:08:57.376839+0000","last_peered":"2026-02-21T00:08:57.376839+0000","last_clean":"2026-02-21T00:08:57.376839+0000","last_became_active":"2026-02-20T23:50:34.693081+0000","last_became_peered":"2026-02-20T23:50:34.693081+0000","last_unstale":"2026-02-21T00:08:57.376839+0000","last_undegraded":"2026-02-21T00:08:57.376839+0000","last_fullsized":"2026-02-21T00:08:57.376839+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.380450+0000","last_change":"2026-02-20T23:50:36.670710+0000","last_active":"2026-02-21T00:08:57.380450+0000","last_peered":"2026-02-21T00:08:57.380450+0000","last_clean":"2026-02-21T00:08:57.380450+0000","last_became_active":"2026-02-20T23:50:34.695736+0000","last_became_peered":"2026-02-20T23:50:34.695736+0000","last_unstale":"2026-02-21T00:08:57.380450+0000","last_undegraded":"2026-02-21T00:08:57.380450+0000","last_fullsized":"2026-02-21T00:08:57.380450+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.379092+0000","last_change":"2026-02-20T23:50:36.668336+0000","last_active":"2026-02-21T00:08:57.379092+0000","last_peered":"2026-02-21T00:08:57.379092+0000","last_clean":"2026-02-21T00:08:57.379092+0000","last_became_active":"2026-02-20T23:50:34.689053+0000","last_became_peered":"2026-02-20T23:50:34.689053+0000","last_unstale":"2026-02-21T00:08:57.379092+0000","last_undegraded":"2026-02-21T00:08:57.379092+0000","last_fullsized":"2026-02-21T00:08:57.379092+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":2199,"reported_epoch":745,"state":"active+clean","last_fresh":"2026-02-21T00:08:57.373928+0000","last_change":"2026-02-20T23:50:36.667747+0000","last_active":"2026-02-21T00:08:57.373928+0000","last_peered":"2026-02-21T00:08:57.373928+0000","last_clean":"2026-02-21T00:08:57.373928+0000","last_became_active":"2026-02-20T23:50:34.683215+0000","last_became_peered":"2026-02-20T23:50:34.683215+0000","last_unstale":"2026-02-21T00:08:57.373928+0000","last_undegraded":"2026-02-21T00:08:57.373928+0000","last_fullsized":"2026-02-21T00:08:57.373928+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T23:50:33.639017+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96609902592,"internally_reserved":0,"allocated":64451321856,"data_stored":64451321856,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":64,"num_write":111,"num_write_kb":1760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32201609216,"internally_reserved":0,"allocated":21485465600,"data_stored":21485465600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":9,"seq":38654706778,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20981812,"kb_used_data":20981812,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31446972,"statfs":{"total":53687074816,"available":32201699328,"internally_reserved":0,"allocated":21485375488,"data_stored":21485375488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949674074,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20980728,"kb_used_data":20980728,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31448056,"statfs":{"total":53687074816,"available":32202809344,"internally_reserved":0,"allocated":21484265472,"data_stored":21484265472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949674074,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20978412,"kb_used_data":20978412,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31450372,"statfs":{"total":53687074816,"available":32205180928,"internally_reserved":0,"allocated":21481893888,"data_stored":21481893888,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32201609216,"internally_reserved":0,"allocated":21485465600,"data_stored":21485465600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32202924032,"internally_reserved":0,"allocated":21484150784,"data_stored":21484150784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32201609216,"internally_reserved":0,"allocated":21485465600,"data_stored":21485465600,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32205369344,"internally_reserved":0,"allocated":21481705472,"data_stored":21481705472,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T00:09:07.958 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-20T23:50:29.624739+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=23, tm_min=50, tm_sec=29, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=0, tm_min=9, tm_sec=5, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T00:09:07.958 INFO:tasks.ceph:pgid 2.3 last_scrub_stamp 2026-02-20T23:50:33.639017+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=23, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=0, tm_min=9, tm_sec=5, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T00:09:07.958 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-20T23:50:33.639017+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=23, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=0, tm_min=9, tm_sec=5, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T00:09:07.958 INFO:tasks.ceph:pgid 2.1 last_scrub_stamp 2026-02-20T23:50:33.639017+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=23, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=0, tm_min=9, tm_sec=5, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T00:09:07.959 INFO:tasks.ceph:pgid 2.2 last_scrub_stamp 2026-02-20T23:50:33.639017+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=23, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=0, tm_min=9, tm_sec=5, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T00:09:07.959 INFO:tasks.ceph:pgid 2.4 last_scrub_stamp 2026-02-20T23:50:33.639017+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=23, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=0, tm_min=9, tm_sec=5, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T00:09:07.959 INFO:tasks.ceph:pgid 2.5 last_scrub_stamp 2026-02-20T23:50:33.639017+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=23, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=0, tm_min=9, tm_sec=5, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T00:09:07.959 INFO:tasks.ceph:pgid 2.6 last_scrub_stamp 2026-02-20T23:50:33.639017+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=23, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=0, tm_min=9, tm_sec=5, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T00:09:07.959 INFO:tasks.ceph:pgid 2.7 last_scrub_stamp 2026-02-20T23:50:33.639017+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=20, tm_hour=23, tm_min=50, tm_sec=33, tm_wday=4, tm_yday=51, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=21, tm_hour=0, tm_min=9, tm_sec=5, tm_wday=5, tm_yday=52, tm_isdst=0) 2026-02-21T00:09:07.959 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-21T00:09:27.961 DEBUG:teuthology.orchestra.run.trial057:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T00:09:28.042 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:28.046+0000 7ff14b61a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:28.045 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:28.051+0000 7ff14b61a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:28.155 INFO:teuthology.orchestra.run.trial057.stdout: 2026-02-21T00:09:28.156 INFO:teuthology.orchestra.run.trial057.stderr:dumped all 2026-02-21T00:09:28.164 INFO:teuthology.orchestra.run.trial057.stdout:{"pg_ready":true,"pg_map":{"version":1309,"stamp":"2026-02-21T00:09:27.622478+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":64,"num_write":113,"num_write_kb":1762,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":83,"ondisk_log_size":83,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62920484,"kb_used_data":62920484,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94365868,"statfs":{"total":161061224448,"available":96630648832,"internally_reserved":0,"allocated":64430575616,"data_stored":64430575616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.001804"},"pg_stats":[{"pgid":"1.0","version":"432'80","reported_seq":2316,"reported_epoch":747,"state":"active+clean","last_fresh":"2026-02-21T00:09:07.441335+0000","last_change":"2026-02-21T00:09:07.466712+0000","last_active":"2026-02-21T00:09:07.441335+0000","last_peered":"2026-02-21T00:09:07.441335+0000","last_clean":"2026-02-21T00:09:07.441335+0000","last_became_active":"2026-02-20T23:50:30.660319+0000","last_became_peered":"2026-02-20T23:50:30.660319+0000","last_unstale":"2026-02-21T00:09:07.441335+0000","last_undegraded":"2026-02-21T00:09:07.441335+0000","last_fullsized":"2026-02-21T00:09:07.441335+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"432'80","last_scrub_stamp":"2026-02-21T00:09:07.466640+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:09:07.466640+0000","last_clean_scrub_stamp":"2026-02-20T23:50:29.624739+0000","objects_scrubbed":0,"log_size":80,"log_dups_size":0,"ondisk_log_size":80,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":64,"num_write":111,"num_write_kb":1760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":2208,"reported_epoch":747,"state":"active+clean","last_fresh":"2026-02-21T00:09:07.441794+0000","last_change":"2026-02-21T00:09:07.456489+0000","last_active":"2026-02-21T00:09:07.441794+0000","last_peered":"2026-02-21T00:09:07.441794+0000","last_clean":"2026-02-21T00:09:07.441794+0000","last_became_active":"2026-02-20T23:50:34.682318+0000","last_became_peered":"2026-02-20T23:50:34.682318+0000","last_unstale":"2026-02-21T00:09:07.441794+0000","last_undegraded":"2026-02-21T00:09:07.441794+0000","last_fullsized":"2026-02-21T00:09:07.441794+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'1","last_scrub_stamp":"2026-02-21T00:09:07.456426+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:09:07.456426+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":2207,"reported_epoch":747,"state":"active+clean","last_fresh":"2026-02-21T00:09:07.747174+0000","last_change":"2026-02-21T00:09:07.756201+0000","last_active":"2026-02-21T00:09:07.747174+0000","last_peered":"2026-02-21T00:09:07.747174+0000","last_clean":"2026-02-21T00:09:07.747174+0000","last_became_active":"2026-02-20T23:50:34.699338+0000","last_became_peered":"2026-02-20T23:50:34.699338+0000","last_unstale":"2026-02-21T00:09:07.747174+0000","last_undegraded":"2026-02-21T00:09:07.747174+0000","last_fullsized":"2026-02-21T00:09:07.747174+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:09:07.756134+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:09:07.756134+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":2207,"reported_epoch":747,"state":"active+clean","last_fresh":"2026-02-21T00:09:07.748145+0000","last_change":"2026-02-21T00:09:07.756886+0000","last_active":"2026-02-21T00:09:07.748145+0000","last_peered":"2026-02-21T00:09:07.748145+0000","last_clean":"2026-02-21T00:09:07.748145+0000","last_became_active":"2026-02-20T23:50:34.690728+0000","last_became_peered":"2026-02-20T23:50:34.690728+0000","last_unstale":"2026-02-21T00:09:07.748145+0000","last_undegraded":"2026-02-21T00:09:07.748145+0000","last_fullsized":"2026-02-21T00:09:07.748145+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:09:07.756811+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:09:07.756811+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":2209,"reported_epoch":747,"state":"active+clean","last_fresh":"2026-02-21T00:09:07.374497+0000","last_change":"2026-02-21T00:09:07.139819+0000","last_active":"2026-02-21T00:09:07.374497+0000","last_peered":"2026-02-21T00:09:07.374497+0000","last_clean":"2026-02-21T00:09:07.374497+0000","last_became_active":"2026-02-20T23:50:34.689002+0000","last_became_peered":"2026-02-20T23:50:34.689002+0000","last_unstale":"2026-02-21T00:09:07.374497+0000","last_undegraded":"2026-02-21T00:09:07.374497+0000","last_fullsized":"2026-02-21T00:09:07.374497+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"17'2","last_scrub_stamp":"2026-02-21T00:09:07.139751+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:09:07.139751+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":2207,"reported_epoch":747,"state":"active+clean","last_fresh":"2026-02-21T00:09:07.442163+0000","last_change":"2026-02-21T00:09:07.451563+0000","last_active":"2026-02-21T00:09:07.442163+0000","last_peered":"2026-02-21T00:09:07.442163+0000","last_clean":"2026-02-21T00:09:07.442163+0000","last_became_active":"2026-02-20T23:50:34.693081+0000","last_became_peered":"2026-02-20T23:50:34.693081+0000","last_unstale":"2026-02-21T00:09:07.442163+0000","last_undegraded":"2026-02-21T00:09:07.442163+0000","last_fullsized":"2026-02-21T00:09:07.442163+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:09:07.451489+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:09:07.451489+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":2207,"reported_epoch":747,"state":"active+clean","last_fresh":"2026-02-21T00:09:07.442669+0000","last_change":"2026-02-21T00:09:07.454909+0000","last_active":"2026-02-21T00:09:07.442669+0000","last_peered":"2026-02-21T00:09:07.442669+0000","last_clean":"2026-02-21T00:09:07.442669+0000","last_became_active":"2026-02-20T23:50:34.695736+0000","last_became_peered":"2026-02-20T23:50:34.695736+0000","last_unstale":"2026-02-21T00:09:07.442669+0000","last_undegraded":"2026-02-21T00:09:07.442669+0000","last_fullsized":"2026-02-21T00:09:07.442669+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:09:07.454850+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:09:07.454850+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":2207,"reported_epoch":747,"state":"active+clean","last_fresh":"2026-02-21T00:09:07.443176+0000","last_change":"2026-02-21T00:09:07.456031+0000","last_active":"2026-02-21T00:09:07.443176+0000","last_peered":"2026-02-21T00:09:07.443176+0000","last_clean":"2026-02-21T00:09:07.443176+0000","last_became_active":"2026-02-20T23:50:34.689053+0000","last_became_peered":"2026-02-20T23:50:34.689053+0000","last_unstale":"2026-02-21T00:09:07.443176+0000","last_undegraded":"2026-02-21T00:09:07.443176+0000","last_fullsized":"2026-02-21T00:09:07.443176+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:09:07.455977+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:09:07.455977+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.7","version":"0'0","reported_seq":2207,"reported_epoch":747,"state":"active+clean","last_fresh":"2026-02-21T00:09:07.443044+0000","last_change":"2026-02-21T00:09:07.457012+0000","last_active":"2026-02-21T00:09:07.443044+0000","last_peered":"2026-02-21T00:09:07.443044+0000","last_clean":"2026-02-21T00:09:07.443044+0000","last_became_active":"2026-02-20T23:50:34.683215+0000","last_became_peered":"2026-02-20T23:50:34.683215+0000","last_unstale":"2026-02-21T00:09:07.443044+0000","last_undegraded":"2026-02-21T00:09:07.443044+0000","last_fullsized":"2026-02-21T00:09:07.443044+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:09:07.456959+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:09:07.456959+0000","last_clean_scrub_stamp":"2026-02-20T23:50:33.639017+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96630648832,"internally_reserved":0,"allocated":64430575616,"data_stored":64430575616,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":73,"num_read_kb":64,"num_write":111,"num_write_kb":1760,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32209883136,"internally_reserved":0,"allocated":21477191680,"data_stored":21477191680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":80,"ondisk_log_size":80,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":9,"seq":38654706798,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20973820,"kb_used_data":20973820,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31454964,"statfs":{"total":53687074816,"available":32209883136,"internally_reserved":0,"allocated":21477191680,"data_stored":21477191680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949674094,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20973724,"kb_used_data":20973724,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31455060,"statfs":{"total":53687074816,"available":32209981440,"internally_reserved":0,"allocated":21477093376,"data_stored":21477093376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949674094,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972940,"kb_used_data":20972940,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31455844,"statfs":{"total":53687074816,"available":32210784256,"internally_reserved":0,"allocated":21476290560,"data_stored":21476290560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32209883136,"internally_reserved":0,"allocated":21477191680,"data_stored":21477191680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32209981440,"internally_reserved":0,"allocated":21477093376,"data_stored":21477093376,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32209883136,"internally_reserved":0,"allocated":21477191680,"data_stored":21477191680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32210784256,"internally_reserved":0,"allocated":21476290560,"data_stored":21476290560,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T00:09:28.166 DEBUG:teuthology.orchestra.run.trial057:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-21T00:09:28.241 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:28.243+0000 7f4df6c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:28.243 INFO:teuthology.orchestra.run.trial057.stderr:2026-02-21T00:09:28.249+0000 7f4df6c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:28.370 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-21T00:09:28.370 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-21T00:09:28.371 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-21T00:09:28.371 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:09:28.440 INFO:tasks.ceph.osd.0:Stopped 2026-02-21T00:09:28.440 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-21T00:09:28.440 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:09:28.507 INFO:tasks.ceph.osd.1:Stopped 2026-02-21T00:09:28.508 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-21T00:09:28.508 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:09:28.577 INFO:tasks.ceph.osd.2:Stopped 2026-02-21T00:09:28.577 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-21T00:09:28.577 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-21T00:09:28.577 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:09:28.612 INFO:tasks.ceph.mgr.x:Stopped 2026-02-21T00:09:28.612 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-21T00:09:28.612 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-21T00:09:28.612 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:09:28.665 INFO:tasks.ceph.mon.a:Stopped 2026-02-21T00:09:28.665 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-21T00:09:28.666 DEBUG:teuthology.orchestra.run.trial057:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v '\(SLOW_OPS\)' | grep -E -v 'slow request' | head -n 1 2026-02-21T00:09:28.732 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial057.front.sepia.ceph.com 2026-02-21T00:09:28.732 DEBUG:teuthology.orchestra.run.trial057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-21T00:09:28.799 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial057.front.sepia.ceph.com 2026-02-21T00:09:28.799 DEBUG:teuthology.orchestra.run.trial057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-21T00:09:28.866 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial057.front.sepia.ceph.com 2026-02-21T00:09:28.866 DEBUG:teuthology.orchestra.run.trial057:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-21T00:09:28.940 INFO:tasks.ceph:Archiving mon data... 2026-02-21T00:09:28.941 DEBUG:teuthology.misc:Transferring archived files from trial057:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61870/data/mon.a.tgz 2026-02-21T00:09:28.942 DEBUG:teuthology.orchestra.run.trial057:> mktemp 2026-02-21T00:09:28.998 INFO:teuthology.orchestra.run.trial057.stdout:/tmp/tmp.IsbBtWynA1 2026-02-21T00:09:28.998 DEBUG:teuthology.orchestra.run.trial057:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.IsbBtWynA1 2026-02-21T00:09:29.093 DEBUG:teuthology.orchestra.run.trial057:> sudo chmod 0666 /tmp/tmp.IsbBtWynA1 2026-02-21T00:09:29.174 DEBUG:teuthology.orchestra.remote:trial057:/tmp/tmp.IsbBtWynA1 is 253KB 2026-02-21T00:09:29.234 DEBUG:teuthology.orchestra.run.trial057:> rm -fr /tmp/tmp.IsbBtWynA1 2026-02-21T00:09:29.251 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-21T00:09:29.251 DEBUG:teuthology.orchestra.run.trial057:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-21T00:09:29.368 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-21T00:09:29.369 INFO:tasks.ceph:Archiving crash dumps... 2026-02-21T00:09:29.370 DEBUG:teuthology.misc:Transferring archived files from trial057:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61870/remote/trial057/crash 2026-02-21T00:09:29.371 DEBUG:teuthology.orchestra.run.trial057:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-21T00:09:29.432 INFO:tasks.ceph:Compressing logs... 2026-02-21T00:09:29.432 DEBUG:teuthology.orchestra.run.trial057:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T00:09:29.454 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-21T00:09:29.454 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37374.log 2026-02-21T00:09:29.454 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-21T00:09:29.454 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37374.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36573.log 2026-02-21T00:09:29.454 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37374.log.gz 2026-02-21T00:09:29.455 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-21T00:09:29.455 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.36573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37026.log 2026-02-21T00:09:29.455 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36573.log.gz 2026-02-21T00:09:29.455 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36977.log 2026-02-21T00:09:29.455 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37026.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37373.log 2026-02-21T00:09:29.455 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37026.log.gz 2026-02-21T00:09:29.456 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.36977.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37227.log 2026-02-21T00:09:29.456 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36977.log.gz 2026-02-21T00:09:29.456 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.35981.log 2026-02-21T00:09:29.456 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37373.log.gz 2026-02-21T00:09:29.456 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37227.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36354.log 2026-02-21T00:09:29.456 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37227.log.gz 2026-02-21T00:09:29.456 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.35981.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37761.log 2026-02-21T00:09:29.457 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.35981.log.gz 2026-02-21T00:09:29.457 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.36354.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41749.log 2026-02-21T00:09:29.457 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36354.log.gz 2026-02-21T00:09:29.457 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36260.log 2026-02-21T00:09:29.457 INFO:teuthology.orchestra.run.trial057.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.37761.log.gz 2026-02-21T00:09:29.458 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.41749.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41504.log 2026-02-21T00:09:29.458 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41749.log.gz 2026-02-21T00:09:29.458 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.36260.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41357.log 2026-02-21T00:09:29.458 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36260.log.gz 2026-02-21T00:09:29.458 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.41504.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37641.log 2026-02-21T00:09:29.458 INFO:teuthology.orchestra.run.trial057.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41504.log.gz 2026-02-21T00:09:29.458 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.41357.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38099.log 2026-02-21T00:09:29.458 INFO:teuthology.orchestra.run.trial057.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41357.log.gz 2026-02-21T00:09:29.459 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37641.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41308.log 2026-02-21T00:09:29.459 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37641.log.gz 2026-02-21T00:09:29.459 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.38099.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36681.log 2026-02-21T00:09:29.459 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.41308.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36817.log 2026-02-21T00:09:29.459 INFO:teuthology.orchestra.run.trial057.stderr: 88.7% -- replaced with /var/log/ceph/ceph-client.admin.38099.log.gz 2026-02-21T00:09:29.459 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41308.log.gz 2026-02-21T00:09:29.459 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.36681.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37614.log 2026-02-21T00:09:29.460 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36681.log.gz 2026-02-21T00:09:29.460 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.36817.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36817.log.gz 2026-02-21T00:09:29.460 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41602.log 2026-02-21T00:09:29.460 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37614.log: gzip 49.6% -5 -- replaced with /var/log/ceph/ceph-client.admin.37614.log.gz --verbose 2026-02-21T00:09:29.460 INFO:teuthology.orchestra.run.trial057.stderr: -- /var/log/ceph/ceph-client.admin.37467.log 2026-02-21T00:09:29.460 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.41602.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41602.log.gz 2026-02-21T00:09:29.460 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37541.log 2026-02-21T00:09:29.460 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37467.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37225.log 2026-02-21T00:09:29.460 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37467.log.gz 2026-02-21T00:09:29.461 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37541.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37541.log.gz 2026-02-21T00:09:29.461 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36750.log 2026-02-21T00:09:29.461 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37225.log: gzip -5 50.4% --verbose -- -- replaced with /var/log/ceph/ceph-client.admin.37225.log.gz 2026-02-21T00:09:29.461 INFO:teuthology.orchestra.run.trial057.stderr: /var/log/ceph/ceph-client.admin.41553.log 2026-02-21T00:09:29.461 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.36750.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41406.log 2026-02-21T00:09:29.461 INFO:teuthology.orchestra.run.trial057.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.36750.log.gz 2026-02-21T00:09:29.461 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.41553.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41553.log.gz 2026-02-21T00:09:29.461 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.34675.log 2026-02-21T00:09:29.462 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.41406.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41406.log.gz 2026-02-21T00:09:29.462 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41651.log 2026-02-21T00:09:29.462 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph.tmp-client.admin.34675.log: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.34675.log.gz 2026-02-21T00:09:29.462 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41455.log 2026-02-21T00:09:29.462 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.41651.log: 50.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.41651.log.gz 2026-02-21T00:09:29.462 INFO:teuthology.orchestra.run.trial057.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.41847.log 2026-02-21T00:09:29.462 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.41455.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41455.log.gz 2026-02-21T00:09:29.462 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36887.log 2026-02-21T00:09:29.463 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.41847.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41847.log.gz 2026-02-21T00:09:29.463 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37075.log 2026-02-21T00:09:29.463 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.36887.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36887.log.gz 2026-02-21T00:09:29.463 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37226.log 2026-02-21T00:09:29.463 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37075.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37075.log.gz 2026-02-21T00:09:29.463 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37810.log 2026-02-21T00:09:29.463 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37226.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-21T00:09:29.463 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37226.log.gz 2026-02-21T00:09:29.464 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37810.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.37810.log.gz 2026-02-21T00:09:29.464 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-21T00:09:29.464 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37320.log 2026-02-21T00:09:29.464 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-osd.1.log: 86.9%gzip -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-21T00:09:29.464 INFO:teuthology.orchestra.run.trial057.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.37688.log 2026-02-21T00:09:29.464 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37320.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36307.log 2026-02-21T00:09:29.465 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37320.log.gz 2026-02-21T00:09:29.465 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37688.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37859.log 2026-02-21T00:09:29.465 INFO:teuthology.orchestra.run.trial057.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.37688.log.gz 2026-02-21T00:09:29.465 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.36307.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37124.log 2026-02-21T00:09:29.465 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.36307.log.gz 2026-02-21T00:09:29.466 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37859.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.37494.log 2026-02-21T00:09:29.466 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.37859.log.gz 2026-02-21T00:09:29.466 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37124.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.37124.log.gz 2026-02-21T00:09:29.466 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-21T00:09:29.466 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.37494.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41798.log 2026-02-21T00:09:29.466 INFO:teuthology.orchestra.run.trial057.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.37494.log.gz 2026-02-21T00:09:29.466 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41894.log 2026-02-21T00:09:29.466 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.41798.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41798.log.gz 2026-02-21T00:09:29.467 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-21T00:09:29.467 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.41894.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36912.log 2026-02-21T00:09:29.467 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41894.log.gz 2026-02-21T00:09:29.467 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36123.log 2026-02-21T00:09:29.467 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.36912.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41700.log 2026-02-21T00:09:29.467 INFO:teuthology.orchestra.run.trial057.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.36912.log.gz 2026-02-21T00:09:29.468 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.36123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.36076.log 2026-02-21T00:09:29.468 INFO:teuthology.orchestra.run.trial057.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36123.log.gz 2026-02-21T00:09:29.468 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.41700.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41700.log.gz 2026-02-21T00:09:29.468 INFO:teuthology.orchestra.run.trial057.stderr:/var/log/ceph/ceph-client.admin.36076.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.36076.log.gz 2026-02-21T00:09:29.468 INFO:teuthology.orchestra.run.trial057.stderr: 87.0% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-21T00:09:29.523 INFO:teuthology.orchestra.run.trial057.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-21T00:09:29.778 INFO:teuthology.orchestra.run.trial057.stderr: 89.8% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-21T00:09:35.753 INFO:teuthology.orchestra.run.trial057.stderr: 93.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-21T00:09:38.547 INFO:teuthology.orchestra.run.trial057.stderr: 93.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-21T00:09:45.784 INFO:teuthology.orchestra.run.trial057.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-21T00:09:45.786 INFO:teuthology.orchestra.run.trial057.stderr: 2026-02-21T00:09:45.786 INFO:teuthology.orchestra.run.trial057.stderr:real 0m16.339s 2026-02-21T00:09:45.786 INFO:teuthology.orchestra.run.trial057.stderr:user 0m30.243s 2026-02-21T00:09:45.786 INFO:teuthology.orchestra.run.trial057.stderr:sys 0m1.826s 2026-02-21T00:09:45.786 INFO:tasks.ceph:Archiving logs... 2026-02-21T00:09:45.787 DEBUG:teuthology.misc:Transferring archived files from trial057:/var/log/ceph to /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61870/remote/trial057/log 2026-02-21T00:09:45.788 DEBUG:teuthology.orchestra.run.trial057:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-21T00:09:48.106 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-21T00:09:48.175 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-21T00:09:48.185 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-21T00:09:48.186 DEBUG:teuthology.orchestra.run.trial057:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-21T00:09:48.212 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-21T00:09:48.223 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-21T00:09:48.223 DEBUG:teuthology.orchestra.run.trial057:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T00:09:48.267 INFO:teuthology.orchestra.run.trial057.stderr:bash: line 1: ntpq: command not found 2026-02-21T00:09:48.293 INFO:teuthology.orchestra.run.trial057.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-21T00:09:48.293 INFO:teuthology.orchestra.run.trial057.stdout:=============================================================================== 2026-02-21T00:09:48.293 INFO:teuthology.orchestra.run.trial057.stdout:^+ time.trtnw.net 2 6 377 25 +6736us[+6736us] +/- 34ms 2026-02-21T00:09:48.294 INFO:teuthology.orchestra.run.trial057.stdout:^+ wdc1.us.ntp.li 3 8 377 218 +7054us[+7054us] +/- 29ms 2026-02-21T00:09:48.294 INFO:teuthology.orchestra.run.trial057.stdout:^* h96-60-247-29.mdsnwi.bro> 2 8 377 219 +6476us[+6531us] +/- 26ms 2026-02-21T00:09:48.294 INFO:teuthology.orchestra.run.trial057.stdout:^+ t2.time.bf1.yahoo.com 2 6 377 22 -28ms[ -28ms] +/- 42ms 2026-02-21T00:09:48.294 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-21T00:09:48.305 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-21T00:09:48.305 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-21T00:09:48.319 DEBUG:teuthology.orchestra.run.trial057:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-21T00:09:48.386 DEBUG:teuthology.orchestra.run.trial057:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-21T00:09:48.412 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T00:09:48.413 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-21T00:09:48.428 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-21T00:09:48.442 INFO:teuthology.task.internal:Duration was 2000.354779 seconds 2026-02-21T00:09:48.443 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-21T00:09:48.456 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-21T00:09:48.457 DEBUG:teuthology.orchestra.run.trial057:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-21T00:09:48.494 INFO:teuthology.orchestra.run.trial057.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-21T00:09:48.821 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-21T00:09:48.821 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial057.front.sepia.ceph.com 2026-02-21T00:09:48.821 DEBUG:teuthology.orchestra.run.trial057:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-21T00:09:48.879 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-21T00:09:48.879 DEBUG:teuthology.orchestra.run.trial057:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T00:09:48.982 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-21T00:09:48.982 DEBUG:teuthology.orchestra.run.trial057:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T00:09:49.007 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T00:09:49.007 INFO:teuthology.orchestra.run.trial057.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T00:09:49.007 INFO:teuthology.orchestra.run.trial057.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T00:09:49.007 INFO:teuthology.orchestra.run.trial057.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-21T00:09:49.007 INFO:teuthology.orchestra.run.trial057.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-21T00:09:49.016 INFO:teuthology.orchestra.run.trial057.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.1% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-21T00:09:49.018 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-21T00:09:49.034 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-21T00:09:49.034 DEBUG:teuthology.orchestra.run.trial057:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-21T00:09:49.082 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-21T00:09:49.096 DEBUG:teuthology.orchestra.run.trial057:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-21T00:09:49.144 INFO:teuthology.orchestra.run.trial057.stdout:kernel.core_pattern = core 2026-02-21T00:09:49.155 DEBUG:teuthology.orchestra.run.trial057:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-21T00:09:49.210 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T00:09:49.211 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-21T00:09:49.222 INFO:teuthology.task.internal:Transferring archived files... 2026-02-21T00:09:49.223 DEBUG:teuthology.misc:Transferring archived files from trial057:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61870/remote/trial057 2026-02-21T00:09:49.223 DEBUG:teuthology.orchestra.run.trial057:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-21T00:09:49.285 INFO:teuthology.task.internal:Removing archive directory... 2026-02-21T00:09:49.285 DEBUG:teuthology.orchestra.run.trial057:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-21T00:09:49.336 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-21T00:09:49.360 INFO:teuthology.task.internal:Not uploading archives. 2026-02-21T00:09:49.360 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-21T00:09:49.378 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-21T00:09:49.378 DEBUG:teuthology.orchestra.run.trial057:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-21T00:09:49.394 INFO:teuthology.orchestra.run.trial057.stdout: 917576 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 21 00:09 /home/ubuntu/cephtest 2026-02-21T00:09:49.395 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-21T00:09:49.423 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-21T00:09:49.456 INFO:teuthology.run:Summary data: description: crimson-rados/rbd/{0-crimson_install clusters/crimson-fixed-1 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} tasks/rbd_python_api_tests} duration: 2000.3547785282135 flavor: debug owner: scheduled_liucm@soko04.front.sepia.ceph.com success: true 2026-02-21T00:09:49.457 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T00:09:49.575 INFO:teuthology.run:pass