2026-02-12T06:33:09.861 INFO:root:teuthology version: 1.2.4.dev5+g8bec0da71 2026-02-12T06:33:09.876 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-12T06:33:09.955 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/moagrawa-2026-02-12_06:23:35-crimson-rados-main-distro-debug-trial/45989 branch: main description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} perf_ignorelist settings/optimized workloads/radosbench_omap_write} email: null first_in_suite: false flavor: debug job_id: '45989' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: moagrawa-2026-02-12_06:23:35-crimson-rados-main-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: global: auth client required: none auth cluster required: none auth service required: none auth supported: none debug asok: 0/0 debug auth: 0/0 debug bdev: 0/0 debug bluefs: 0/0 debug bluestore: 0/0 debug buffer: 0/0 debug civetweb: 0/0 debug client: 0/0 debug compressor: 0/0 debug context: 0/0 debug crush: 0/0 debug crypto: 0/0 debug dpdk: 0/0 debug eventtrace: 0/0 debug filer: 0/0 debug filestore: 0/0 debug finisher: 0/0 debug fuse: 0/0 debug heartbeatmap: 0/0 debug javaclient: 0/0 debug journal: 0/0 debug journaler: 0/0 debug kstore: 0/0 debug leveldb: 0/0 debug lockdep: 0/0 debug mds: 0/0 debug mds balancer: 0/0 debug mds locker: 0/0 debug mds log: 0/0 debug mds log expire: 0/0 debug mds migrator: 0/0 debug memdb: 0/0 debug mgr: 0/0 debug mgrc: 0/0 debug mon: 0/0 debug monc: 0/0 debug ms: 0/0 debug objclass: 0/0 debug objectcacher: 0/0 debug objecter: 0/0 debug optracker: 0/0 debug osd: 0/0 debug paxos: 0/0 debug perfcounter: 0/0 debug rados: 0/0 debug rbd: 0/0 debug rbd mirror: 0/0 debug rbd replay: 0/0 debug refs: 0/0 debug rgw: 0/0 debug rgw lifecycle: 0/0 debug rgw notification: 0/0 debug rgw sync: 0/0 debug rocksdb: 0/0 debug striper: 0/0 debug throttle: 0/0 debug timer: 0/0 debug tp: 0/0 enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 0/0 debug ms: 0/0 debug paxos: 0/0 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug filestore: 0/0 debug journal: 0/0 debug ms: 0/0 debug osd: 0/0 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore main device type: RANDOM_BLOCK_SSD seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - \(PG_ - \(OSD_ - \(OBJECT_ - overall HEALTH sha1: 5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: flavor: debug sha1: 5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 workunit: branch: seastore_io_uring env: CRIMSON_COMPAT: '1' sha1: 5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 owner: scheduled_moagrawa@soko04.front.sepia.ceph.com priority: 20 repo: https://github.com/ceph/ceph.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 4661 sha1: 5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 sleep_before_teardown: 0 suite: crimson-rados suite_branch: seastore_io_uring suite_path: /home/teuthworker/src/github.com_mohit84_ceph_5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/qa suite_relpath: qa suite_repo: https://github.com/mohit84/ceph.git suite_sha1: 5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 targets: trial028.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNVE/0HhYQzW42G38rTjqdntFpns3RvRehmIe6cqOFB3Nc9/qt84vtFVVXfZ9Xis+wFctgP+xJ6ZpurHPdcOFvs= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: osd: debug monc: 20 wait-for-scrub: false - ssh_keys: null - radosbench: clients: - client.0 objectsize: 4096 size: 4096 time: 300 write-omap: true teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: 8bec0da71becad44414c54979f64c9ef0e7099c6 timestamp: 2026-02-12_06:23:35 tube: trial user: moagrawa verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.2161924 2026-02-12T06:33:09.955 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_mohit84_ceph_5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/qa; will attempt to use it 2026-02-12T06:33:09.956 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_mohit84_ceph_5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/qa/tasks 2026-02-12T06:33:09.956 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-12T06:33:09.957 INFO:teuthology.task.internal:Checking packages... 2026-02-12T06:33:09.957 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0' 2026-02-12T06:33:09.957 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-12T06:33:09.957 INFO:teuthology.packaging:ref: None 2026-02-12T06:33:09.957 INFO:teuthology.packaging:tag: None 2026-02-12T06:33:09.957 INFO:teuthology.packaging:branch: main 2026-02-12T06:33:09.957 INFO:teuthology.packaging:sha1: 5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 2026-02-12T06:33:09.958 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=main 2026-02-12T06:33:10.113 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5256.gf2a98edf 2026-02-12T06:33:10.115 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-12T06:33:10.154 INFO:teuthology.task.internal:no buildpackages task found 2026-02-12T06:33:10.154 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-12T06:33:10.189 INFO:teuthology.task.internal:Saving configuration 2026-02-12T06:33:10.221 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-12T06:33:10.242 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-12T06:33:10.338 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial028.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/moagrawa-2026-02-12_06:23:35-crimson-rados-main-distro-debug-trial/45989', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-12 06:29:51.199478', 'locked_by': 'scheduled_moagrawa@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNVE/0HhYQzW42G38rTjqdntFpns3RvRehmIe6cqOFB3Nc9/qt84vtFVVXfZ9Xis+wFctgP+xJ6ZpurHPdcOFvs='} 2026-02-12T06:33:10.338 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-12T06:33:10.350 INFO:teuthology.task.internal:roles: ubuntu@trial028.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-12T06:33:10.350 INFO:teuthology.run_tasks:Running task console_log... 2026-02-12T06:33:10.466 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x777808ade160>, signals=[15]) 2026-02-12T06:33:10.466 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-12T06:33:10.477 INFO:teuthology.task.internal:Opening connections... 2026-02-12T06:33:10.478 DEBUG:teuthology.task.internal:connecting to ubuntu@trial028.front.sepia.ceph.com 2026-02-12T06:33:10.478 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T06:33:10.543 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-12T06:33:10.554 DEBUG:teuthology.orchestra.run.trial028:> uname -m 2026-02-12T06:33:10.614 INFO:teuthology.orchestra.run.trial028.stdout:x86_64 2026-02-12T06:33:10.614 DEBUG:teuthology.orchestra.run.trial028:> cat /etc/os-release 2026-02-12T06:33:10.670 INFO:teuthology.orchestra.run.trial028.stdout:NAME="CentOS Stream" 2026-02-12T06:33:10.670 INFO:teuthology.orchestra.run.trial028.stdout:VERSION="9" 2026-02-12T06:33:10.671 INFO:teuthology.orchestra.run.trial028.stdout:ID="centos" 2026-02-12T06:33:10.671 INFO:teuthology.orchestra.run.trial028.stdout:ID_LIKE="rhel fedora" 2026-02-12T06:33:10.671 INFO:teuthology.orchestra.run.trial028.stdout:VERSION_ID="9" 2026-02-12T06:33:10.671 INFO:teuthology.orchestra.run.trial028.stdout:PLATFORM_ID="platform:el9" 2026-02-12T06:33:10.671 INFO:teuthology.orchestra.run.trial028.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-12T06:33:10.671 INFO:teuthology.orchestra.run.trial028.stdout:ANSI_COLOR="0;31" 2026-02-12T06:33:10.671 INFO:teuthology.orchestra.run.trial028.stdout:LOGO="fedora-logo-icon" 2026-02-12T06:33:10.671 INFO:teuthology.orchestra.run.trial028.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-12T06:33:10.671 INFO:teuthology.orchestra.run.trial028.stdout:HOME_URL="https://centos.org/" 2026-02-12T06:33:10.671 INFO:teuthology.orchestra.run.trial028.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-12T06:33:10.671 INFO:teuthology.orchestra.run.trial028.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-12T06:33:10.671 INFO:teuthology.orchestra.run.trial028.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-12T06:33:10.672 INFO:teuthology.lock.ops:Updating trial028.front.sepia.ceph.com on lock server 2026-02-12T06:33:10.766 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-12T06:33:10.779 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-12T06:33:10.791 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-12T06:33:10.791 DEBUG:teuthology.orchestra.run.trial028:> test '!' -e /home/ubuntu/cephtest 2026-02-12T06:33:10.805 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-12T06:33:10.816 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-12T06:33:10.817 DEBUG:teuthology.orchestra.run.trial028:> test -z $(ls -A /var/lib/ceph) 2026-02-12T06:33:10.861 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-12T06:33:10.966 INFO:teuthology.run_tasks:Running task kernel... 2026-02-12T06:33:10.987 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-12T06:33:10.987 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-12T06:33:10.987 DEBUG:teuthology.orchestra.run.trial028:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-12T06:33:11.002 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T06:33:11.002 DEBUG:teuthology.orchestra.run.trial028:> uname -r 2026-02-12T06:33:11.058 INFO:teuthology.orchestra.run.trial028.stdout:5.14.0-645.el9.x86_64 2026-02-12T06:33:11.058 INFO:teuthology.task.kernel:Running kernel on trial028: 5.14.0-645.el9.x86_64 2026-02-12T06:33:11.058 DEBUG:teuthology.orchestra.run.trial028:> sudo yum install -y kernel 2026-02-12T06:33:19.635 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 8.9 MB 00:08 2026-02-12T06:33:27.283 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - AppStream 3.8 MB/s | 27 MB 00:07 2026-02-12T06:33:30.913 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - CRB 6.2 MB/s | 7.8 MB 00:01 2026-02-12T06:33:31.725 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - Extras packages 197 kB/s | 20 kB 00:00 2026-02-12T06:33:32.528 INFO:teuthology.orchestra.run.trial028.stdout:Extra Packages for Enterprise Linux 28 MB/s | 20 MB 00:00 2026-02-12T06:33:35.755 INFO:teuthology.orchestra.run.trial028.stdout:lab-extras 1.7 MB/s | 50 kB 00:00 2026-02-12T06:33:36.830 INFO:teuthology.orchestra.run.trial028.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-12T06:33:36.850 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:33:36.850 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:33:36.851 INFO:teuthology.orchestra.run.trial028.stdout: Package Architecture Version Repository Size 2026-02-12T06:33:36.851 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:33:36.851 INFO:teuthology.orchestra.run.trial028.stdout:Installing: 2026-02-12T06:33:36.851 INFO:teuthology.orchestra.run.trial028.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-12T06:33:36.851 INFO:teuthology.orchestra.run.trial028.stdout:Installing dependencies: 2026-02-12T06:33:36.851 INFO:teuthology.orchestra.run.trial028.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-12T06:33:36.851 INFO:teuthology.orchestra.run.trial028.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-12T06:33:36.851 INFO:teuthology.orchestra.run.trial028.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-12T06:33:36.851 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:33:36.851 INFO:teuthology.orchestra.run.trial028.stdout:Transaction Summary 2026-02-12T06:33:36.851 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:33:36.851 INFO:teuthology.orchestra.run.trial028.stdout:Install 4 Packages 2026-02-12T06:33:36.851 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:33:36.851 INFO:teuthology.orchestra.run.trial028.stdout:Total download size: 88 M 2026-02-12T06:33:36.851 INFO:teuthology.orchestra.run.trial028.stdout:Installed size: 135 M 2026-02-12T06:33:36.851 INFO:teuthology.orchestra.run.trial028.stdout:Downloading Packages: 2026-02-12T06:33:37.352 INFO:teuthology.orchestra.run.trial028.stdout:(1/4): kernel-5.14.0-677.el9.x86_64.rpm 4.6 MB/s | 768 kB 00:00 2026-02-12T06:33:37.511 INFO:teuthology.orchestra.run.trial028.stdout:(2/4): kernel-core-5.14.0-677.el9.x86_64.rpm 53 MB/s | 17 MB 00:00 2026-02-12T06:33:37.646 INFO:teuthology.orchestra.run.trial028.stdout:(3/4): kernel-modules-core-5.14.0-677.el9.x86_6 105 MB/s | 31 MB 00:00 2026-02-12T06:33:37.769 INFO:teuthology.orchestra.run.trial028.stdout:(4/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 68 MB/s | 40 MB 00:00 2026-02-12T06:33:37.770 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-12T06:33:37.770 INFO:teuthology.orchestra.run.trial028.stdout:Total 96 MB/s | 88 MB 00:00 2026-02-12T06:33:37.959 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction check 2026-02-12T06:33:38.161 INFO:teuthology.orchestra.run.trial028.stdout:Transaction check succeeded. 2026-02-12T06:33:38.162 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction test 2026-02-12T06:33:38.507 INFO:teuthology.orchestra.run.trial028.stdout:Transaction test succeeded. 2026-02-12T06:33:38.507 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction 2026-02-12T06:33:39.592 INFO:teuthology.orchestra.run.trial028.stdout: Preparing : 1/1 2026-02-12T06:33:39.876 INFO:teuthology.orchestra.run.trial028.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-12T06:33:39.894 INFO:teuthology.orchestra.run.trial028.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T06:33:41.647 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T06:33:41.676 INFO:teuthology.orchestra.run.trial028.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T06:33:44.701 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T06:33:44.707 INFO:teuthology.orchestra.run.trial028.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T06:33:47.740 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T06:34:18.156 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T06:34:18.176 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-12T06:34:18.242 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T06:34:18.242 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-12T06:34:18.242 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T06:34:18.242 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T06:34:18.294 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T06:34:18.294 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:34:18.294 INFO:teuthology.orchestra.run.trial028.stdout:Installed: 2026-02-12T06:34:18.295 INFO:teuthology.orchestra.run.trial028.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-12T06:34:18.295 INFO:teuthology.orchestra.run.trial028.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-12T06:34:18.295 INFO:teuthology.orchestra.run.trial028.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-12T06:34:18.295 INFO:teuthology.orchestra.run.trial028.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-12T06:34:18.295 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:34:18.295 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:34:18.352 DEBUG:teuthology.orchestra.run.trial028:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T06:34:18.377 INFO:teuthology.orchestra.run.trial028.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T06:34:18.378 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T06:34:18.378 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-12T06:34:18.378 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-12T06:34:18.378 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-12T06:34:18.378 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-12T06:34:18.378 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial028.front.sepia.ceph.com, path=None, version=distro) 2026-02-12T06:34:18.378 DEBUG:teuthology.orchestra.run.trial028:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T06:34:18.439 INFO:teuthology.orchestra.run.trial028.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T06:34:18.440 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T06:34:18.440 DEBUG:teuthology.orchestra.run.trial028:> sudo rpm -qi grub2-tools 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:Name : grub2-tools 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:Epoch : 1 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:Version : 2.06 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:Release : 118.el9 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:Architecture: x86_64 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:Group : Unspecified 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:Size : 8393909 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:License : GPLv3+ 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:Packager : builder@centos.org 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:Vendor : CentOS 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:Summary : Support tools for GRUB. 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:Description : 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:hardware devices. 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:34:18.508 INFO:teuthology.orchestra.run.trial028.stdout:This subpackage provides tools for support of all platforms. 2026-02-12T06:34:18.509 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-12T06:34:18.509 INFO:teuthology.task.kernel:Updating grub on trial028 to boot 5.14.0-677.el9.x86_64 2026-02-12T06:34:18.509 DEBUG:teuthology.orchestra.run.trial028:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-12T06:34:18.575 INFO:teuthology.orchestra.run.trial028.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-12T06:34:18.576 DEBUG:teuthology.orchestra.run.trial028:> sudo command -v grubby && echo yes || echo no 2026-02-12T06:34:18.641 INFO:teuthology.orchestra.run.trial028.stdout:/sbin/grubby 2026-02-12T06:34:18.642 INFO:teuthology.orchestra.run.trial028.stdout:yes 2026-02-12T06:34:18.642 DEBUG:teuthology.orchestra.run.trial028:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-12T06:34:18.708 INFO:teuthology.orchestra.run.trial028.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-12T06:34:18.709 DEBUG:teuthology.orchestra.run.trial028:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-12T06:34:18.781 DEBUG:teuthology.orchestra.run.trial028:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-12T06:34:18.989 INFO:teuthology.orchestra.run.trial028.stderr:Generating grub configuration file ... 2026-02-12T06:34:19.052 INFO:teuthology.orchestra.run.trial028.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-12T06:34:19.058 INFO:teuthology.orchestra.run.trial028.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-12T06:34:19.309 INFO:teuthology.orchestra.run.trial028.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-12T06:34:19.316 INFO:teuthology.orchestra.run.trial028.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-12T06:34:19.348 INFO:teuthology.orchestra.run.trial028.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-12T06:34:19.354 INFO:teuthology.orchestra.run.trial028.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-12T06:34:19.377 INFO:teuthology.orchestra.run.trial028.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-12T06:34:19.384 INFO:teuthology.orchestra.run.trial028.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-12T06:34:19.467 INFO:teuthology.orchestra.run.trial028.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-12T06:34:19.476 INFO:teuthology.orchestra.run.trial028.stderr:done 2026-02-12T06:34:19.478 DEBUG:teuthology.orchestra.run.trial028:> test -d /sys/firmware/efi 2026-02-12T06:34:19.493 DEBUG:teuthology.orchestra.run.trial028:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-12T06:34:19.557 INFO:teuthology.orchestra.run.trial028.stdout:/boot/efi/EFI/BOOT 2026-02-12T06:34:19.558 INFO:teuthology.orchestra.run.trial028.stdout:/boot/efi/EFI/centos 2026-02-12T06:34:19.559 DEBUG:teuthology.orchestra.run.trial028:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-12T06:34:19.624 INFO:teuthology.orchestra.run.trial028.stdout:no 2026-02-12T06:34:19.624 DEBUG:teuthology.orchestra.run.trial028:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-12T06:34:19.688 INFO:teuthology.orchestra.run.trial028.stdout:yes 2026-02-12T06:34:19.689 DEBUG:teuthology.orchestra.run.trial028:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-12T06:34:19.755 DEBUG:teuthology.orchestra.run.trial028:> sudo shutdown -r now 2026-02-12T06:34:19.827 ERROR:paramiko.transport:Socket exception: Connection reset by peer (104) 2026-02-12T06:34:49.828 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-12T06:34:49.828 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial028.front.sepia.ceph.com' 2026-02-12T06:34:49.828 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T06:35:49.890 DEBUG:teuthology.orchestra.remote:timed out 2026-02-12T06:35:58.899 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial028.front.sepia.ceph.com' 2026-02-12T06:35:58.900 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T06:35:59.045 DEBUG:teuthology.orchestra.run.trial028:> true 2026-02-12T06:35:59.199 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial028.front.sepia.ceph.com' 2026-02-12T06:35:59.199 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-677.el9.x86_64"... 2026-02-12T06:35:59.199 DEBUG:teuthology.orchestra.run.trial028:> uname -r 2026-02-12T06:35:59.255 INFO:teuthology.orchestra.run.trial028.stdout:5.14.0-677.el9.x86_64 2026-02-12T06:35:59.255 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-677.el9.x86_64 vs 5.14.0-677.el9.x86_64 2026-02-12T06:35:59.255 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-12T06:35:59.255 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-12T06:36:00.256 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-12T06:36:00.256 DEBUG:teuthology.orchestra.run.trial028:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-12T06:36:00.286 INFO:teuthology.orchestra.run.trial028.stdout:ttyS1 2026-02-12T06:36:00.317 DEBUG:teuthology.parallel:result is None 2026-02-12T06:36:00.317 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-12T06:36:00.328 INFO:teuthology.task.internal:Creating test directory... 2026-02-12T06:36:00.328 DEBUG:teuthology.orchestra.run.trial028:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-12T06:36:00.373 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-12T06:36:00.385 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-12T06:36:00.398 INFO:teuthology.task.internal:Creating archive directory... 2026-02-12T06:36:00.398 DEBUG:teuthology.orchestra.run.trial028:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-12T06:36:00.436 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-12T06:36:00.447 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-12T06:36:00.447 DEBUG:teuthology.orchestra.run.trial028:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-12T06:36:00.493 INFO:teuthology.orchestra.run.trial028.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T06:36:00.502 INFO:teuthology.orchestra.run.trial028.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T06:36:00.504 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-12T06:36:00.516 INFO:teuthology.task.internal:Configuring sudo... 2026-02-12T06:36:00.516 DEBUG:teuthology.orchestra.run.trial028:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-12T06:36:00.568 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-12T06:36:00.585 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-12T06:36:00.585 DEBUG:teuthology.orchestra.run.trial028:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-12T06:36:00.624 DEBUG:teuthology.orchestra.run.trial028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T06:36:00.786 DEBUG:teuthology.orchestra.run.trial028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T06:36:00.810 DEBUG:teuthology.orchestra.run.trial028:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T06:36:00.960 DEBUG:teuthology.orchestra.run.trial028:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T06:36:00.985 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:36:00.985 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-12T06:36:01.048 DEBUG:teuthology.orchestra.run.trial028:> sudo service rsyslog restart 2026-02-12T06:36:01.116 INFO:teuthology.orchestra.run.trial028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-12T06:36:01.510 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-12T06:36:01.528 INFO:teuthology.task.internal:Starting timer... 2026-02-12T06:36:01.528 INFO:teuthology.run_tasks:Running task pcp... 2026-02-12T06:36:01.556 INFO:teuthology.run_tasks:Running task selinux... 2026-02-12T06:36:01.580 DEBUG:teuthology.orchestra.run.trial028:> sudo service auditd rotate 2026-02-12T06:36:01.619 INFO:teuthology.orchestra.run.trial028.stdout:Rotating logs: 2026-02-12T06:36:01.621 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-12T06:36:01.621 DEBUG:teuthology.orchestra.run.trial028:> /usr/sbin/getenforce 2026-02-12T06:36:01.685 INFO:teuthology.orchestra.run.trial028.stdout:Permissive 2026-02-12T06:36:01.686 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial028.front.sepia.ceph.com': 'permissive'} 2026-02-12T06:36:01.686 DEBUG:teuthology.orchestra.run.trial028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-12T06:36:01.753 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T06:36:01.754 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-12T06:36:01.754 DEBUG:teuthology.orchestra.run.trial028:> sudo /usr/sbin/setenforce permissive 2026-02-12T06:36:01.819 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-12T06:36:01.834 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-12T06:36:01.839 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-12T06:36:01.947 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-12T06:36:01.960 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-12T06:36:01.962 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial028.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-12T06:37:09.602 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial028.front.sepia.ceph.com')] 2026-02-12T06:37:09.604 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial028.front.sepia.ceph.com' 2026-02-12T06:37:09.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial028.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T06:37:09.717 DEBUG:teuthology.orchestra.run.trial028:> true 2026-02-12T06:37:09.793 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial028.front.sepia.ceph.com' 2026-02-12T06:37:09.793 INFO:teuthology.run_tasks:Running task clock... 2026-02-12T06:37:09.816 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-12T06:37:09.816 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-12T06:37:09.817 DEBUG:teuthology.orchestra.run.trial028:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-12T06:37:09.862 INFO:teuthology.orchestra.run.trial028.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-12T06:37:09.874 INFO:teuthology.orchestra.run.trial028.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-12T06:37:09.895 INFO:teuthology.orchestra.run.trial028.stderr:sudo: ntpd: command not found 2026-02-12T06:37:09.906 INFO:teuthology.orchestra.run.trial028.stdout:506 Cannot talk to daemon 2026-02-12T06:37:09.918 INFO:teuthology.orchestra.run.trial028.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-12T06:37:09.930 INFO:teuthology.orchestra.run.trial028.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-12T06:37:09.977 INFO:teuthology.orchestra.run.trial028.stderr:bash: line 1: ntpq: command not found 2026-02-12T06:37:09.979 INFO:teuthology.orchestra.run.trial028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-12T06:37:09.979 INFO:teuthology.orchestra.run.trial028.stdout:=============================================================================== 2026-02-12T06:37:09.980 INFO:teuthology.run_tasks:Running task install... 2026-02-12T06:37:09.995 DEBUG:teuthology.task.install:project ceph 2026-02-12T06:37:09.996 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'}} 2026-02-12T06:37:09.996 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'} 2026-02-12T06:37:09.996 INFO:teuthology.task.install:Using flavor: debug 2026-02-12T06:37:10.006 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-12T06:37:10.006 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-12T06:37:10.007 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0', 'tag': None, 'wait_for_package': False} 2026-02-12T06:37:10.008 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 2026-02-12T06:37:10.184 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-moagrawa-seastore-io-uring-again-crimson-only/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/centos/9/flavors/debug/ 2026-02-12T06:37:10.185 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5145.g5fdb7d8f 2026-02-12T06:37:10.533 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-moagrawa-seastore-io-uring-again-crimson-only/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-moagrawa-seastore-io-uring-again-crimson-only/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-moagrawa-seastore-io-uring-again-crimson-only/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-12T06:37:10.533 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:37:10.533 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-12T06:37:10.559 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-12T06:37:10.560 DEBUG:teuthology.orchestra.run.trial028:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-12T06:37:10.625 DEBUG:teuthology.orchestra.run.trial028:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-12T06:37:10.700 DEBUG:teuthology.orchestra.run.trial028:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-12T06:37:10.763 INFO:teuthology.orchestra.run.trial028.stdout:check_obsoletes = 1 2026-02-12T06:37:10.765 DEBUG:teuthology.orchestra.run.trial028:> sudo yum clean all 2026-02-12T06:37:10.917 INFO:teuthology.orchestra.run.trial028.stdout:41 files removed 2026-02-12T06:37:10.932 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-radosgw 2026-02-12T06:37:11.757 INFO:teuthology.orchestra.run.trial028.stdout:ceph packages for x86_64 138 kB/s | 94 kB 00:00 2026-02-12T06:37:12.247 INFO:teuthology.orchestra.run.trial028.stdout:ceph noarch packages 52 kB/s | 24 kB 00:00 2026-02-12T06:37:12.684 INFO:teuthology.orchestra.run.trial028.stdout:ceph source packages 5.0 kB/s | 2.1 kB 00:00 2026-02-12T06:37:14.241 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - BaseOS 5.8 MB/s | 8.9 MB 00:01 2026-02-12T06:37:17.272 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-12T06:37:27.868 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - CRB 968 kB/s | 7.8 MB 00:08 2026-02-12T06:37:28.732 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - Extras packages 145 kB/s | 20 kB 00:00 2026-02-12T06:37:30.067 INFO:teuthology.orchestra.run.trial028.stdout:Extra Packages for Enterprise Linux 16 MB/s | 20 MB 00:01 2026-02-12T06:37:33.369 INFO:teuthology.orchestra.run.trial028.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-12T06:37:34.666 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:37:34.667 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:37:34.667 INFO:teuthology.orchestra.run.trial028.stdout: Package Arch Version Repository Size 2026-02-12T06:37:34.667 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:37:34.667 INFO:teuthology.orchestra.run.trial028.stdout:Installing: 2026-02-12T06:37:34.667 INFO:teuthology.orchestra.run.trial028.stdout: ceph-radosgw x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 11 M 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout:Upgrading: 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: librados2 x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 3.3 M 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: librbd1 x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 2.7 M 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout:Installing dependencies: 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: ceph-base x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 5.1 M 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: ceph-common x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 21 M 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: ceph-selinux x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 25 k 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs2 x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 860 k 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: libradosstriper1 x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 263 k 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: librgw2 x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 5.6 M 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-12T06:37:34.668 INFO:teuthology.orchestra.run.trial028.stdout: python3-ceph-argparse x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 45 k 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout: python3-ceph-common x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 184 k 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout: python3-cephfs x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 203 k 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout: python3-rados x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 324 k 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout: python3-rbd x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 305 k 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout: python3-rgw x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 100 k 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout:Installing weak dependencies: 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs-daemon x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 35 k 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 24 k 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout: nvme-cli x86_64 2.13-1.el9 baseos 1.1 M 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout:Transaction Summary 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout:Install 35 Packages 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout:Upgrade 2 Packages 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout:Total download size: 61 M 2026-02-12T06:37:34.669 INFO:teuthology.orchestra.run.trial028.stdout:Downloading Packages: 2026-02-12T06:37:35.714 INFO:teuthology.orchestra.run.trial028.stdout:(1/37): ceph-base-20.3.0-5145.g5fdb7d8f.el9.x86 18 MB/s | 5.1 MB 00:00 2026-02-12T06:37:35.747 INFO:teuthology.orchestra.run.trial028.stdout:(2/37): ceph-selinux-20.3.0-5145.g5fdb7d8f.el9. 749 kB/s | 25 kB 00:00 2026-02-12T06:37:35.786 INFO:teuthology.orchestra.run.trial028.stdout:(3/37): libcephfs-daemon-20.3.0-5145.g5fdb7d8f. 916 kB/s | 35 kB 00:00 2026-02-12T06:37:35.825 INFO:teuthology.orchestra.run.trial028.stdout:(4/37): libcephfs-proxy2-20.3.0-5145.g5fdb7d8f. 612 kB/s | 24 kB 00:00 2026-02-12T06:37:35.883 INFO:teuthology.orchestra.run.trial028.stdout:(5/37): libcephfs2-20.3.0-5145.g5fdb7d8f.el9.x8 15 MB/s | 860 kB 00:00 2026-02-12T06:37:35.931 INFO:teuthology.orchestra.run.trial028.stdout:(6/37): libradosstriper1-20.3.0-5145.g5fdb7d8f. 5.4 MB/s | 263 kB 00:00 2026-02-12T06:37:36.039 INFO:teuthology.orchestra.run.trial028.stdout:(7/37): ceph-radosgw-20.3.0-5145.g5fdb7d8f.el9. 18 MB/s | 11 MB 00:00 2026-02-12T06:37:36.092 INFO:teuthology.orchestra.run.trial028.stdout:(8/37): python3-ceph-argparse-20.3.0-5145.g5fdb 838 kB/s | 45 kB 00:00 2026-02-12T06:37:36.150 INFO:teuthology.orchestra.run.trial028.stdout:(9/37): python3-ceph-common-20.3.0-5145.g5fdb7d 3.1 MB/s | 184 kB 00:00 2026-02-12T06:37:36.178 INFO:teuthology.orchestra.run.trial028.stdout:(10/37): librgw2-20.3.0-5145.g5fdb7d8f.el9.x86_ 23 MB/s | 5.6 MB 00:00 2026-02-12T06:37:36.202 INFO:teuthology.orchestra.run.trial028.stdout:(11/37): python3-cephfs-20.3.0-5145.g5fdb7d8f.e 3.8 MB/s | 203 kB 00:00 2026-02-12T06:37:36.220 INFO:teuthology.orchestra.run.trial028.stdout:(12/37): python3-rados-20.3.0-5145.g5fdb7d8f.el 7.5 MB/s | 324 kB 00:00 2026-02-12T06:37:36.245 INFO:teuthology.orchestra.run.trial028.stdout:(13/37): python3-rbd-20.3.0-5145.g5fdb7d8f.el9. 7.0 MB/s | 305 kB 00:00 2026-02-12T06:37:36.256 INFO:teuthology.orchestra.run.trial028.stdout:(14/37): mailcap-2.1.49-5.el9.noarch.rpm 3.0 MB/s | 33 kB 00:00 2026-02-12T06:37:36.264 INFO:teuthology.orchestra.run.trial028.stdout:(15/37): python3-rgw-20.3.0-5145.g5fdb7d8f.el9. 2.5 MB/s | 100 kB 00:00 2026-02-12T06:37:36.275 INFO:teuthology.orchestra.run.trial028.stdout:(16/37): pciutils-3.7.0-7.el9.x86_64.rpm 6.5 MB/s | 93 kB 00:00 2026-02-12T06:37:36.276 INFO:teuthology.orchestra.run.trial028.stdout:(17/37): nvme-cli-2.13-1.el9.x86_64.rpm 55 MB/s | 1.1 MB 00:00 2026-02-12T06:37:36.308 INFO:teuthology.orchestra.run.trial028.stdout:(18/37): ceph-common-20.3.0-5145.g5fdb7d8f.el9. 24 MB/s | 21 MB 00:00 2026-02-12T06:37:36.430 INFO:teuthology.orchestra.run.trial028.stdout:(19/37): boost-program-options-1.75.0-13.el9.x8 668 kB/s | 104 kB 00:00 2026-02-12T06:37:36.445 INFO:teuthology.orchestra.run.trial028.stdout:(20/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2026-02-12T06:37:36.446 INFO:teuthology.orchestra.run.trial028.stdout:(21/37): libnbd-1.20.3-4.el9.x86_64.rpm 963 kB/s | 164 kB 00:00 2026-02-12T06:37:36.462 INFO:teuthology.orchestra.run.trial028.stdout:(22/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.4 MB/s | 45 kB 00:00 2026-02-12T06:37:36.505 INFO:teuthology.orchestra.run.trial028.stdout:(23/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 4.8 MB/s | 292 kB 00:00 2026-02-12T06:37:36.524 INFO:teuthology.orchestra.run.trial028.stdout:(24/37): qatlib-25.08.0-2.el9.x86_64.rpm 3.8 MB/s | 240 kB 00:00 2026-02-12T06:37:36.528 INFO:teuthology.orchestra.run.trial028.stdout:(25/37): librdkafka-1.6.1-102.el9.x86_64.rpm 7.8 MB/s | 662 kB 00:00 2026-02-12T06:37:36.530 INFO:teuthology.orchestra.run.trial028.stdout:(26/37): qatlib-service-25.08.0-2.el9.x86_64.rp 1.4 MB/s | 37 kB 00:00 2026-02-12T06:37:36.551 INFO:teuthology.orchestra.run.trial028.stdout:(27/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 2.4 MB/s | 66 kB 00:00 2026-02-12T06:37:36.744 INFO:teuthology.orchestra.run.trial028.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 128 kB/s | 25 kB 00:00 2026-02-12T06:37:36.874 INFO:teuthology.orchestra.run.trial028.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 376 kB/s | 49 kB 00:00 2026-02-12T06:37:36.951 INFO:teuthology.orchestra.run.trial028.stdout:(30/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 728 kB/s | 308 kB 00:00 2026-02-12T06:37:36.970 INFO:teuthology.orchestra.run.trial028.stdout:(31/37): libunwind-1.6.2-1.el9.x86_64.rpm 704 kB/s | 67 kB 00:00 2026-02-12T06:37:37.126 INFO:teuthology.orchestra.run.trial028.stdout:(32/37): re2-20211101-20.el9.x86_64.rpm 1.2 MB/s | 191 kB 00:00 2026-02-12T06:37:37.136 INFO:teuthology.orchestra.run.trial028.stdout:(33/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 4.4 MB/s | 838 kB 00:00 2026-02-12T06:37:37.266 INFO:teuthology.orchestra.run.trial028.stdout:(34/37): librados2-20.3.0-5145.g5fdb7d8f.el9.x8 25 MB/s | 3.3 MB 00:00 2026-02-12T06:37:37.317 INFO:teuthology.orchestra.run.trial028.stdout:(35/37): librbd1-20.3.0-5145.g5fdb7d8f.el9.x86_ 53 MB/s | 2.7 MB 00:00 2026-02-12T06:37:37.383 INFO:teuthology.orchestra.run.trial028.stdout:(36/37): thrift-0.15.0-4.el9.x86_64.rpm 6.1 MB/s | 1.6 MB 00:00 2026-02-12T06:37:37.396 INFO:teuthology.orchestra.run.trial028.stdout:(37/37): libarrow-9.0.0-15.el9.x86_64.rpm 5.1 MB/s | 4.4 MB 00:00 2026-02-12T06:37:37.398 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-12T06:37:37.398 INFO:teuthology.orchestra.run.trial028.stdout:Total 22 MB/s | 61 MB 00:02 2026-02-12T06:37:37.496 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction check 2026-02-12T06:37:37.514 INFO:teuthology.orchestra.run.trial028.stdout:Transaction check succeeded. 2026-02-12T06:37:37.514 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction test 2026-02-12T06:37:37.759 INFO:teuthology.orchestra.run.trial028.stdout:Transaction test succeeded. 2026-02-12T06:37:37.759 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction 2026-02-12T06:37:38.048 INFO:teuthology.orchestra.run.trial028.stdout: Preparing : 1/1 2026-02-12T06:37:38.198 INFO:teuthology.orchestra.run.trial028.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-12T06:37:38.203 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : librados2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2/39 2026-02-12T06:37:38.254 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: librados2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2/39 2026-02-12T06:37:38.255 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libcephfs2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 3/39 2026-02-12T06:37:38.267 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libcephfs2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 3/39 2026-02-12T06:37:38.269 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libcephfs-daemon-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 4/39 2026-02-12T06:37:38.270 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libcephfs-proxy2-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 5/39 2026-02-12T06:37:38.298 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 5/39 2026-02-12T06:37:38.301 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-rados-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 6/39 2026-02-12T06:37:38.309 INFO:teuthology.orchestra.run.trial028.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-12T06:37:38.312 INFO:teuthology.orchestra.run.trial028.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-12T06:37:38.315 INFO:teuthology.orchestra.run.trial028.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-12T06:37:38.319 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-12T06:37:38.330 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-ceph-argparse-2:20.3.0-5145.g5fdb7d8f.el9. 11/39 2026-02-12T06:37:38.343 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cephfs-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 12/39 2026-02-12T06:37:38.343 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libradosstriper1-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 13/39 2026-02-12T06:37:38.366 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 13/39 2026-02-12T06:37:38.370 INFO:teuthology.orchestra.run.trial028.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-12T06:37:38.400 INFO:teuthology.orchestra.run.trial028.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-12T06:37:38.403 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-12T06:37:38.412 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-12T06:37:38.414 INFO:teuthology.orchestra.run.trial028.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-12T06:37:38.418 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-12T06:37:38.536 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-12T06:37:38.538 INFO:teuthology.orchestra.run.trial028.stdout: Upgrading : librbd1-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 21/39 2026-02-12T06:37:38.564 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: librbd1-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 21/39 2026-02-12T06:37:38.567 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-rbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 22/39 2026-02-12T06:37:38.572 INFO:teuthology.orchestra.run.trial028.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-12T06:37:38.835 INFO:teuthology.orchestra.run.trial028.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-12T06:37:38.836 INFO:teuthology.orchestra.run.trial028.stdout: Installing : librgw2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 25/39 2026-02-12T06:37:38.852 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: librgw2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 25/39 2026-02-12T06:37:38.856 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-rgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 26/39 2026-02-12T06:37:38.860 INFO:teuthology.orchestra.run.trial028.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-12T06:37:38.863 INFO:teuthology.orchestra.run.trial028.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-12T06:37:38.864 INFO:teuthology.orchestra.run.trial028.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-12T06:37:38.881 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-12T06:37:38.992 INFO:teuthology.orchestra.run.trial028.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-12T06:37:38.994 INFO:teuthology.orchestra.run.trial028.stdout: Installing : nvme-cli-2.13-1.el9.x86_64 31/39 2026-02-12T06:37:39.266 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: nvme-cli-2.13-1.el9.x86_64 31/39 2026-02-12T06:37:39.266 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-12T06:37:39.267 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:37:39.311 INFO:teuthology.orchestra.run.trial028.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-12T06:37:39.314 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-ceph-common-2:20.3.0-5145.g5fdb7d8f.el9.x8 33/39 2026-02-12T06:37:40.299 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-common-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 34/39 2026-02-12T06:37:40.301 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-common-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 34/39 2026-02-12T06:37:40.573 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-common-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 34/39 2026-02-12T06:37:40.576 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-base-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 35/39 2026-02-12T06:37:40.614 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-base-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 35/39 2026-02-12T06:37:40.614 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-12T06:37:40.614 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-12T06:37:40.614 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:37:40.617 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-selinux-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 36/39 2026-02-12T06:37:45.212 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 36/39 2026-02-12T06:37:45.213 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /sys 2026-02-12T06:37:45.213 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /proc 2026-02-12T06:37:45.213 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /mnt 2026-02-12T06:37:45.213 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /var/tmp 2026-02-12T06:37:45.213 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /home 2026-02-12T06:37:45.213 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /root 2026-02-12T06:37:45.213 INFO:teuthology.orchestra.run.trial028.stdout:skipping the directory /tmp 2026-02-12T06:37:45.213 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:37:45.723 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-radosgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 37/39 2026-02-12T06:37:45.748 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 37/39 2026-02-12T06:37:45.748 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T06:37:45.748 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-12T06:37:45.748 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-12T06:37:45.748 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-12T06:37:45.748 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:37:45.749 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-12T06:37:45.763 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-12T06:37:45.763 INFO:teuthology.orchestra.run.trial028.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-12T06:37:45.818 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-12T06:37:45.818 INFO:teuthology.orchestra.run.trial028.stdout:Creating group 'qat' with GID 992. 2026-02-12T06:37:45.819 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:37:46.097 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-base-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/39 2026-02-12T06:37:46.098 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-common-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2/39 2026-02-12T06:37:46.098 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-radosgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 3/39 2026-02-12T06:37:46.098 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-selinux-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 4/39 2026-02-12T06:37:46.098 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libcephfs-daemon-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 5/39 2026-02-12T06:37:46.098 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 6/39 2026-02-12T06:37:46.098 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libcephfs2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 7/39 2026-02-12T06:37:46.098 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libradosstriper1-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 8/39 2026-02-12T06:37:46.098 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librgw2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 9/39 2026-02-12T06:37:46.098 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5145.g5fdb7d8f.el9. 10/39 2026-02-12T06:37:46.098 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-ceph-common-2:20.3.0-5145.g5fdb7d8f.el9.x8 11/39 2026-02-12T06:37:46.098 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cephfs-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 12/39 2026-02-12T06:37:46.098 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-rados-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 13/39 2026-02-12T06:37:46.098 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-rbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 14/39 2026-02-12T06:37:46.098 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-rgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 15/39 2026-02-12T06:37:46.098 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-12T06:37:46.098 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : nvme-cli-2.13-1.el9.x86_64 17/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-12T06:37:46.099 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-12T06:37:46.100 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-12T06:37:46.100 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librados2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 36/39 2026-02-12T06:37:46.100 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-12T06:37:46.100 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librbd1-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 38/39 2026-02-12T06:37:46.159 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-12T06:37:46.159 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:37:46.159 INFO:teuthology.orchestra.run.trial028.stdout:Upgraded: 2026-02-12T06:37:46.159 INFO:teuthology.orchestra.run.trial028.stdout: librados2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.159 INFO:teuthology.orchestra.run.trial028.stdout: librbd1-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.159 INFO:teuthology.orchestra.run.trial028.stdout:Installed: 2026-02-12T06:37:46.159 INFO:teuthology.orchestra.run.trial028.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-12T06:37:46.159 INFO:teuthology.orchestra.run.trial028.stdout: ceph-base-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.159 INFO:teuthology.orchestra.run.trial028.stdout: ceph-common-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.159 INFO:teuthology.orchestra.run.trial028.stdout: ceph-radosgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: ceph-selinux-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs-daemon-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs-proxy2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: libradosstriper1-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: librgw2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-12T06:37:46.160 INFO:teuthology.orchestra.run.trial028.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-12T06:37:46.161 INFO:teuthology.orchestra.run.trial028.stdout: nvme-cli-2.13-1.el9.x86_64 2026-02-12T06:37:46.161 INFO:teuthology.orchestra.run.trial028.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-12T06:37:46.161 INFO:teuthology.orchestra.run.trial028.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-12T06:37:46.161 INFO:teuthology.orchestra.run.trial028.stdout: python3-ceph-argparse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.161 INFO:teuthology.orchestra.run.trial028.stdout: python3-ceph-common-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.161 INFO:teuthology.orchestra.run.trial028.stdout: python3-cephfs-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.161 INFO:teuthology.orchestra.run.trial028.stdout: python3-rados-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.161 INFO:teuthology.orchestra.run.trial028.stdout: python3-rbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.161 INFO:teuthology.orchestra.run.trial028.stdout: python3-rgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:46.161 INFO:teuthology.orchestra.run.trial028.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-12T06:37:46.161 INFO:teuthology.orchestra.run.trial028.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-12T06:37:46.161 INFO:teuthology.orchestra.run.trial028.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-12T06:37:46.161 INFO:teuthology.orchestra.run.trial028.stdout: re2-1:20211101-20.el9.x86_64 2026-02-12T06:37:46.161 INFO:teuthology.orchestra.run.trial028.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-12T06:37:46.161 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:37:46.161 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:37:46.201 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-test 2026-02-12T06:37:46.503 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:13 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:37:46.588 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:37:46.589 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:37:46.589 INFO:teuthology.orchestra.run.trial028.stdout: Package Arch Version Repository Size 2026-02-12T06:37:46.589 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:37:46.589 INFO:teuthology.orchestra.run.trial028.stdout:Installing: 2026-02-12T06:37:46.590 INFO:teuthology.orchestra.run.trial028.stdout: ceph-test x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 126 M 2026-02-12T06:37:46.590 INFO:teuthology.orchestra.run.trial028.stdout:Installing dependencies: 2026-02-12T06:37:46.590 INFO:teuthology.orchestra.run.trial028.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-12T06:37:46.590 INFO:teuthology.orchestra.run.trial028.stdout: libasan8 x86_64 15.1.1-2.5.el9 appstream 523 k 2026-02-12T06:37:46.590 INFO:teuthology.orchestra.run.trial028.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-12T06:37:46.590 INFO:teuthology.orchestra.run.trial028.stdout: libcephsqlite x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 153 k 2026-02-12T06:37:46.590 INFO:teuthology.orchestra.run.trial028.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-12T06:37:46.590 INFO:teuthology.orchestra.run.trial028.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-12T06:37:46.590 INFO:teuthology.orchestra.run.trial028.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-12T06:37:46.590 INFO:teuthology.orchestra.run.trial028.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-12T06:37:46.591 INFO:teuthology.orchestra.run.trial028.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-12T06:37:46.591 INFO:teuthology.orchestra.run.trial028.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-12T06:37:46.591 INFO:teuthology.orchestra.run.trial028.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-12T06:37:46.591 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:37:46.591 INFO:teuthology.orchestra.run.trial028.stdout:Transaction Summary 2026-02-12T06:37:46.591 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:37:46.591 INFO:teuthology.orchestra.run.trial028.stdout:Install 12 Packages 2026-02-12T06:37:46.591 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:37:46.592 INFO:teuthology.orchestra.run.trial028.stdout:Total download size: 131 M 2026-02-12T06:37:46.592 INFO:teuthology.orchestra.run.trial028.stdout:Installed size: 977 M 2026-02-12T06:37:46.592 INFO:teuthology.orchestra.run.trial028.stdout:Downloading Packages: 2026-02-12T06:37:46.927 INFO:teuthology.orchestra.run.trial028.stdout:(1/12): libcephsqlite-20.3.0-5145.g5fdb7d8f.el9 1.5 MB/s | 153 kB 00:00 2026-02-12T06:37:47.470 INFO:teuthology.orchestra.run.trial028.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 42 kB/s | 23 kB 00:00 2026-02-12T06:37:47.866 INFO:teuthology.orchestra.run.trial028.stdout:(3/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 2.0 MB/s | 2.1 MB 00:01 2026-02-12T06:37:47.920 INFO:teuthology.orchestra.run.trial028.stdout:(4/12): libasan8-15.1.1-2.5.el9.x86_64.rpm 9.4 MB/s | 523 kB 00:00 2026-02-12T06:37:47.930 INFO:teuthology.orchestra.run.trial028.stdout:(5/12): libubsan-11.5.0-14.el9.x86_64.rpm 17 MB/s | 176 kB 00:00 2026-02-12T06:37:47.941 INFO:teuthology.orchestra.run.trial028.stdout:(6/12): libxslt-1.1.34-12.el9.x86_64.rpm 22 MB/s | 233 kB 00:00 2026-02-12T06:37:47.968 INFO:teuthology.orchestra.run.trial028.stdout:(7/12): protobuf-3.14.0-17.el9.x86_64.rpm 36 MB/s | 1.0 MB 00:00 2026-02-12T06:37:47.979 INFO:teuthology.orchestra.run.trial028.stdout:(8/12): socat-1.7.4.1-8.el9.x86_64.rpm 29 MB/s | 303 kB 00:00 2026-02-12T06:37:47.988 INFO:teuthology.orchestra.run.trial028.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 7.0 MB/s | 64 kB 00:00 2026-02-12T06:37:48.125 INFO:teuthology.orchestra.run.trial028.stdout:(10/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 143 kB/s | 94 kB 00:00 2026-02-12T06:37:48.193 INFO:teuthology.orchestra.run.trial028.stdout:(11/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 595 kB/s | 122 kB 00:00 2026-02-12T06:37:50.242 INFO:teuthology.orchestra.run.trial028.stdout:(12/12): ceph-test-20.3.0-5145.g5fdb7d8f.el9.x8 37 MB/s | 126 MB 00:03 2026-02-12T06:37:50.243 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-12T06:37:50.243 INFO:teuthology.orchestra.run.trial028.stdout:Total 36 MB/s | 131 MB 00:03 2026-02-12T06:37:50.327 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction check 2026-02-12T06:37:50.339 INFO:teuthology.orchestra.run.trial028.stdout:Transaction check succeeded. 2026-02-12T06:37:50.339 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction test 2026-02-12T06:37:50.763 INFO:teuthology.orchestra.run.trial028.stdout:Transaction test succeeded. 2026-02-12T06:37:50.763 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction 2026-02-12T06:37:51.217 INFO:teuthology.orchestra.run.trial028.stdout: Preparing : 1/1 2026-02-12T06:37:51.225 INFO:teuthology.orchestra.run.trial028.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-12T06:37:51.234 INFO:teuthology.orchestra.run.trial028.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-12T06:37:51.239 INFO:teuthology.orchestra.run.trial028.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-12T06:37:51.243 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-12T06:37:51.245 INFO:teuthology.orchestra.run.trial028.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-12T06:37:51.249 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-12T06:37:51.250 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libasan8-15.1.1-2.5.el9.x86_64 7/12 2026-02-12T06:37:51.267 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libasan8-15.1.1-2.5.el9.x86_64 7/12 2026-02-12T06:37:51.269 INFO:teuthology.orchestra.run.trial028.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-12T06:37:51.279 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-12T06:37:51.287 INFO:teuthology.orchestra.run.trial028.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-12T06:37:51.288 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libcephsqlite-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 11/12 2026-02-12T06:37:58.289 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 11/12 2026-02-12T06:37:58.307 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-test-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 12/12 2026-02-12T06:37:59.083 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-test-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 12/12 2026-02-12T06:37:59.083 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-test-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/12 2026-02-12T06:37:59.084 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libcephsqlite-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2/12 2026-02-12T06:37:59.084 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-12T06:37:59.084 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-12T06:37:59.084 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-12T06:37:59.084 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libasan8-15.1.1-2.5.el9.x86_64 6/12 2026-02-12T06:37:59.084 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-12T06:37:59.084 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-12T06:37:59.084 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-12T06:37:59.084 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-12T06:37:59.084 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-12T06:37:59.137 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-12T06:37:59.137 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:37:59.137 INFO:teuthology.orchestra.run.trial028.stdout:Installed: 2026-02-12T06:37:59.137 INFO:teuthology.orchestra.run.trial028.stdout: ceph-test-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:59.137 INFO:teuthology.orchestra.run.trial028.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-12T06:37:59.137 INFO:teuthology.orchestra.run.trial028.stdout: libasan8-15.1.1-2.5.el9.x86_64 2026-02-12T06:37:59.137 INFO:teuthology.orchestra.run.trial028.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-12T06:37:59.137 INFO:teuthology.orchestra.run.trial028.stdout: libcephsqlite-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:37:59.137 INFO:teuthology.orchestra.run.trial028.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-12T06:37:59.138 INFO:teuthology.orchestra.run.trial028.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-12T06:37:59.138 INFO:teuthology.orchestra.run.trial028.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-12T06:37:59.138 INFO:teuthology.orchestra.run.trial028.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-12T06:37:59.138 INFO:teuthology.orchestra.run.trial028.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-12T06:37:59.138 INFO:teuthology.orchestra.run.trial028.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-12T06:37:59.138 INFO:teuthology.orchestra.run.trial028.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-12T06:37:59.138 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:37:59.138 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:37:59.180 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph 2026-02-12T06:37:59.478 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:26 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:37:59.564 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:37:59.567 INFO:teuthology.orchestra.run.trial028.stdout:======================================================================================= 2026-02-12T06:37:59.567 INFO:teuthology.orchestra.run.trial028.stdout: Package Arch Version Repository Size 2026-02-12T06:37:59.567 INFO:teuthology.orchestra.run.trial028.stdout:======================================================================================= 2026-02-12T06:37:59.567 INFO:teuthology.orchestra.run.trial028.stdout:Installing: 2026-02-12T06:37:59.567 INFO:teuthology.orchestra.run.trial028.stdout: ceph x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 6.5 k 2026-02-12T06:37:59.567 INFO:teuthology.orchestra.run.trial028.stdout:Installing dependencies: 2026-02-12T06:37:59.567 INFO:teuthology.orchestra.run.trial028.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-12T06:37:59.567 INFO:teuthology.orchestra.run.trial028.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 57 k 2026-02-12T06:37:59.567 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mds x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 2.0 M 2026-02-12T06:37:59.567 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 937 k 2026-02-12T06:37:59.568 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 297 k 2026-02-12T06:37:59.568 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mon x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 4.8 M 2026-02-12T06:37:59.568 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 163 k 2026-02-12T06:37:59.568 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-classic x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 15 M 2026-02-12T06:37:59.568 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-crimson x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 87 M 2026-02-12T06:37:59.568 INFO:teuthology.orchestra.run.trial028.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 17 k 2026-02-12T06:37:59.568 INFO:teuthology.orchestra.run.trial028.stdout: cephadm noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 1.0 M 2026-02-12T06:37:59.568 INFO:teuthology.orchestra.run.trial028.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-12T06:37:59.568 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2026-02-12T06:37:59.568 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2026-02-12T06:37:59.568 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2026-02-12T06:37:59.568 INFO:teuthology.orchestra.run.trial028.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-12T06:37:59.569 INFO:teuthology.orchestra.run.trial028.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-12T06:37:59.569 INFO:teuthology.orchestra.run.trial028.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-12T06:37:59.569 INFO:teuthology.orchestra.run.trial028.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-12T06:37:59.569 INFO:teuthology.orchestra.run.trial028.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-12T06:37:59.569 INFO:teuthology.orchestra.run.trial028.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-12T06:37:59.569 INFO:teuthology.orchestra.run.trial028.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-12T06:37:59.569 INFO:teuthology.orchestra.run.trial028.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-12T06:37:59.569 INFO:teuthology.orchestra.run.trial028.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-12T06:37:59.569 INFO:teuthology.orchestra.run.trial028.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-12T06:37:59.569 INFO:teuthology.orchestra.run.trial028.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-12T06:37:59.569 INFO:teuthology.orchestra.run.trial028.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-12T06:37:59.569 INFO:teuthology.orchestra.run.trial028.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-12T06:37:59.570 INFO:teuthology.orchestra.run.trial028.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-12T06:37:59.570 INFO:teuthology.orchestra.run.trial028.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-12T06:37:59.570 INFO:teuthology.orchestra.run.trial028.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-12T06:37:59.570 INFO:teuthology.orchestra.run.trial028.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-12T06:37:59.570 INFO:teuthology.orchestra.run.trial028.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-12T06:37:59.570 INFO:teuthology.orchestra.run.trial028.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-12T06:37:59.570 INFO:teuthology.orchestra.run.trial028.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-12T06:37:59.570 INFO:teuthology.orchestra.run.trial028.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-12T06:37:59.570 INFO:teuthology.orchestra.run.trial028.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-12T06:37:59.570 INFO:teuthology.orchestra.run.trial028.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-12T06:37:59.570 INFO:teuthology.orchestra.run.trial028.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-12T06:37:59.570 INFO:teuthology.orchestra.run.trial028.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-12T06:37:59.570 INFO:teuthology.orchestra.run.trial028.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-12T06:37:59.571 INFO:teuthology.orchestra.run.trial028.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-12T06:37:59.571 INFO:teuthology.orchestra.run.trial028.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-12T06:37:59.571 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-12T06:37:59.571 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-12T06:37:59.571 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-12T06:37:59.571 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-12T06:37:59.571 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-12T06:37:59.571 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-12T06:37:59.571 INFO:teuthology.orchestra.run.trial028.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-12T06:37:59.571 INFO:teuthology.orchestra.run.trial028.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-12T06:37:59.571 INFO:teuthology.orchestra.run.trial028.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-12T06:37:59.571 INFO:teuthology.orchestra.run.trial028.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-12T06:37:59.571 INFO:teuthology.orchestra.run.trial028.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-12T06:37:59.572 INFO:teuthology.orchestra.run.trial028.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-12T06:37:59.572 INFO:teuthology.orchestra.run.trial028.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-12T06:37:59.572 INFO:teuthology.orchestra.run.trial028.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-12T06:37:59.572 INFO:teuthology.orchestra.run.trial028.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-12T06:37:59.572 INFO:teuthology.orchestra.run.trial028.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-12T06:37:59.572 INFO:teuthology.orchestra.run.trial028.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-12T06:37:59.572 INFO:teuthology.orchestra.run.trial028.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-12T06:37:59.572 INFO:teuthology.orchestra.run.trial028.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-12T06:37:59.572 INFO:teuthology.orchestra.run.trial028.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-12T06:37:59.572 INFO:teuthology.orchestra.run.trial028.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-12T06:37:59.572 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-12T06:37:59.572 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-12T06:37:59.572 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-12T06:37:59.573 INFO:teuthology.orchestra.run.trial028.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-12T06:37:59.573 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-12T06:37:59.573 INFO:teuthology.orchestra.run.trial028.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-12T06:37:59.573 INFO:teuthology.orchestra.run.trial028.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-12T06:37:59.573 INFO:teuthology.orchestra.run.trial028.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-12T06:37:59.573 INFO:teuthology.orchestra.run.trial028.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-12T06:37:59.573 INFO:teuthology.orchestra.run.trial028.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-12T06:37:59.573 INFO:teuthology.orchestra.run.trial028.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-12T06:37:59.573 INFO:teuthology.orchestra.run.trial028.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-12T06:37:59.573 INFO:teuthology.orchestra.run.trial028.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-12T06:37:59.573 INFO:teuthology.orchestra.run.trial028.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-12T06:37:59.573 INFO:teuthology.orchestra.run.trial028.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-12T06:37:59.574 INFO:teuthology.orchestra.run.trial028.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-12T06:37:59.574 INFO:teuthology.orchestra.run.trial028.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-12T06:37:59.574 INFO:teuthology.orchestra.run.trial028.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-12T06:37:59.574 INFO:teuthology.orchestra.run.trial028.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-12T06:37:59.574 INFO:teuthology.orchestra.run.trial028.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-12T06:37:59.574 INFO:teuthology.orchestra.run.trial028.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-12T06:37:59.574 INFO:teuthology.orchestra.run.trial028.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-12T06:37:59.574 INFO:teuthology.orchestra.run.trial028.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-12T06:37:59.574 INFO:teuthology.orchestra.run.trial028.stdout:Installing weak dependencies: 2026-02-12T06:37:59.574 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 183 k 2026-02-12T06:37:59.574 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 15 M 2026-02-12T06:37:59.574 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 7.4 M 2026-02-12T06:37:59.575 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 22 k 2026-02-12T06:37:59.575 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-rook noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 50 k 2026-02-12T06:37:59.575 INFO:teuthology.orchestra.run.trial028.stdout: ceph-volume noarch 2:20.3.0-5145.g5fdb7d8f.el9 ceph-noarch 299 k 2026-02-12T06:37:59.575 INFO:teuthology.orchestra.run.trial028.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-12T06:37:59.575 INFO:teuthology.orchestra.run.trial028.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-12T06:37:59.575 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:37:59.575 INFO:teuthology.orchestra.run.trial028.stdout:Transaction Summary 2026-02-12T06:37:59.575 INFO:teuthology.orchestra.run.trial028.stdout:======================================================================================= 2026-02-12T06:37:59.575 INFO:teuthology.orchestra.run.trial028.stdout:Install 96 Packages 2026-02-12T06:37:59.575 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:37:59.576 INFO:teuthology.orchestra.run.trial028.stdout:Total download size: 189 M 2026-02-12T06:37:59.576 INFO:teuthology.orchestra.run.trial028.stdout:Installed size: 1.4 G 2026-02-12T06:37:59.576 INFO:teuthology.orchestra.run.trial028.stdout:Downloading Packages: 2026-02-12T06:38:00.609 INFO:teuthology.orchestra.run.trial028.stdout:(1/96): ceph-20.3.0-5145.g5fdb7d8f.el9.x86_64.r 26 kB/s | 6.5 kB 00:00 2026-02-12T06:38:01.551 INFO:teuthology.orchestra.run.trial028.stdout:(2/96): ceph-mgr-20.3.0-5145.g5fdb7d8f.el9.x86_ 788 kB/s | 937 kB 00:01 2026-02-12T06:38:01.665 INFO:teuthology.orchestra.run.trial028.stdout:(3/96): ceph-osd-20.3.0-5145.g5fdb7d8f.el9.x86_ 1.4 MB/s | 163 kB 00:00 2026-02-12T06:38:02.116 INFO:teuthology.orchestra.run.trial028.stdout:(4/96): ceph-mds-20.3.0-5145.g5fdb7d8f.el9.x86_ 1.2 MB/s | 2.0 MB 00:01 2026-02-12T06:38:02.511 INFO:teuthology.orchestra.run.trial028.stdout:(5/96): ceph-mon-20.3.0-5145.g5fdb7d8f.el9.x86_ 2.5 MB/s | 4.8 MB 00:01 2026-02-12T06:38:02.568 INFO:teuthology.orchestra.run.trial028.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5145.g5f 1.0 MB/s | 57 kB 00:00 2026-02-12T06:38:02.682 INFO:teuthology.orchestra.run.trial028.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5145.g5fdb7d8f. 1.6 MB/s | 183 kB 00:00 2026-02-12T06:38:04.296 INFO:teuthology.orchestra.run.trial028.stdout:(8/96): ceph-osd-classic-20.3.0-5145.g5fdb7d8f. 5.6 MB/s | 15 MB 00:02 2026-02-12T06:38:04.787 INFO:teuthology.orchestra.run.trial028.stdout:(9/96): ceph-mgr-diskprediction-local-20.3.0-51 15 MB/s | 7.4 MB 00:00 2026-02-12T06:38:04.830 INFO:teuthology.orchestra.run.trial028.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5145.g5fdb7d 518 kB/s | 22 kB 00:00 2026-02-12T06:38:04.879 INFO:teuthology.orchestra.run.trial028.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5145.g5fd 5.9 MB/s | 297 kB 00:00 2026-02-12T06:38:04.925 INFO:teuthology.orchestra.run.trial028.stdout:(12/96): ceph-mgr-rook-20.3.0-5145.g5fdb7d8f.el 1.1 MB/s | 50 kB 00:00 2026-02-12T06:38:04.969 INFO:teuthology.orchestra.run.trial028.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5145.g5f 399 kB/s | 17 kB 00:00 2026-02-12T06:38:05.018 INFO:teuthology.orchestra.run.trial028.stdout:(14/96): ceph-volume-20.3.0-5145.g5fdb7d8f.el9. 6.0 MB/s | 299 kB 00:00 2026-02-12T06:38:05.082 INFO:teuthology.orchestra.run.trial028.stdout:(15/96): cephadm-20.3.0-5145.g5fdb7d8f.el9.noar 16 MB/s | 1.0 MB 00:00 2026-02-12T06:38:05.197 INFO:teuthology.orchestra.run.trial028.stdout:(16/96): cryptsetup-2.8.1-2.el9.x86_64.rpm 3.0 MB/s | 350 kB 00:00 2026-02-12T06:38:05.218 INFO:teuthology.orchestra.run.trial028.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.9 MB/s | 40 kB 00:00 2026-02-12T06:38:05.238 INFO:teuthology.orchestra.run.trial028.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 3.5 MB/s | 72 kB 00:00 2026-02-12T06:38:05.279 INFO:teuthology.orchestra.run.trial028.stdout:(19/96): libgfortran-11.5.0-14.el9.x86_64.rpm 19 MB/s | 794 kB 00:00 2026-02-12T06:38:05.302 INFO:teuthology.orchestra.run.trial028.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 8.0 MB/s | 184 kB 00:00 2026-02-12T06:38:05.325 INFO:teuthology.orchestra.run.trial028.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 11 MB/s | 253 kB 00:00 2026-02-12T06:38:05.387 INFO:teuthology.orchestra.run.trial028.stdout:(22/96): python3-cryptography-36.0.1-5.el9.x86_ 20 MB/s | 1.2 MB 00:00 2026-02-12T06:38:05.409 INFO:teuthology.orchestra.run.trial028.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 4.9 MB/s | 106 kB 00:00 2026-02-12T06:38:05.430 INFO:teuthology.orchestra.run.trial028.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 6.1 MB/s | 135 kB 00:00 2026-02-12T06:38:05.453 INFO:teuthology.orchestra.run.trial028.stdout:(25/96): python3-pyparsing-2.4.7-9.el9.noarch.r 6.7 MB/s | 150 kB 00:00 2026-02-12T06:38:05.474 INFO:teuthology.orchestra.run.trial028.stdout:(26/96): python3-requests-2.25.1-10.el9.noarch. 5.7 MB/s | 126 kB 00:00 2026-02-12T06:38:05.497 INFO:teuthology.orchestra.run.trial028.stdout:(27/96): python3-urllib3-1.26.5-7.el9.noarch.rp 9.5 MB/s | 218 kB 00:00 2026-02-12T06:38:05.520 INFO:teuthology.orchestra.run.trial028.stdout:(28/96): unzip-6.0-59.el9.x86_64.rpm 7.9 MB/s | 182 kB 00:00 2026-02-12T06:38:05.541 INFO:teuthology.orchestra.run.trial028.stdout:(29/96): zip-3.0-35.el9.x86_64.rpm 12 MB/s | 266 kB 00:00 2026-02-12T06:38:05.604 INFO:teuthology.orchestra.run.trial028.stdout:(30/96): flexiblas-3.0.4-8.el9.x86_64.rpm 523 kB/s | 33 kB 00:00 2026-02-12T06:38:05.733 INFO:teuthology.orchestra.run.trial028.stdout:(31/96): flexiblas-netlib-3.0.4-8.el9.x86_64.rp 23 MB/s | 3.0 MB 00:00 2026-02-12T06:38:05.754 INFO:teuthology.orchestra.run.trial028.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-8.el9. 853 kB/s | 18 kB 00:00 2026-02-12T06:38:05.778 INFO:teuthology.orchestra.run.trial028.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 10 MB/s | 246 kB 00:00 2026-02-12T06:38:05.801 INFO:teuthology.orchestra.run.trial028.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 7.9 MB/s | 188 kB 00:00 2026-02-12T06:38:05.823 INFO:teuthology.orchestra.run.trial028.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 1.9 MB/s | 42 kB 00:00 2026-02-12T06:38:05.874 INFO:teuthology.orchestra.run.trial028.stdout:(36/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 102 MB/s | 5.3 MB 00:00 2026-02-12T06:38:05.930 INFO:teuthology.orchestra.run.trial028.stdout:(37/96): python3-babel-2.9.1-2.el9.noarch.rpm 108 MB/s | 6.0 MB 00:00 2026-02-12T06:38:05.955 INFO:teuthology.orchestra.run.trial028.stdout:(38/96): python3-devel-3.9.25-2.el9.x86_64.rpm 9.7 MB/s | 244 kB 00:00 2026-02-12T06:38:05.979 INFO:teuthology.orchestra.run.trial028.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 10 MB/s | 249 kB 00:00 2026-02-12T06:38:06.001 INFO:teuthology.orchestra.run.trial028.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 2.1 MB/s | 48 kB 00:00 2026-02-12T06:38:06.024 INFO:teuthology.orchestra.run.trial028.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 7.5 MB/s | 177 kB 00:00 2026-02-12T06:38:06.051 INFO:teuthology.orchestra.run.trial028.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 46 MB/s | 1.2 MB 00:00 2026-02-12T06:38:06.073 INFO:teuthology.orchestra.run.trial028.stdout:(43/96): python3-markupsafe-1.1.1-12.el9.x86_64 1.5 MB/s | 35 kB 00:00 2026-02-12T06:38:06.128 INFO:teuthology.orchestra.run.trial028.stdout:(44/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 113 MB/s | 6.1 MB 00:00 2026-02-12T06:38:06.153 INFO:teuthology.orchestra.run.trial028.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 17 MB/s | 442 kB 00:00 2026-02-12T06:38:06.176 INFO:teuthology.orchestra.run.trial028.stdout:(46/96): python3-packaging-20.9-5.el9.noarch.rp 3.3 MB/s | 77 kB 00:00 2026-02-12T06:38:06.199 INFO:teuthology.orchestra.run.trial028.stdout:(47/96): python3-protobuf-3.14.0-17.el9.noarch. 11 MB/s | 267 kB 00:00 2026-02-12T06:38:06.223 INFO:teuthology.orchestra.run.trial028.stdout:(48/96): python3-pyasn1-0.4.8-6.el9.noarch.rpm 6.5 MB/s | 159 kB 00:00 2026-02-12T06:38:06.247 INFO:teuthology.orchestra.run.trial028.stdout:(49/96): python3-pyasn1-modules-0.4.8-6.el9.noa 11 MB/s | 279 kB 00:00 2026-02-12T06:38:06.269 INFO:teuthology.orchestra.run.trial028.stdout:(50/96): python3-requests-oauthlib-1.3.0-12.el9 2.4 MB/s | 54 kB 00:00 2026-02-12T06:38:06.420 INFO:teuthology.orchestra.run.trial028.stdout:(51/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 128 MB/s | 19 MB 00:00 2026-02-12T06:38:06.443 INFO:teuthology.orchestra.run.trial028.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 1.8 MB/s | 42 kB 00:00 2026-02-12T06:38:06.465 INFO:teuthology.orchestra.run.trial028.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 8.1 MB/s | 189 kB 00:00 2026-02-12T06:38:06.488 INFO:teuthology.orchestra.run.trial028.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 3.9 MB/s | 90 kB 00:00 2026-02-12T06:38:06.919 INFO:teuthology.orchestra.run.trial028.stdout:(55/96): lua-devel-5.4.4-4.el9.x86_64.rpm 52 kB/s | 22 kB 00:00 2026-02-12T06:38:07.823 INFO:teuthology.orchestra.run.trial028.stdout:(56/96): ceph-mgr-dashboard-20.3.0-5145.g5fdb7d 2.9 MB/s | 15 MB 00:05 2026-02-12T06:38:08.002 INFO:teuthology.orchestra.run.trial028.stdout:(57/96): protobuf-compiler-3.14.0-17.el9.x86_64 796 kB/s | 862 kB 00:01 2026-02-12T06:38:08.390 INFO:teuthology.orchestra.run.trial028.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 50 kB/s | 19 kB 00:00 2026-02-12T06:38:08.666 INFO:teuthology.orchestra.run.trial028.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 548 kB/s | 151 kB 00:00 2026-02-12T06:38:08.835 INFO:teuthology.orchestra.run.trial028.stdout:(60/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 545 kB/s | 551 kB 00:01 2026-02-12T06:38:08.923 INFO:teuthology.orchestra.run.trial028.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 334 kB/s | 29 kB 00:00 2026-02-12T06:38:09.002 INFO:teuthology.orchestra.run.trial028.stdout:(62/96): python3-backports-tarfile-1.2.0-1.el9. 769 kB/s | 60 kB 00:00 2026-02-12T06:38:09.087 INFO:teuthology.orchestra.run.trial028.stdout:(63/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 511 kB/s | 43 kB 00:00 2026-02-12T06:38:09.164 INFO:teuthology.orchestra.run.trial028.stdout:(64/96): python3-cachetools-4.2.4-1.el9.noarch. 416 kB/s | 32 kB 00:00 2026-02-12T06:38:09.242 INFO:teuthology.orchestra.run.trial028.stdout:(65/96): python3-certifi-2023.05.07-4.el9.noarc 182 kB/s | 14 kB 00:00 2026-02-12T06:38:09.288 INFO:teuthology.orchestra.run.trial028.stdout:(66/96): python3-asyncssh-2.13.2-5.el9.noarch.r 881 kB/s | 548 kB 00:00 2026-02-12T06:38:09.483 INFO:teuthology.orchestra.run.trial028.stdout:(67/96): python3-cheroot-10.0.1-4.el9.noarch.rp 719 kB/s | 173 kB 00:00 2026-02-12T06:38:09.752 INFO:teuthology.orchestra.run.trial028.stdout:(68/96): python3-cherrypy-18.6.1-2.el9.noarch.r 773 kB/s | 358 kB 00:00 2026-02-12T06:38:09.790 INFO:teuthology.orchestra.run.trial028.stdout:(69/96): python3-google-auth-2.45.0-1.el9.noarc 829 kB/s | 254 kB 00:00 2026-02-12T06:38:10.027 INFO:teuthology.orchestra.run.trial028.stdout:(70/96): python3-grpcio-tools-1.46.7-10.el9.x86 609 kB/s | 144 kB 00:00 2026-02-12T06:38:10.192 INFO:teuthology.orchestra.run.trial028.stdout:(71/96): python3-influxdb-5.3.1-1.el9.noarch.rp 842 kB/s | 139 kB 00:00 2026-02-12T06:38:10.281 INFO:teuthology.orchestra.run.trial028.stdout:(72/96): python3-isodate-0.6.1-3.el9.noarch.rpm 632 kB/s | 56 kB 00:00 2026-02-12T06:38:10.366 INFO:teuthology.orchestra.run.trial028.stdout:(73/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 126 kB/s | 11 kB 00:00 2026-02-12T06:38:10.453 INFO:teuthology.orchestra.run.trial028.stdout:(74/96): python3-jaraco-classes-3.2.1-5.el9.noa 203 kB/s | 18 kB 00:00 2026-02-12T06:38:10.534 INFO:teuthology.orchestra.run.trial028.stdout:(75/96): python3-jaraco-collections-3.0.0-8.el9 287 kB/s | 23 kB 00:00 2026-02-12T06:38:10.619 INFO:teuthology.orchestra.run.trial028.stdout:(76/96): python3-jaraco-context-6.0.1-3.el9.noa 230 kB/s | 20 kB 00:00 2026-02-12T06:38:10.697 INFO:teuthology.orchestra.run.trial028.stdout:(77/96): python3-jaraco-functools-3.5.0-2.el9.n 251 kB/s | 19 kB 00:00 2026-02-12T06:38:10.777 INFO:teuthology.orchestra.run.trial028.stdout:(78/96): python3-jaraco-text-4.0.0-2.el9.noarch 328 kB/s | 26 kB 00:00 2026-02-12T06:38:11.926 INFO:teuthology.orchestra.run.trial028.stdout:(79/96): python3-kubernetes-26.1.0-3.el9.noarch 909 kB/s | 1.0 MB 00:01 2026-02-12T06:38:12.044 INFO:teuthology.orchestra.run.trial028.stdout:(80/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 910 kB/s | 2.0 MB 00:02 2026-02-12T06:38:12.088 INFO:teuthology.orchestra.run.trial028.stdout:(81/96): python3-more-itertools-8.12.0-2.el9.no 487 kB/s | 79 kB 00:00 2026-02-12T06:38:12.171 INFO:teuthology.orchestra.run.trial028.stdout:(82/96): python3-natsort-7.1.1-5.el9.noarch.rpm 700 kB/s | 58 kB 00:00 2026-02-12T06:38:12.204 INFO:teuthology.orchestra.run.trial028.stdout:(83/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 540 kB/s | 86 kB 00:00 2026-02-12T06:38:12.249 INFO:teuthology.orchestra.run.trial028.stdout:(84/96): python3-portend-3.1.0-2.el9.noarch.rpm 212 kB/s | 16 kB 00:00 2026-02-12T06:38:12.335 INFO:teuthology.orchestra.run.trial028.stdout:(85/96): python3-repoze-lru-0.7-16.el9.noarch.r 356 kB/s | 31 kB 00:00 2026-02-12T06:38:12.357 INFO:teuthology.orchestra.run.trial028.stdout:(86/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 589 kB/s | 90 kB 00:00 2026-02-12T06:38:12.434 INFO:teuthology.orchestra.run.trial028.stdout:(87/96): python3-rsa-4.9-2.el9.noarch.rpm 770 kB/s | 59 kB 00:00 2026-02-12T06:38:12.573 INFO:teuthology.orchestra.run.trial028.stdout:(88/96): python3-routes-2.5.1-5.el9.noarch.rpm 791 kB/s | 188 kB 00:00 2026-02-12T06:38:12.594 INFO:teuthology.orchestra.run.trial028.stdout:(89/96): python3-saml-1.16.0-1.el9.noarch.rpm 779 kB/s | 125 kB 00:00 2026-02-12T06:38:12.658 INFO:teuthology.orchestra.run.trial028.stdout:(90/96): python3-tempora-5.0.0-2.el9.noarch.rpm 420 kB/s | 36 kB 00:00 2026-02-12T06:38:12.755 INFO:teuthology.orchestra.run.trial028.stdout:(91/96): python3-typing-extensions-4.15.0-1.el9 537 kB/s | 86 kB 00:00 2026-02-12T06:38:12.812 INFO:teuthology.orchestra.run.trial028.stdout:(92/96): python3-websocket-client-1.2.3-2.el9.n 583 kB/s | 90 kB 00:00 2026-02-12T06:38:12.851 INFO:teuthology.orchestra.run.trial028.stdout:(93/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 505 kB/s | 48 kB 00:00 2026-02-12T06:38:12.890 INFO:teuthology.orchestra.run.trial028.stdout:(94/96): python3-xmltodict-0.12.0-15.el9.noarch 284 kB/s | 22 kB 00:00 2026-02-12T06:38:12.937 INFO:teuthology.orchestra.run.trial028.stdout:(95/96): python3-zc-lockfile-2.0-10.el9.noarch. 232 kB/s | 20 kB 00:00 2026-02-12T06:38:31.459 INFO:teuthology.orchestra.run.trial028.stdout:(96/96): ceph-osd-crimson-20.3.0-5145.g5fdb7d8f 3.0 MB/s | 87 MB 00:29 2026-02-12T06:38:31.462 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-12T06:38:31.462 INFO:teuthology.orchestra.run.trial028.stdout:Total 5.9 MB/s | 189 MB 00:31 2026-02-12T06:38:31.857 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction check 2026-02-12T06:38:31.894 INFO:teuthology.orchestra.run.trial028.stdout:Transaction check succeeded. 2026-02-12T06:38:31.895 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction test 2026-02-12T06:38:32.584 INFO:teuthology.orchestra.run.trial028.stdout:Transaction test succeeded. 2026-02-12T06:38:32.584 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction 2026-02-12T06:38:33.307 INFO:teuthology.orchestra.run.trial028.stdout: Preparing : 1/1 2026-02-12T06:38:33.309 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-12T06:38:33.313 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-12T06:38:33.334 INFO:teuthology.orchestra.run.trial028.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-12T06:38:33.367 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 4/96 2026-02-12T06:38:33.374 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-12T06:38:33.376 INFO:teuthology.orchestra.run.trial028.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-12T06:38:33.381 INFO:teuthology.orchestra.run.trial028.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 7/96 2026-02-12T06:38:33.404 INFO:teuthology.orchestra.run.trial028.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-12T06:38:33.417 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-12T06:38:33.420 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-12T06:38:33.426 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-12T06:38:33.428 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-12T06:38:33.439 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-12T06:38:33.445 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-12T06:38:33.469 INFO:teuthology.orchestra.run.trial028.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-12T06:38:33.472 INFO:teuthology.orchestra.run.trial028.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-12T06:38:33.522 INFO:teuthology.orchestra.run.trial028.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-12T06:38:33.535 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 18/96 2026-02-12T06:38:33.537 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-12T06:38:33.542 INFO:teuthology.orchestra.run.trial028.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-12T06:38:33.547 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-12T06:38:33.552 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-12T06:38:33.555 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-12T06:38:33.569 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-12T06:38:33.573 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-12T06:38:33.578 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-12T06:38:33.590 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-12T06:38:33.599 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-12T06:38:33.605 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-12T06:38:33.625 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-12T06:38:33.635 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-12T06:38:33.651 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-12T06:38:33.656 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-12T06:38:33.663 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-12T06:38:33.703 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-12T06:38:34.026 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-12T06:38:34.036 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-12T06:38:34.040 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-12T06:38:34.045 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-12T06:38:34.047 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-12T06:38:34.074 INFO:teuthology.orchestra.run.trial028.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-12T06:38:34.117 INFO:teuthology.orchestra.run.trial028.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-12T06:38:34.126 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-12T06:38:34.131 INFO:teuthology.orchestra.run.trial028.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-12T06:38:34.134 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-12T06:38:34.140 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-12T06:38:34.144 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-12T06:38:34.151 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-12T06:38:34.154 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-12T06:38:34.182 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-12T06:38:34.192 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-12T06:38:34.198 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-12T06:38:34.204 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-12T06:38:34.239 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-12T06:38:34.462 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 55/96 2026-02-12T06:38:34.485 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-12T06:38:34.489 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-12T06:38:34.547 INFO:teuthology.orchestra.run.trial028.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-12T06:38:34.549 INFO:teuthology.orchestra.run.trial028.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-12T06:38:34.570 INFO:teuthology.orchestra.run.trial028.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 60/96 2026-02-12T06:38:34.894 INFO:teuthology.orchestra.run.trial028.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 61/96 2026-02-12T06:38:34.967 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-12T06:38:35.636 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-12T06:38:35.653 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-12T06:38:35.663 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-12T06:38:35.670 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-12T06:38:35.683 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-12T06:38:35.699 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-12T06:38:35.774 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-12T06:38:35.783 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-12T06:38:35.806 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-12T06:38:35.836 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-12T06:38:35.885 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-12T06:38:35.891 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-12T06:38:35.895 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-12T06:38:35.896 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-12T06:38:35.913 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-12T06:38:35.913 INFO:teuthology.orchestra.run.trial028.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-12T06:38:35.913 INFO:teuthology.orchestra.run.trial028.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-12T06:38:35.913 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:35.920 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-12T06:38:35.945 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-12T06:38:35.945 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-12T06:38:35.946 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:35.979 INFO:teuthology.orchestra.run.trial028.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-12T06:38:36.598 INFO:teuthology.orchestra.run.trial028.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 79/96 2026-02-12T06:38:42.058 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-osd-classic-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 80/96 2026-02-12T06:38:42.120 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-osd-crimson-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 81/96 2026-02-12T06:38:42.122 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-volume-2:20.3.0-5145.g5fdb7d8f.el9.noarch 82/96 2026-02-12T06:38:42.136 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-volume-2:20.3.0-5145.g5fdb7d8f.el9.noarch 82/96 2026-02-12T06:38:42.137 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T06:38:42.137 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-12T06:38:42.137 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:42.147 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-osd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 83/96 2026-02-12T06:38:42.169 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-osd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 83/96 2026-02-12T06:38:42.170 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T06:38:42.170 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-12T06:38:42.170 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-12T06:38:42.170 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-12T06:38:42.170 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:42.230 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarch 84/96 2026-02-12T06:38:42.231 INFO:teuthology.orchestra.run.trial028.stdout: Installing : cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarch 84/96 2026-02-12T06:38:42.240 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5145.g5fdb7d8f.el9 85/96 2026-02-12T06:38:42.264 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5145.g5fdb7d8f.el 86/96 2026-02-12T06:38:42.266 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarc 87/96 2026-02-12T06:38:43.854 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarc 87/96 2026-02-12T06:38:43.861 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5145.g5fdb7d8f.el9.noa 88/96 2026-02-12T06:38:44.327 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5145.g5fdb7d8f.el9.noa 88/96 2026-02-12T06:38:44.328 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5145.g5fdb7 89/96 2026-02-12T06:38:44.340 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5145.g5fdb7 89/96 2026-02-12T06:38:44.340 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5145.g5fdb7d8f.el9.noa 90/96 2026-02-12T06:38:44.402 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5145.g5fdb7d8f.el9.noa 90/96 2026-02-12T06:38:44.445 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5145.g5fdb7d8f.el9. 91/96 2026-02-12T06:38:44.446 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 92/96 2026-02-12T06:38:44.468 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 92/96 2026-02-12T06:38:44.468 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T06:38:44.468 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-12T06:38:44.468 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-12T06:38:44.468 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-12T06:38:44.468 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:44.479 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mgr-rook-2:20.3.0-5145.g5fdb7d8f.el9.noarch 93/96 2026-02-12T06:38:44.703 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5145.g5fdb7d8f.el9.noarch 93/96 2026-02-12T06:38:44.705 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mon-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 94/96 2026-02-12T06:38:44.727 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mon-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 94/96 2026-02-12T06:38:44.727 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T06:38:44.727 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-12T06:38:44.727 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-12T06:38:44.727 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-12T06:38:44.727 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:44.814 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-mds-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 95/96 2026-02-12T06:38:44.835 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-mds-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 95/96 2026-02-12T06:38:44.835 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T06:38:44.835 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-12T06:38:44.835 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-12T06:38:44.835 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-12T06:38:44.836 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:44.836 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 96/96 2026-02-12T06:38:44.843 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 96/96 2026-02-12T06:38:44.866 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 96/96 2026-02-12T06:38:46.197 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 96/96 2026-02-12T06:38:46.197 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/96 2026-02-12T06:38:46.197 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mds-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2/96 2026-02-12T06:38:46.197 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 3/96 2026-02-12T06:38:46.197 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mon-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 4/96 2026-02-12T06:38:46.198 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-osd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 5/96 2026-02-12T06:38:46.198 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-osd-classic-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 6/96 2026-02-12T06:38:46.198 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5145.g5fdb7d8f.el9.x86_6 7/96 2026-02-12T06:38:46.198 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5145.g5fdb7d8f.el 8/96 2026-02-12T06:38:46.198 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarc 9/96 2026-02-12T06:38:46.198 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5145.g5fdb7d8f.el9.noa 10/96 2026-02-12T06:38:46.198 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5145.g5fdb7 11/96 2026-02-12T06:38:46.198 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5145.g5fdb7d8f.el9.noa 12/96 2026-02-12T06:38:46.198 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5145.g5fdb7d8f.el9. 13/96 2026-02-12T06:38:46.198 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5145.g5fdb7d8f.el9.noarch 14/96 2026-02-12T06:38:46.199 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5145.g5fdb7d8f.el9 15/96 2026-02-12T06:38:46.199 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-volume-2:20.3.0-5145.g5fdb7d8f.el9.noarch 16/96 2026-02-12T06:38:46.199 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarch 17/96 2026-02-12T06:38:46.199 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 18/96 2026-02-12T06:38:46.199 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-12T06:38:46.199 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-12T06:38:46.199 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-12T06:38:46.199 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-12T06:38:46.199 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-12T06:38:46.199 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-12T06:38:46.199 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-12T06:38:46.200 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-12T06:38:46.200 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-12T06:38:46.200 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-12T06:38:46.200 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-12T06:38:46.200 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-12T06:38:46.200 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-12T06:38:46.200 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 32/96 2026-02-12T06:38:46.200 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 33/96 2026-02-12T06:38:46.200 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 34/96 2026-02-12T06:38:46.200 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-12T06:38:46.200 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-12T06:38:46.200 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-12T06:38:46.201 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-12T06:38:46.201 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-12T06:38:46.201 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 40/96 2026-02-12T06:38:46.201 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-12T06:38:46.201 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-12T06:38:46.201 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-12T06:38:46.201 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-12T06:38:46.201 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-12T06:38:46.201 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-12T06:38:46.201 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-12T06:38:46.201 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-12T06:38:46.201 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-12T06:38:46.202 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 50/96 2026-02-12T06:38:46.202 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 51/96 2026-02-12T06:38:46.202 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-12T06:38:46.202 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-12T06:38:46.202 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-12T06:38:46.202 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-12T06:38:46.202 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-12T06:38:46.202 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-12T06:38:46.202 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-12T06:38:46.202 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-12T06:38:46.202 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-12T06:38:46.202 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-12T06:38:46.202 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-12T06:38:46.203 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-12T06:38:46.203 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-12T06:38:46.203 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-12T06:38:46.203 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-12T06:38:46.203 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-12T06:38:46.203 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-12T06:38:46.203 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-12T06:38:46.203 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-12T06:38:46.203 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-12T06:38:46.203 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-12T06:38:46.203 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-12T06:38:46.203 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-12T06:38:46.203 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-12T06:38:46.204 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-12T06:38:46.204 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-12T06:38:46.204 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-12T06:38:46.204 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-12T06:38:46.204 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-12T06:38:46.204 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-12T06:38:46.204 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-12T06:38:46.204 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-12T06:38:46.204 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-12T06:38:46.204 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-12T06:38:46.204 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-12T06:38:46.204 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-12T06:38:46.204 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-12T06:38:46.205 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-12T06:38:46.205 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-12T06:38:46.205 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-12T06:38:46.205 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-12T06:38:46.205 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-12T06:38:46.205 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-12T06:38:46.205 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-12T06:38:46.260 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-12T06:38:46.260 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:46.260 INFO:teuthology.orchestra.run.trial028.stdout:Installed: 2026-02-12T06:38:46.260 INFO:teuthology.orchestra.run.trial028.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-12T06:38:46.260 INFO:teuthology.orchestra.run.trial028.stdout: ceph-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:38:46.260 INFO:teuthology.orchestra.run.trial028.stdout: ceph-grafana-dashboards-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T06:38:46.260 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mds-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:38:46.260 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:38:46.260 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T06:38:46.260 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-dashboard-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T06:38:46.260 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T06:38:46.261 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-k8sevents-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T06:38:46.261 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-modules-core-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T06:38:46.261 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mgr-rook-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T06:38:46.261 INFO:teuthology.orchestra.run.trial028.stdout: ceph-mon-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:38:46.261 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:38:46.261 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-classic-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:38:46.261 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-crimson-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:38:46.261 INFO:teuthology.orchestra.run.trial028.stdout: ceph-prometheus-alerts-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T06:38:46.261 INFO:teuthology.orchestra.run.trial028.stdout: ceph-volume-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T06:38:46.261 INFO:teuthology.orchestra.run.trial028.stdout: cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarch 2026-02-12T06:38:46.261 INFO:teuthology.orchestra.run.trial028.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-12T06:38:46.261 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas-3.0.4-8.el9.x86_64 2026-02-12T06:38:46.262 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2026-02-12T06:38:46.262 INFO:teuthology.orchestra.run.trial028.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2026-02-12T06:38:46.262 INFO:teuthology.orchestra.run.trial028.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-12T06:38:46.262 INFO:teuthology.orchestra.run.trial028.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-12T06:38:46.262 INFO:teuthology.orchestra.run.trial028.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-12T06:38:46.262 INFO:teuthology.orchestra.run.trial028.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-12T06:38:46.262 INFO:teuthology.orchestra.run.trial028.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-12T06:38:46.262 INFO:teuthology.orchestra.run.trial028.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-12T06:38:46.262 INFO:teuthology.orchestra.run.trial028.stdout: lua-5.4.4-4.el9.x86_64 2026-02-12T06:38:46.262 INFO:teuthology.orchestra.run.trial028.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-12T06:38:46.262 INFO:teuthology.orchestra.run.trial028.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-12T06:38:46.262 INFO:teuthology.orchestra.run.trial028.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-12T06:38:46.263 INFO:teuthology.orchestra.run.trial028.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-12T06:38:46.263 INFO:teuthology.orchestra.run.trial028.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-12T06:38:46.263 INFO:teuthology.orchestra.run.trial028.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-12T06:38:46.263 INFO:teuthology.orchestra.run.trial028.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-12T06:38:46.263 INFO:teuthology.orchestra.run.trial028.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-12T06:38:46.263 INFO:teuthology.orchestra.run.trial028.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-12T06:38:46.263 INFO:teuthology.orchestra.run.trial028.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-12T06:38:46.263 INFO:teuthology.orchestra.run.trial028.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-12T06:38:46.263 INFO:teuthology.orchestra.run.trial028.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-12T06:38:46.263 INFO:teuthology.orchestra.run.trial028.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-12T06:38:46.263 INFO:teuthology.orchestra.run.trial028.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-12T06:38:46.263 INFO:teuthology.orchestra.run.trial028.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-12T06:38:46.263 INFO:teuthology.orchestra.run.trial028.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-12T06:38:46.264 INFO:teuthology.orchestra.run.trial028.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-12T06:38:46.264 INFO:teuthology.orchestra.run.trial028.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-12T06:38:46.264 INFO:teuthology.orchestra.run.trial028.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-12T06:38:46.264 INFO:teuthology.orchestra.run.trial028.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-12T06:38:46.264 INFO:teuthology.orchestra.run.trial028.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-12T06:38:46.264 INFO:teuthology.orchestra.run.trial028.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-12T06:38:46.264 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-12T06:38:46.264 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-12T06:38:46.264 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-12T06:38:46.264 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-12T06:38:46.264 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-12T06:38:46.264 INFO:teuthology.orchestra.run.trial028.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-12T06:38:46.264 INFO:teuthology.orchestra.run.trial028.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-12T06:38:46.265 INFO:teuthology.orchestra.run.trial028.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-12T06:38:46.265 INFO:teuthology.orchestra.run.trial028.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-12T06:38:46.265 INFO:teuthology.orchestra.run.trial028.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-12T06:38:46.265 INFO:teuthology.orchestra.run.trial028.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-12T06:38:46.265 INFO:teuthology.orchestra.run.trial028.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-12T06:38:46.265 INFO:teuthology.orchestra.run.trial028.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-12T06:38:46.265 INFO:teuthology.orchestra.run.trial028.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-12T06:38:46.265 INFO:teuthology.orchestra.run.trial028.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-12T06:38:46.265 INFO:teuthology.orchestra.run.trial028.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-12T06:38:46.265 INFO:teuthology.orchestra.run.trial028.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-12T06:38:46.265 INFO:teuthology.orchestra.run.trial028.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-12T06:38:46.265 INFO:teuthology.orchestra.run.trial028.stdout: python3-ply-3.11-14.el9.noarch 2026-02-12T06:38:46.265 INFO:teuthology.orchestra.run.trial028.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-12T06:38:46.266 INFO:teuthology.orchestra.run.trial028.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-12T06:38:46.266 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-12T06:38:46.266 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-12T06:38:46.266 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-12T06:38:46.266 INFO:teuthology.orchestra.run.trial028.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-12T06:38:46.266 INFO:teuthology.orchestra.run.trial028.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-12T06:38:46.266 INFO:teuthology.orchestra.run.trial028.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-12T06:38:46.266 INFO:teuthology.orchestra.run.trial028.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-12T06:38:46.266 INFO:teuthology.orchestra.run.trial028.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-12T06:38:46.266 INFO:teuthology.orchestra.run.trial028.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-12T06:38:46.266 INFO:teuthology.orchestra.run.trial028.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-12T06:38:46.266 INFO:teuthology.orchestra.run.trial028.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-12T06:38:46.266 INFO:teuthology.orchestra.run.trial028.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-12T06:38:46.267 INFO:teuthology.orchestra.run.trial028.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-12T06:38:46.267 INFO:teuthology.orchestra.run.trial028.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-12T06:38:46.267 INFO:teuthology.orchestra.run.trial028.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-12T06:38:46.267 INFO:teuthology.orchestra.run.trial028.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-12T06:38:46.267 INFO:teuthology.orchestra.run.trial028.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-12T06:38:46.267 INFO:teuthology.orchestra.run.trial028.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-12T06:38:46.267 INFO:teuthology.orchestra.run.trial028.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-12T06:38:46.267 INFO:teuthology.orchestra.run.trial028.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-12T06:38:46.267 INFO:teuthology.orchestra.run.trial028.stdout: unzip-6.0-59.el9.x86_64 2026-02-12T06:38:46.267 INFO:teuthology.orchestra.run.trial028.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-12T06:38:46.267 INFO:teuthology.orchestra.run.trial028.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-12T06:38:46.267 INFO:teuthology.orchestra.run.trial028.stdout: zip-3.0-35.el9.x86_64 2026-02-12T06:38:46.267 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:46.268 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:46.327 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-base 2026-02-12T06:38:46.635 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:13 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:46.700 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-base-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:38:46.719 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:46.719 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:46.720 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:46.741 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install cephadm 2026-02-12T06:38:47.037 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:14 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:47.102 INFO:teuthology.orchestra.run.trial028.stdout:Package cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T06:38:47.120 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:47.120 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:47.121 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:47.141 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-immutable-object-cache 2026-02-12T06:38:47.445 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:14 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:47.529 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:47.529 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:47.529 INFO:teuthology.orchestra.run.trial028.stdout: Package Arch Version Repo Size 2026-02-12T06:38:47.530 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:47.530 INFO:teuthology.orchestra.run.trial028.stdout:Installing: 2026-02-12T06:38:47.530 INFO:teuthology.orchestra.run.trial028.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 149 k 2026-02-12T06:38:47.530 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:47.530 INFO:teuthology.orchestra.run.trial028.stdout:Transaction Summary 2026-02-12T06:38:47.530 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:47.530 INFO:teuthology.orchestra.run.trial028.stdout:Install 1 Package 2026-02-12T06:38:47.530 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:47.530 INFO:teuthology.orchestra.run.trial028.stdout:Total download size: 149 k 2026-02-12T06:38:47.530 INFO:teuthology.orchestra.run.trial028.stdout:Installed size: 455 k 2026-02-12T06:38:47.531 INFO:teuthology.orchestra.run.trial028.stdout:Downloading Packages: 2026-02-12T06:38:47.614 INFO:teuthology.orchestra.run.trial028.stdout:ceph-immutable-object-cache-20.3.0-5145.g5fdb7d 1.7 MB/s | 149 kB 00:00 2026-02-12T06:38:47.615 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-12T06:38:47.615 INFO:teuthology.orchestra.run.trial028.stdout:Total 1.7 MB/s | 149 kB 00:00 2026-02-12T06:38:47.616 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction check 2026-02-12T06:38:47.619 INFO:teuthology.orchestra.run.trial028.stdout:Transaction check succeeded. 2026-02-12T06:38:47.620 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction test 2026-02-12T06:38:47.652 INFO:teuthology.orchestra.run.trial028.stdout:Transaction test succeeded. 2026-02-12T06:38:47.652 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction 2026-02-12T06:38:47.707 INFO:teuthology.orchestra.run.trial028.stdout: Preparing : 1/1 2026-02-12T06:38:47.709 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5145.g5fdb7d8f. 1/1 2026-02-12T06:38:47.731 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5145.g5fdb7d8f. 1/1 2026-02-12T06:38:47.731 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T06:38:47.731 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-12T06:38:47.731 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:48.043 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5145.g5fdb7d8f. 1/1 2026-02-12T06:38:48.043 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:48.043 INFO:teuthology.orchestra.run.trial028.stdout:Installed: 2026-02-12T06:38:48.044 INFO:teuthology.orchestra.run.trial028.stdout: ceph-immutable-object-cache-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:38:48.044 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:48.044 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:48.072 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-mgr 2026-02-12T06:38:48.376 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:15 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:48.441 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-mgr-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:38:48.460 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:48.461 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:48.461 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:48.482 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-mgr-dashboard 2026-02-12T06:38:48.782 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:15 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:48.846 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-mgr-dashboard-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T06:38:48.864 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:48.865 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:48.865 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:48.886 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-12T06:38:49.182 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:16 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:49.246 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T06:38:49.265 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:49.265 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:49.265 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:49.286 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-mgr-rook 2026-02-12T06:38:49.584 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:16 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:49.648 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-mgr-rook-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T06:38:49.666 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:49.667 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:49.667 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:49.687 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-mgr-cephadm 2026-02-12T06:38:49.988 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:16 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:50.053 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-mgr-cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T06:38:50.071 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:50.072 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:50.072 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:50.094 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-osd 2026-02-12T06:38:50.396 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:17 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:50.461 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-osd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:38:50.479 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:50.480 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:50.480 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:50.501 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-osd-classic 2026-02-12T06:38:50.802 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:17 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:50.868 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-osd-classic-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:38:50.886 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:50.887 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:50.887 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:50.908 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-fuse 2026-02-12T06:38:51.205 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:18 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:51.289 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:51.290 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:51.290 INFO:teuthology.orchestra.run.trial028.stdout: Package Arch Version Repo Size 2026-02-12T06:38:51.290 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:51.290 INFO:teuthology.orchestra.run.trial028.stdout:Installing: 2026-02-12T06:38:51.290 INFO:teuthology.orchestra.run.trial028.stdout: ceph-fuse x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 914 k 2026-02-12T06:38:51.291 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:51.291 INFO:teuthology.orchestra.run.trial028.stdout:Transaction Summary 2026-02-12T06:38:51.291 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:51.291 INFO:teuthology.orchestra.run.trial028.stdout:Install 1 Package 2026-02-12T06:38:51.291 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:51.291 INFO:teuthology.orchestra.run.trial028.stdout:Total download size: 914 k 2026-02-12T06:38:51.291 INFO:teuthology.orchestra.run.trial028.stdout:Installed size: 2.9 M 2026-02-12T06:38:51.291 INFO:teuthology.orchestra.run.trial028.stdout:Downloading Packages: 2026-02-12T06:38:52.268 INFO:teuthology.orchestra.run.trial028.stdout:ceph-fuse-20.3.0-5145.g5fdb7d8f.el9.x86_64.rpm 935 kB/s | 914 kB 00:00 2026-02-12T06:38:52.269 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-12T06:38:52.269 INFO:teuthology.orchestra.run.trial028.stdout:Total 935 kB/s | 914 kB 00:00 2026-02-12T06:38:52.269 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction check 2026-02-12T06:38:52.274 INFO:teuthology.orchestra.run.trial028.stdout:Transaction check succeeded. 2026-02-12T06:38:52.274 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction test 2026-02-12T06:38:52.308 INFO:teuthology.orchestra.run.trial028.stdout:Transaction test succeeded. 2026-02-12T06:38:52.308 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction 2026-02-12T06:38:52.398 INFO:teuthology.orchestra.run.trial028.stdout: Preparing : 1/1 2026-02-12T06:38:52.415 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:38:52.653 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:38:52.710 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:38:52.710 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:52.710 INFO:teuthology.orchestra.run.trial028.stdout:Installed: 2026-02-12T06:38:52.710 INFO:teuthology.orchestra.run.trial028.stdout: ceph-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:38:52.711 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:52.711 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:52.737 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-volume 2026-02-12T06:38:53.037 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:20 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:53.101 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-volume-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T06:38:53.119 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:53.119 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:53.120 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:53.141 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install librados-devel 2026-02-12T06:38:53.446 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:20 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:53.528 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:53.529 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:53.529 INFO:teuthology.orchestra.run.trial028.stdout: Package Arch Version Repo Size 2026-02-12T06:38:53.530 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:53.530 INFO:teuthology.orchestra.run.trial028.stdout:Installing: 2026-02-12T06:38:53.530 INFO:teuthology.orchestra.run.trial028.stdout: librados-devel x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 124 k 2026-02-12T06:38:53.530 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:53.530 INFO:teuthology.orchestra.run.trial028.stdout:Transaction Summary 2026-02-12T06:38:53.530 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:53.530 INFO:teuthology.orchestra.run.trial028.stdout:Install 1 Package 2026-02-12T06:38:53.530 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:53.530 INFO:teuthology.orchestra.run.trial028.stdout:Total download size: 124 k 2026-02-12T06:38:53.530 INFO:teuthology.orchestra.run.trial028.stdout:Installed size: 434 k 2026-02-12T06:38:53.530 INFO:teuthology.orchestra.run.trial028.stdout:Downloading Packages: 2026-02-12T06:38:53.614 INFO:teuthology.orchestra.run.trial028.stdout:librados-devel-20.3.0-5145.g5fdb7d8f.el9.x86_64 1.4 MB/s | 124 kB 00:00 2026-02-12T06:38:53.615 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-12T06:38:53.615 INFO:teuthology.orchestra.run.trial028.stdout:Total 1.4 MB/s | 124 kB 00:00 2026-02-12T06:38:53.615 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction check 2026-02-12T06:38:53.617 INFO:teuthology.orchestra.run.trial028.stdout:Transaction check succeeded. 2026-02-12T06:38:53.618 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction test 2026-02-12T06:38:53.650 INFO:teuthology.orchestra.run.trial028.stdout:Transaction test succeeded. 2026-02-12T06:38:53.650 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction 2026-02-12T06:38:53.704 INFO:teuthology.orchestra.run.trial028.stdout: Preparing : 1/1 2026-02-12T06:38:53.721 INFO:teuthology.orchestra.run.trial028.stdout: Installing : librados-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:38:53.805 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: librados-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:38:53.862 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : librados-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:38:53.862 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:53.862 INFO:teuthology.orchestra.run.trial028.stdout:Installed: 2026-02-12T06:38:53.862 INFO:teuthology.orchestra.run.trial028.stdout: librados-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:38:53.862 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:53.863 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:53.890 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install libcephfs2 2026-02-12T06:38:54.197 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:21 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:54.263 INFO:teuthology.orchestra.run.trial028.stdout:Package libcephfs2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:38:54.280 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:54.280 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:54.281 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:54.303 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install libcephfs-devel 2026-02-12T06:38:54.605 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:21 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:54.689 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:54.689 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:54.689 INFO:teuthology.orchestra.run.trial028.stdout: Package Arch Version Repo Size 2026-02-12T06:38:54.689 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:54.689 INFO:teuthology.orchestra.run.trial028.stdout:Installing: 2026-02-12T06:38:54.689 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs-devel x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 37 k 2026-02-12T06:38:54.689 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:54.690 INFO:teuthology.orchestra.run.trial028.stdout:Transaction Summary 2026-02-12T06:38:54.690 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:54.690 INFO:teuthology.orchestra.run.trial028.stdout:Install 1 Package 2026-02-12T06:38:54.690 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:54.690 INFO:teuthology.orchestra.run.trial028.stdout:Total download size: 37 k 2026-02-12T06:38:54.690 INFO:teuthology.orchestra.run.trial028.stdout:Installed size: 171 k 2026-02-12T06:38:54.690 INFO:teuthology.orchestra.run.trial028.stdout:Downloading Packages: 2026-02-12T06:38:54.862 INFO:teuthology.orchestra.run.trial028.stdout:libcephfs-devel-20.3.0-5145.g5fdb7d8f.el9.x86_6 216 kB/s | 37 kB 00:00 2026-02-12T06:38:54.862 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-12T06:38:54.862 INFO:teuthology.orchestra.run.trial028.stdout:Total 214 kB/s | 37 kB 00:00 2026-02-12T06:38:54.863 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction check 2026-02-12T06:38:54.865 INFO:teuthology.orchestra.run.trial028.stdout:Transaction check succeeded. 2026-02-12T06:38:54.866 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction test 2026-02-12T06:38:54.868 INFO:teuthology.orchestra.run.trial028.stdout:Transaction test succeeded. 2026-02-12T06:38:54.869 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction 2026-02-12T06:38:54.891 INFO:teuthology.orchestra.run.trial028.stdout: Preparing : 1/1 2026-02-12T06:38:54.907 INFO:teuthology.orchestra.run.trial028.stdout: Installing : libcephfs-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:38:54.962 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:38:55.019 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : libcephfs-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:38:55.019 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:55.019 INFO:teuthology.orchestra.run.trial028.stdout:Installed: 2026-02-12T06:38:55.020 INFO:teuthology.orchestra.run.trial028.stdout: libcephfs-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:38:55.020 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:55.020 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:55.047 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install librados2 2026-02-12T06:38:55.348 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:22 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:55.412 INFO:teuthology.orchestra.run.trial028.stdout:Package librados2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:38:55.429 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:55.430 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:55.430 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:55.451 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install librbd1 2026-02-12T06:38:55.750 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:22 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:55.814 INFO:teuthology.orchestra.run.trial028.stdout:Package librbd1-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:38:55.832 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:55.832 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:55.833 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:55.853 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install python3-rados 2026-02-12T06:38:56.151 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:23 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:56.216 INFO:teuthology.orchestra.run.trial028.stdout:Package python3-rados-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:38:56.233 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:56.234 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:56.234 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:56.255 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install python3-rgw 2026-02-12T06:38:56.556 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:23 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:56.622 INFO:teuthology.orchestra.run.trial028.stdout:Package python3-rgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:38:56.640 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:56.640 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:56.640 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:56.662 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install python3-cephfs 2026-02-12T06:38:56.963 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:23 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:57.029 INFO:teuthology.orchestra.run.trial028.stdout:Package python3-cephfs-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:38:57.047 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:57.048 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:57.048 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:57.069 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install python3-rbd 2026-02-12T06:38:57.367 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:24 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:57.432 INFO:teuthology.orchestra.run.trial028.stdout:Package python3-rbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:38:57.449 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:57.450 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:38:57.450 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:57.471 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install rbd-fuse 2026-02-12T06:38:57.771 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:24 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:57.855 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:57.856 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:57.856 INFO:teuthology.orchestra.run.trial028.stdout: Package Architecture Version Repository Size 2026-02-12T06:38:57.856 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:57.856 INFO:teuthology.orchestra.run.trial028.stdout:Installing: 2026-02-12T06:38:57.856 INFO:teuthology.orchestra.run.trial028.stdout: rbd-fuse x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 87 k 2026-02-12T06:38:57.856 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:57.856 INFO:teuthology.orchestra.run.trial028.stdout:Transaction Summary 2026-02-12T06:38:57.856 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:57.856 INFO:teuthology.orchestra.run.trial028.stdout:Install 1 Package 2026-02-12T06:38:57.857 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:57.857 INFO:teuthology.orchestra.run.trial028.stdout:Total download size: 87 k 2026-02-12T06:38:57.857 INFO:teuthology.orchestra.run.trial028.stdout:Installed size: 236 k 2026-02-12T06:38:57.857 INFO:teuthology.orchestra.run.trial028.stdout:Downloading Packages: 2026-02-12T06:38:58.159 INFO:teuthology.orchestra.run.trial028.stdout:rbd-fuse-20.3.0-5145.g5fdb7d8f.el9.x86_64.rpm 286 kB/s | 87 kB 00:00 2026-02-12T06:38:58.159 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-12T06:38:58.160 INFO:teuthology.orchestra.run.trial028.stdout:Total 285 kB/s | 87 kB 00:00 2026-02-12T06:38:58.160 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction check 2026-02-12T06:38:58.164 INFO:teuthology.orchestra.run.trial028.stdout:Transaction check succeeded. 2026-02-12T06:38:58.164 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction test 2026-02-12T06:38:58.196 INFO:teuthology.orchestra.run.trial028.stdout:Transaction test succeeded. 2026-02-12T06:38:58.196 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction 2026-02-12T06:38:58.247 INFO:teuthology.orchestra.run.trial028.stdout: Preparing : 1/1 2026-02-12T06:38:58.263 INFO:teuthology.orchestra.run.trial028.stdout: Installing : rbd-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:38:58.343 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:38:58.400 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : rbd-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:38:58.400 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:58.400 INFO:teuthology.orchestra.run.trial028.stdout:Installed: 2026-02-12T06:38:58.400 INFO:teuthology.orchestra.run.trial028.stdout: rbd-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:38:58.400 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:58.400 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:38:58.427 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install rbd-mirror 2026-02-12T06:38:58.730 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:25 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:38:58.813 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:38:58.813 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:58.814 INFO:teuthology.orchestra.run.trial028.stdout: Package Arch Version Repo Size 2026-02-12T06:38:58.814 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:58.814 INFO:teuthology.orchestra.run.trial028.stdout:Installing: 2026-02-12T06:38:58.814 INFO:teuthology.orchestra.run.trial028.stdout: rbd-mirror x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 2.6 M 2026-02-12T06:38:58.814 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:58.814 INFO:teuthology.orchestra.run.trial028.stdout:Transaction Summary 2026-02-12T06:38:58.814 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:38:58.814 INFO:teuthology.orchestra.run.trial028.stdout:Install 1 Package 2026-02-12T06:38:58.814 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:38:58.814 INFO:teuthology.orchestra.run.trial028.stdout:Total download size: 2.6 M 2026-02-12T06:38:58.815 INFO:teuthology.orchestra.run.trial028.stdout:Installed size: 11 M 2026-02-12T06:38:58.815 INFO:teuthology.orchestra.run.trial028.stdout:Downloading Packages: 2026-02-12T06:38:59.751 INFO:teuthology.orchestra.run.trial028.stdout:rbd-mirror-20.3.0-5145.g5fdb7d8f.el9.x86_64.rpm 2.8 MB/s | 2.6 MB 00:00 2026-02-12T06:38:59.752 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-12T06:38:59.752 INFO:teuthology.orchestra.run.trial028.stdout:Total 2.8 MB/s | 2.6 MB 00:00 2026-02-12T06:38:59.752 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction check 2026-02-12T06:38:59.757 INFO:teuthology.orchestra.run.trial028.stdout:Transaction check succeeded. 2026-02-12T06:38:59.757 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction test 2026-02-12T06:38:59.796 INFO:teuthology.orchestra.run.trial028.stdout:Transaction test succeeded. 2026-02-12T06:38:59.796 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction 2026-02-12T06:38:59.961 INFO:teuthology.orchestra.run.trial028.stdout: Preparing : 1/1 2026-02-12T06:38:59.963 INFO:teuthology.orchestra.run.trial028.stdout: Installing : rbd-mirror-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:38:59.986 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:38:59.986 INFO:teuthology.orchestra.run.trial028.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T06:38:59.986 INFO:teuthology.orchestra.run.trial028.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-12T06:38:59.986 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-12T06:38:59.986 INFO:teuthology.orchestra.run.trial028.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-12T06:38:59.987 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:39:00.275 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : rbd-mirror-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:39:00.275 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:39:00.275 INFO:teuthology.orchestra.run.trial028.stdout:Installed: 2026-02-12T06:39:00.275 INFO:teuthology.orchestra.run.trial028.stdout: rbd-mirror-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:39:00.275 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:39:00.276 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:39:00.303 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install rbd-nbd 2026-02-12T06:39:00.606 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:27 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:39:00.688 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:39:00.688 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:39:00.689 INFO:teuthology.orchestra.run.trial028.stdout: Package Architecture Version Repository Size 2026-02-12T06:39:00.689 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:39:00.689 INFO:teuthology.orchestra.run.trial028.stdout:Installing: 2026-02-12T06:39:00.689 INFO:teuthology.orchestra.run.trial028.stdout: rbd-nbd x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 167 k 2026-02-12T06:39:00.689 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:39:00.689 INFO:teuthology.orchestra.run.trial028.stdout:Transaction Summary 2026-02-12T06:39:00.689 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:39:00.689 INFO:teuthology.orchestra.run.trial028.stdout:Install 1 Package 2026-02-12T06:39:00.689 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:39:00.690 INFO:teuthology.orchestra.run.trial028.stdout:Total download size: 167 k 2026-02-12T06:39:00.690 INFO:teuthology.orchestra.run.trial028.stdout:Installed size: 496 k 2026-02-12T06:39:00.690 INFO:teuthology.orchestra.run.trial028.stdout:Downloading Packages: 2026-02-12T06:39:00.774 INFO:teuthology.orchestra.run.trial028.stdout:rbd-nbd-20.3.0-5145.g5fdb7d8f.el9.x86_64.rpm 1.9 MB/s | 167 kB 00:00 2026-02-12T06:39:00.775 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-12T06:39:00.775 INFO:teuthology.orchestra.run.trial028.stdout:Total 1.9 MB/s | 167 kB 00:00 2026-02-12T06:39:00.775 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction check 2026-02-12T06:39:00.780 INFO:teuthology.orchestra.run.trial028.stdout:Transaction check succeeded. 2026-02-12T06:39:00.780 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction test 2026-02-12T06:39:00.812 INFO:teuthology.orchestra.run.trial028.stdout:Transaction test succeeded. 2026-02-12T06:39:00.812 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction 2026-02-12T06:39:00.867 INFO:teuthology.orchestra.run.trial028.stdout: Preparing : 1/1 2026-02-12T06:39:00.884 INFO:teuthology.orchestra.run.trial028.stdout: Installing : rbd-nbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:39:00.963 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:39:01.020 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : rbd-nbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/1 2026-02-12T06:39:01.020 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:39:01.020 INFO:teuthology.orchestra.run.trial028.stdout:Installed: 2026-02-12T06:39:01.020 INFO:teuthology.orchestra.run.trial028.stdout: rbd-nbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:39:01.020 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:39:01.020 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:39:01.047 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-osd-crimson 2026-02-12T06:39:01.353 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:28 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:39:01.417 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-osd-crimson-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:39:01.435 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:39:01.435 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:39:01.436 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:39:01.456 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-12T06:39:01.759 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:01:28 ago on Thu 12 Feb 2026 06:37:33 AM UTC. 2026-02-12T06:39:01.842 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:39:01.843 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:39:01.843 INFO:teuthology.orchestra.run.trial028.stdout: Package Arch Version Repo Size 2026-02-12T06:39:01.843 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:39:01.843 INFO:teuthology.orchestra.run.trial028.stdout:Installing: 2026-02-12T06:39:01.844 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 429 M 2026-02-12T06:39:01.844 INFO:teuthology.orchestra.run.trial028.stdout:Installing weak dependencies: 2026-02-12T06:39:01.844 INFO:teuthology.orchestra.run.trial028.stdout: ceph-debugsource x86_64 2:20.3.0-5145.g5fdb7d8f.el9 ceph 12 M 2026-02-12T06:39:01.844 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:39:01.844 INFO:teuthology.orchestra.run.trial028.stdout:Transaction Summary 2026-02-12T06:39:01.844 INFO:teuthology.orchestra.run.trial028.stdout:================================================================================ 2026-02-12T06:39:01.844 INFO:teuthology.orchestra.run.trial028.stdout:Install 2 Packages 2026-02-12T06:39:01.844 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:39:01.844 INFO:teuthology.orchestra.run.trial028.stdout:Total download size: 441 M 2026-02-12T06:39:01.844 INFO:teuthology.orchestra.run.trial028.stdout:Installed size: 2.7 G 2026-02-12T06:39:01.844 INFO:teuthology.orchestra.run.trial028.stdout:Downloading Packages: 2026-02-12T06:39:05.280 INFO:teuthology.orchestra.run.trial028.stdout:(1/2): ceph-debugsource-20.3.0-5145.g5fdb7d8f.e 3.5 MB/s | 12 MB 00:03 2026-02-12T06:39:52.578 INFO:teuthology.orchestra.run.trial028.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5145.g 8.5 MB/s | 429 MB 00:50 2026-02-12T06:39:52.579 INFO:teuthology.orchestra.run.trial028.stdout:-------------------------------------------------------------------------------- 2026-02-12T06:39:52.579 INFO:teuthology.orchestra.run.trial028.stdout:Total 8.7 MB/s | 441 MB 00:50 2026-02-12T06:39:52.579 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction check 2026-02-12T06:39:52.585 INFO:teuthology.orchestra.run.trial028.stdout:Transaction check succeeded. 2026-02-12T06:39:52.585 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction test 2026-02-12T06:39:53.946 INFO:teuthology.orchestra.run.trial028.stdout:Transaction test succeeded. 2026-02-12T06:39:53.946 INFO:teuthology.orchestra.run.trial028.stdout:Running transaction 2026-02-12T06:39:57.514 INFO:teuthology.orchestra.run.trial028.stdout: Preparing : 1/1 2026-02-12T06:40:19.597 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-debugsource-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/2 2026-02-12T06:40:19.615 INFO:teuthology.orchestra.run.trial028.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5145.g5fdb7d8f.e 2/2 2026-02-12T06:40:21.674 INFO:teuthology.orchestra.run.trial028.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5145.g5fdb7d8f.e 2/2 2026-02-12T06:40:21.674 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-debugsource-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 1/2 2026-02-12T06:40:21.732 INFO:teuthology.orchestra.run.trial028.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5145.g5fdb7d8f.e 2/2 2026-02-12T06:40:21.732 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:21.732 INFO:teuthology.orchestra.run.trial028.stdout:Installed: 2026-02-12T06:40:21.732 INFO:teuthology.orchestra.run.trial028.stdout: ceph-debugsource-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:40:21.732 INFO:teuthology.orchestra.run.trial028.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 2026-02-12T06:40:21.732 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:21.733 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:21.805 DEBUG:teuthology.parallel:result is None 2026-02-12T06:40:21.805 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 2026-02-12T06:40:21.923 DEBUG:teuthology.orchestra.run.trial028:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-12T06:40:21.943 INFO:teuthology.orchestra.run.trial028.stdout:20.3.0-5145.g5fdb7d8f.el9 2026-02-12T06:40:21.943 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5145.g5fdb7d8f.el9 2026-02-12T06:40:21.943 INFO:teuthology.task.install:The correct ceph version 20.3.0-5145.g5fdb7d8f is installed. 2026-02-12T06:40:21.945 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-12T06:40:21.945 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:40:21.945 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-12T06:40:22.009 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-12T06:40:22.010 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:40:22.010 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/usr/bin/daemon-helper 2026-02-12T06:40:22.073 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-12T06:40:22.137 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-12T06:40:22.138 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:40:22.138 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-12T06:40:22.202 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-12T06:40:22.266 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-12T06:40:22.266 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:40:22.266 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/usr/bin/stdin-killer 2026-02-12T06:40:22.329 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-12T06:40:22.393 INFO:teuthology.run_tasks:Running task pexec... 2026-02-12T06:40:22.407 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-12T06:40:22.407 DEBUG:teuthology.orchestra.run.trial028:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-12T06:40:22.434 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial028.front.sepia.ceph.com 2026-02-12T06:40:22.435 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-12T06:40:22.484 DEBUG:teuthology.parallel:result is None 2026-02-12T06:40:22.484 INFO:teuthology.run_tasks:Running task install... 2026-02-12T06:40:22.497 DEBUG:teuthology.task.install:project ceph 2026-02-12T06:40:22.497 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'}} 2026-02-12T06:40:22.497 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'} 2026-02-12T06:40:22.497 INFO:teuthology.task.install:Using flavor: debug 2026-02-12T06:40:22.499 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-12T06:40:22.499 INFO:teuthology.task.install:extra packages: [] 2026-02-12T06:40:22.499 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0', 'tag': None, 'wait_for_package': False} 2026-02-12T06:40:22.500 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 2026-02-12T06:40:22.643 INFO:teuthology.task.install.rpm:Pulling from https://1.chacra.ceph.com/r/ceph/wip-moagrawa-seastore-io-uring-again-crimson-only/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/centos/9/flavors/debug/ 2026-02-12T06:40:22.643 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5145.g5fdb7d8f 2026-02-12T06:40:22.903 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://1.chacra.ceph.com/r/ceph/wip-moagrawa-seastore-io-uring-again-crimson-only/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-moagrawa-seastore-io-uring-again-crimson-only/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://1.chacra.ceph.com/r/ceph/wip-moagrawa-seastore-io-uring-again-crimson-only/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-12T06:40:22.903 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:40:22.903 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-12T06:40:22.928 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-12T06:40:22.928 DEBUG:teuthology.orchestra.run.trial028:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-12T06:40:22.993 DEBUG:teuthology.orchestra.run.trial028:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-12T06:40:23.056 DEBUG:teuthology.orchestra.run.trial028:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-12T06:40:23.114 INFO:teuthology.orchestra.run.trial028.stdout:check_obsoletes = 1 2026-02-12T06:40:23.124 DEBUG:teuthology.orchestra.run.trial028:> sudo yum clean all 2026-02-12T06:40:23.283 INFO:teuthology.orchestra.run.trial028.stdout:56 files removed 2026-02-12T06:40:23.297 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-radosgw 2026-02-12T06:40:24.063 INFO:teuthology.orchestra.run.trial028.stdout:ceph packages for x86_64 152 kB/s | 94 kB 00:00 2026-02-12T06:40:24.468 INFO:teuthology.orchestra.run.trial028.stdout:ceph noarch packages 64 kB/s | 24 kB 00:00 2026-02-12T06:40:24.965 INFO:teuthology.orchestra.run.trial028.stdout:ceph source packages 4.4 kB/s | 2.1 kB 00:00 2026-02-12T06:40:26.413 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - BaseOS 6.2 MB/s | 8.9 MB 00:01 2026-02-12T06:40:29.464 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-12T06:40:32.850 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 7.8 MB 00:01 2026-02-12T06:40:33.792 INFO:teuthology.orchestra.run.trial028.stdout:CentOS Stream 9 - Extras packages 97 kB/s | 20 kB 00:00 2026-02-12T06:40:34.938 INFO:teuthology.orchestra.run.trial028.stdout:Extra Packages for Enterprise Linux 19 MB/s | 20 MB 00:01 2026-02-12T06:40:38.175 INFO:teuthology.orchestra.run.trial028.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-12T06:40:39.438 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-radosgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:39.456 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:39.456 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:39.457 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:39.481 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-test 2026-02-12T06:40:39.778 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:01 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:39.842 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-test-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:39.860 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:39.860 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:39.860 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:39.881 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph 2026-02-12T06:40:40.179 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:02 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:40.243 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:40.260 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:40.261 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:40.261 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:40.283 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-base 2026-02-12T06:40:40.577 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:02 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:40.641 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-base-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:40.658 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:40.659 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:40.659 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:40.680 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install cephadm 2026-02-12T06:40:40.977 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:02 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:41.042 INFO:teuthology.orchestra.run.trial028.stdout:Package cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T06:40:41.059 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:41.060 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:41.060 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:41.081 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-immutable-object-cache 2026-02-12T06:40:41.380 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:03 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:41.446 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-immutable-object-cache-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:41.463 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:41.464 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:41.465 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:41.487 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-mgr 2026-02-12T06:40:41.787 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:03 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:41.851 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-mgr-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:41.868 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:41.869 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:41.869 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:41.890 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-mgr-dashboard 2026-02-12T06:40:42.186 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:04 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:42.250 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-mgr-dashboard-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T06:40:42.267 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:42.268 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:42.268 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:42.290 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-12T06:40:42.593 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:04 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:42.658 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T06:40:42.676 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:42.676 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:42.677 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:42.698 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-mgr-rook 2026-02-12T06:40:42.999 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:04 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:43.064 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-mgr-rook-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T06:40:43.081 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:43.082 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:43.082 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:43.104 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-mgr-cephadm 2026-02-12T06:40:43.404 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:05 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:43.469 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-mgr-cephadm-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T06:40:43.487 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:43.487 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:43.487 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:43.509 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-osd 2026-02-12T06:40:43.809 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:05 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:43.873 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-osd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:43.890 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:43.891 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:43.891 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:43.914 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-osd-classic 2026-02-12T06:40:44.218 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:06 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:44.283 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-osd-classic-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:44.301 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:44.301 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:44.301 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:44.323 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-fuse 2026-02-12T06:40:44.624 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:06 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:44.688 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:44.706 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:44.706 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:44.706 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:44.729 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install ceph-volume 2026-02-12T06:40:45.031 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:07 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:45.096 INFO:teuthology.orchestra.run.trial028.stdout:Package ceph-volume-2:20.3.0-5145.g5fdb7d8f.el9.noarch is already installed. 2026-02-12T06:40:45.113 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:45.114 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:45.114 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:45.136 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install librados-devel 2026-02-12T06:40:45.435 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:07 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:45.500 INFO:teuthology.orchestra.run.trial028.stdout:Package librados-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:45.517 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:45.518 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:45.518 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:45.541 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install libcephfs2 2026-02-12T06:40:45.842 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:07 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:45.906 INFO:teuthology.orchestra.run.trial028.stdout:Package libcephfs2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:45.924 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:45.924 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:45.925 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:45.947 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install libcephfs-devel 2026-02-12T06:40:46.246 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:08 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:46.310 INFO:teuthology.orchestra.run.trial028.stdout:Package libcephfs-devel-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:46.328 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:46.329 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:46.329 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:46.349 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install librados2 2026-02-12T06:40:46.645 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:08 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:46.709 INFO:teuthology.orchestra.run.trial028.stdout:Package librados2-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:46.727 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:46.727 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:46.728 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:46.749 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install librbd1 2026-02-12T06:40:47.050 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:09 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:47.114 INFO:teuthology.orchestra.run.trial028.stdout:Package librbd1-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:47.131 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:47.132 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:47.132 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:47.154 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install python3-rados 2026-02-12T06:40:47.449 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:09 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:47.514 INFO:teuthology.orchestra.run.trial028.stdout:Package python3-rados-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:47.532 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:47.532 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:47.533 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:47.554 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install python3-rgw 2026-02-12T06:40:47.856 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:09 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:47.921 INFO:teuthology.orchestra.run.trial028.stdout:Package python3-rgw-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:47.939 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:47.939 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:47.939 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:47.961 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install python3-cephfs 2026-02-12T06:40:48.260 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:10 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:48.325 INFO:teuthology.orchestra.run.trial028.stdout:Package python3-cephfs-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:48.342 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:48.343 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:48.343 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:48.365 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install python3-rbd 2026-02-12T06:40:48.668 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:10 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:48.733 INFO:teuthology.orchestra.run.trial028.stdout:Package python3-rbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:48.750 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:48.751 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:48.751 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:48.772 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install rbd-fuse 2026-02-12T06:40:49.072 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:11 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:49.136 INFO:teuthology.orchestra.run.trial028.stdout:Package rbd-fuse-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:49.154 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:49.154 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:49.154 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:49.176 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install rbd-mirror 2026-02-12T06:40:49.479 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:11 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:49.544 INFO:teuthology.orchestra.run.trial028.stdout:Package rbd-mirror-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:49.561 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:49.562 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:49.562 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:49.585 DEBUG:teuthology.orchestra.run.trial028:> sudo yum -y install rbd-nbd 2026-02-12T06:40:49.883 INFO:teuthology.orchestra.run.trial028.stdout:Last metadata expiration check: 0:00:11 ago on Thu 12 Feb 2026 06:40:38 AM UTC. 2026-02-12T06:40:49.946 INFO:teuthology.orchestra.run.trial028.stdout:Package rbd-nbd-2:20.3.0-5145.g5fdb7d8f.el9.x86_64 is already installed. 2026-02-12T06:40:49.964 INFO:teuthology.orchestra.run.trial028.stdout:Dependencies resolved. 2026-02-12T06:40:49.964 INFO:teuthology.orchestra.run.trial028.stdout:Nothing to do. 2026-02-12T06:40:49.965 INFO:teuthology.orchestra.run.trial028.stdout:Complete! 2026-02-12T06:40:49.991 DEBUG:teuthology.parallel:result is None 2026-02-12T06:40:49.991 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0 2026-02-12T06:40:50.153 DEBUG:teuthology.orchestra.run.trial028:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-12T06:40:50.175 INFO:teuthology.orchestra.run.trial028.stdout:20.3.0-5145.g5fdb7d8f.el9 2026-02-12T06:40:50.175 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5145.g5fdb7d8f.el9 2026-02-12T06:40:50.175 INFO:teuthology.task.install:The correct ceph version 20.3.0-5145.g5fdb7d8f is installed. 2026-02-12T06:40:50.177 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-12T06:40:50.177 INFO:teuthology.run_tasks:Running task ceph... 2026-02-12T06:40:50.275 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-12T06:40:50.275 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod 777 /var/log/ceph 2026-02-12T06:40:50.300 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-12T06:40:50.300 DEBUG:teuthology.orchestra.run.trial028:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-12T06:40:50.364 INFO:tasks.ceph:Creating extra log directories... 2026-02-12T06:40:50.364 DEBUG:teuthology.orchestra.run.trial028:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-12T06:40:50.439 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-12T06:40:50.440 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-12T06:40:50.440 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/moagrawa-2026-02-12_06:23:35-crimson-rados-main-distro-debug-trial/45989', 'branch': 'main', 'description': 'crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} perf_ignorelist settings/optimized workloads/radosbench_omap_write}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '45989', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'moagrawa-2026-02-12_06:23:35-crimson-rados-main-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'}}, 'workunit': {'branch': 'seastore_io_uring', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'}}, 'owner': 'scheduled_moagrawa@soko04.front.sepia.ceph.com', 'priority': 20, 'repo': 'https://github.com/ceph/ceph.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 4661, 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'seastore_io_uring', 'suite_path': '/home/teuthworker/src/github.com_mohit84_ceph_5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/mohit84/ceph.git', 'suite_sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0', 'targets': {'trial028.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNVE/0HhYQzW42G38rTjqdntFpns3RvRehmIe6cqOFB3Nc9/qt84vtFVVXfZ9Xis+wFctgP+xJ6ZpurHPdcOFvs='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'wait-for-scrub': False, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': '5fdb7d8f131dcee024b0dd7d68acdff57e55c4e0', 'cluster': 'ceph'}}, {'ssh_keys': None}, {'radosbench': {'clients': ['client.0'], 'objectsize': 4096, 'size': 4096, 'time': 300, 'write-omap': True}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': '8bec0da71becad44414c54979f64c9ef0e7099c6', 'timestamp': '2026-02-12_06:23:35', 'tube': 'trial', 'user': 'moagrawa', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.2161924'} 2026-02-12T06:40:50.441 DEBUG:teuthology.orchestra.run.trial028:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-12T06:40:50.484 DEBUG:teuthology.orchestra.run.trial028:> sudo install -d -m0777 -- /var/run/ceph 2026-02-12T06:40:50.548 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:40:50.548 DEBUG:teuthology.orchestra.run.trial028:> dd if=/scratch_devs of=/dev/stdout 2026-02-12T06:40:50.607 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-12T06:40:50.607 DEBUG:teuthology.orchestra.run.trial028:> stat /dev/vg_nvme/lv_1 2026-02-12T06:40:50.663 INFO:teuthology.orchestra.run.trial028.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-12T06:40:50.663 INFO:teuthology.orchestra.run.trial028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-12T06:40:50.664 INFO:teuthology.orchestra.run.trial028.stdout:Device: 6h/6d Inode: 1311 Links: 1 2026-02-12T06:40:50.664 INFO:teuthology.orchestra.run.trial028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-12T06:40:50.664 INFO:teuthology.orchestra.run.trial028.stdout:Context: system_u:object_r:device_t:s0 2026-02-12T06:40:50.664 INFO:teuthology.orchestra.run.trial028.stdout:Access: 2026-02-12 06:39:00.118666610 +0000 2026-02-12T06:40:50.664 INFO:teuthology.orchestra.run.trial028.stdout:Modify: 2026-02-12 06:37:39.293044032 +0000 2026-02-12T06:40:50.664 INFO:teuthology.orchestra.run.trial028.stdout:Change: 2026-02-12 06:37:39.293044032 +0000 2026-02-12T06:40:50.664 INFO:teuthology.orchestra.run.trial028.stdout: Birth: 2026-02-12 06:37:39.293044032 +0000 2026-02-12T06:40:50.664 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-12T06:40:50.727 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records in 2026-02-12T06:40:50.727 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records out 2026-02-12T06:40:50.727 INFO:teuthology.orchestra.run.trial028.stderr:512 bytes copied, 0.000160928 s, 3.2 MB/s 2026-02-12T06:40:50.729 DEBUG:teuthology.orchestra.run.trial028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-12T06:40:50.787 DEBUG:teuthology.orchestra.run.trial028:> stat /dev/vg_nvme/lv_2 2026-02-12T06:40:50.843 INFO:teuthology.orchestra.run.trial028.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-12T06:40:50.843 INFO:teuthology.orchestra.run.trial028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-12T06:40:50.843 INFO:teuthology.orchestra.run.trial028.stdout:Device: 6h/6d Inode: 1299 Links: 1 2026-02-12T06:40:50.843 INFO:teuthology.orchestra.run.trial028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-12T06:40:50.843 INFO:teuthology.orchestra.run.trial028.stdout:Context: system_u:object_r:device_t:s0 2026-02-12T06:40:50.843 INFO:teuthology.orchestra.run.trial028.stdout:Access: 2026-02-12 06:39:00.118666610 +0000 2026-02-12T06:40:50.843 INFO:teuthology.orchestra.run.trial028.stdout:Modify: 2026-02-12 06:37:39.292044049 +0000 2026-02-12T06:40:50.843 INFO:teuthology.orchestra.run.trial028.stdout:Change: 2026-02-12 06:37:39.292044049 +0000 2026-02-12T06:40:50.844 INFO:teuthology.orchestra.run.trial028.stdout: Birth: 2026-02-12 06:37:39.292044049 +0000 2026-02-12T06:40:50.844 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-12T06:40:50.905 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records in 2026-02-12T06:40:50.905 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records out 2026-02-12T06:40:50.905 INFO:teuthology.orchestra.run.trial028.stderr:512 bytes copied, 0.000148195 s, 3.5 MB/s 2026-02-12T06:40:50.907 DEBUG:teuthology.orchestra.run.trial028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-12T06:40:50.964 DEBUG:teuthology.orchestra.run.trial028:> stat /dev/vg_nvme/lv_3 2026-02-12T06:40:51.021 INFO:teuthology.orchestra.run.trial028.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-12T06:40:51.022 INFO:teuthology.orchestra.run.trial028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-12T06:40:51.022 INFO:teuthology.orchestra.run.trial028.stdout:Device: 6h/6d Inode: 1307 Links: 1 2026-02-12T06:40:51.022 INFO:teuthology.orchestra.run.trial028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-12T06:40:51.022 INFO:teuthology.orchestra.run.trial028.stdout:Context: system_u:object_r:device_t:s0 2026-02-12T06:40:51.022 INFO:teuthology.orchestra.run.trial028.stdout:Access: 2026-02-12 06:39:00.118666610 +0000 2026-02-12T06:40:51.022 INFO:teuthology.orchestra.run.trial028.stdout:Modify: 2026-02-12 06:37:39.293044032 +0000 2026-02-12T06:40:51.022 INFO:teuthology.orchestra.run.trial028.stdout:Change: 2026-02-12 06:37:39.293044032 +0000 2026-02-12T06:40:51.022 INFO:teuthology.orchestra.run.trial028.stdout: Birth: 2026-02-12 06:37:39.293044032 +0000 2026-02-12T06:40:51.023 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-12T06:40:51.084 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records in 2026-02-12T06:40:51.084 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records out 2026-02-12T06:40:51.084 INFO:teuthology.orchestra.run.trial028.stderr:512 bytes copied, 0.000143526 s, 3.6 MB/s 2026-02-12T06:40:51.086 DEBUG:teuthology.orchestra.run.trial028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-12T06:40:51.143 DEBUG:teuthology.orchestra.run.trial028:> stat /dev/vg_nvme/lv_4 2026-02-12T06:40:51.200 INFO:teuthology.orchestra.run.trial028.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-12T06:40:51.200 INFO:teuthology.orchestra.run.trial028.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-12T06:40:51.200 INFO:teuthology.orchestra.run.trial028.stdout:Device: 6h/6d Inode: 1295 Links: 1 2026-02-12T06:40:51.200 INFO:teuthology.orchestra.run.trial028.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-12T06:40:51.200 INFO:teuthology.orchestra.run.trial028.stdout:Context: system_u:object_r:device_t:s0 2026-02-12T06:40:51.200 INFO:teuthology.orchestra.run.trial028.stdout:Access: 2026-02-12 06:39:00.118666610 +0000 2026-02-12T06:40:51.201 INFO:teuthology.orchestra.run.trial028.stdout:Modify: 2026-02-12 06:37:39.292044049 +0000 2026-02-12T06:40:51.201 INFO:teuthology.orchestra.run.trial028.stdout:Change: 2026-02-12 06:37:39.292044049 +0000 2026-02-12T06:40:51.201 INFO:teuthology.orchestra.run.trial028.stdout: Birth: 2026-02-12 06:37:39.292044049 +0000 2026-02-12T06:40:51.201 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-12T06:40:51.262 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records in 2026-02-12T06:40:51.262 INFO:teuthology.orchestra.run.trial028.stderr:1+0 records out 2026-02-12T06:40:51.262 INFO:teuthology.orchestra.run.trial028.stderr:512 bytes copied, 0.000133568 s, 3.8 MB/s 2026-02-12T06:40:51.263 DEBUG:teuthology.orchestra.run.trial028:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-12T06:40:51.321 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-12T06:40:51.321 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial028.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-12T06:40:51.321 INFO:tasks.ceph:Generating config... 2026-02-12T06:40:51.323 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-12T06:40:51.324 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-12T06:40:51.324 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-12T06:40:51.324 INFO:tasks.ceph:[osd] debug filestore = 0/0 2026-02-12T06:40:51.324 INFO:tasks.ceph:[osd] debug journal = 0/0 2026-02-12T06:40:51.324 INFO:tasks.ceph:[osd] debug ms = 0/0 2026-02-12T06:40:51.324 INFO:tasks.ceph:[osd] debug osd = 0/0 2026-02-12T06:40:51.324 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-12T06:40:51.324 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-12T06:40:51.324 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-12T06:40:51.325 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-12T06:40:51.325 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-12T06:40:51.325 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-12T06:40:51.325 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-12T06:40:51.325 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-12T06:40:51.325 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-12T06:40:51.325 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-12T06:40:51.325 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-12T06:40:51.325 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-12T06:40:51.325 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-12T06:40:51.325 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-12T06:40:51.325 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-12T06:40:51.325 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-12T06:40:51.325 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-12T06:40:51.326 INFO:tasks.ceph:[osd] seastore main device type = RANDOM_BLOCK_SSD 2026-02-12T06:40:51.326 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-12T06:40:51.326 INFO:tasks.ceph:[global] auth client required = none 2026-02-12T06:40:51.326 INFO:tasks.ceph:[global] auth cluster required = none 2026-02-12T06:40:51.326 INFO:tasks.ceph:[global] auth service required = none 2026-02-12T06:40:51.326 INFO:tasks.ceph:[global] auth supported = none 2026-02-12T06:40:51.326 INFO:tasks.ceph:[global] debug asok = 0/0 2026-02-12T06:40:51.326 INFO:tasks.ceph:[global] debug auth = 0/0 2026-02-12T06:40:51.326 INFO:tasks.ceph:[global] debug bdev = 0/0 2026-02-12T06:40:51.326 INFO:tasks.ceph:[global] debug bluefs = 0/0 2026-02-12T06:40:51.326 INFO:tasks.ceph:[global] debug bluestore = 0/0 2026-02-12T06:40:51.326 INFO:tasks.ceph:[global] debug buffer = 0/0 2026-02-12T06:40:51.327 INFO:tasks.ceph:[global] debug civetweb = 0/0 2026-02-12T06:40:51.327 INFO:tasks.ceph:[global] debug client = 0/0 2026-02-12T06:40:51.327 INFO:tasks.ceph:[global] debug compressor = 0/0 2026-02-12T06:40:51.327 INFO:tasks.ceph:[global] debug context = 0/0 2026-02-12T06:40:51.327 INFO:tasks.ceph:[global] debug crush = 0/0 2026-02-12T06:40:51.327 INFO:tasks.ceph:[global] debug crypto = 0/0 2026-02-12T06:40:51.327 INFO:tasks.ceph:[global] debug dpdk = 0/0 2026-02-12T06:40:51.327 INFO:tasks.ceph:[global] debug eventtrace = 0/0 2026-02-12T06:40:51.327 INFO:tasks.ceph:[global] debug filer = 0/0 2026-02-12T06:40:51.327 INFO:tasks.ceph:[global] debug filestore = 0/0 2026-02-12T06:40:51.327 INFO:tasks.ceph:[global] debug finisher = 0/0 2026-02-12T06:40:51.327 INFO:tasks.ceph:[global] debug fuse = 0/0 2026-02-12T06:40:51.327 INFO:tasks.ceph:[global] debug heartbeatmap = 0/0 2026-02-12T06:40:51.328 INFO:tasks.ceph:[global] debug javaclient = 0/0 2026-02-12T06:40:51.328 INFO:tasks.ceph:[global] debug journal = 0/0 2026-02-12T06:40:51.328 INFO:tasks.ceph:[global] debug journaler = 0/0 2026-02-12T06:40:51.328 INFO:tasks.ceph:[global] debug kstore = 0/0 2026-02-12T06:40:51.328 INFO:tasks.ceph:[global] debug leveldb = 0/0 2026-02-12T06:40:51.328 INFO:tasks.ceph:[global] debug lockdep = 0/0 2026-02-12T06:40:51.328 INFO:tasks.ceph:[global] debug mds = 0/0 2026-02-12T06:40:51.328 INFO:tasks.ceph:[global] debug mds balancer = 0/0 2026-02-12T06:40:51.328 INFO:tasks.ceph:[global] debug mds locker = 0/0 2026-02-12T06:40:51.328 INFO:tasks.ceph:[global] debug mds log = 0/0 2026-02-12T06:40:51.328 INFO:tasks.ceph:[global] debug mds log expire = 0/0 2026-02-12T06:40:51.328 INFO:tasks.ceph:[global] debug mds migrator = 0/0 2026-02-12T06:40:51.328 INFO:tasks.ceph:[global] debug memdb = 0/0 2026-02-12T06:40:51.329 INFO:tasks.ceph:[global] debug mgr = 0/0 2026-02-12T06:40:51.329 INFO:tasks.ceph:[global] debug mgrc = 0/0 2026-02-12T06:40:51.329 INFO:tasks.ceph:[global] debug mon = 0/0 2026-02-12T06:40:51.329 INFO:tasks.ceph:[global] debug monc = 0/0 2026-02-12T06:40:51.329 INFO:tasks.ceph:[global] debug ms = 0/0 2026-02-12T06:40:51.329 INFO:tasks.ceph:[global] debug objclass = 0/0 2026-02-12T06:40:51.329 INFO:tasks.ceph:[global] debug objectcacher = 0/0 2026-02-12T06:40:51.329 INFO:tasks.ceph:[global] debug objecter = 0/0 2026-02-12T06:40:51.329 INFO:tasks.ceph:[global] debug optracker = 0/0 2026-02-12T06:40:51.329 INFO:tasks.ceph:[global] debug osd = 0/0 2026-02-12T06:40:51.329 INFO:tasks.ceph:[global] debug paxos = 0/0 2026-02-12T06:40:51.329 INFO:tasks.ceph:[global] debug perfcounter = 0/0 2026-02-12T06:40:51.329 INFO:tasks.ceph:[global] debug rados = 0/0 2026-02-12T06:40:51.330 INFO:tasks.ceph:[global] debug rbd = 0/0 2026-02-12T06:40:51.330 INFO:tasks.ceph:[global] debug rbd mirror = 0/0 2026-02-12T06:40:51.330 INFO:tasks.ceph:[global] debug rbd replay = 0/0 2026-02-12T06:40:51.330 INFO:tasks.ceph:[global] debug refs = 0/0 2026-02-12T06:40:51.330 INFO:tasks.ceph:[global] debug rgw = 0/0 2026-02-12T06:40:51.330 INFO:tasks.ceph:[global] debug rgw lifecycle = 0/0 2026-02-12T06:40:51.330 INFO:tasks.ceph:[global] debug rgw notification = 0/0 2026-02-12T06:40:51.330 INFO:tasks.ceph:[global] debug rgw sync = 0/0 2026-02-12T06:40:51.330 INFO:tasks.ceph:[global] debug rocksdb = 0/0 2026-02-12T06:40:51.330 INFO:tasks.ceph:[global] debug striper = 0/0 2026-02-12T06:40:51.330 INFO:tasks.ceph:[global] debug throttle = 0/0 2026-02-12T06:40:51.330 INFO:tasks.ceph:[global] debug timer = 0/0 2026-02-12T06:40:51.330 INFO:tasks.ceph:[global] debug tp = 0/0 2026-02-12T06:40:51.331 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-12T06:40:51.331 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-12T06:40:51.331 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-12T06:40:51.331 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-12T06:40:51.331 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-12T06:40:51.331 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-12T06:40:51.331 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-12T06:40:51.331 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-12T06:40:51.331 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-12T06:40:51.331 INFO:tasks.ceph:[mon] debug mon = 0/0 2026-02-12T06:40:51.331 INFO:tasks.ceph:[mon] debug ms = 0/0 2026-02-12T06:40:51.331 INFO:tasks.ceph:[mon] debug paxos = 0/0 2026-02-12T06:40:51.331 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-12T06:40:51.332 INFO:tasks.ceph:Setting up mon.a... 2026-02-12T06:40:51.332 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-12T06:40:51.403 INFO:teuthology.orchestra.run.trial028.stdout:creating /etc/ceph/ceph.keyring 2026-02-12T06:40:51.405 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-12T06:40:51.491 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-12T06:40:51.557 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.28')] 2026-02-12T06:40:51.558 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'none', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.28', 'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': '0/0', 'debug mon': '0/0', 'debug paxos': '0/0', 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-12T06:40:51.560 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:40:51.560 DEBUG:teuthology.orchestra.run.trial028:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-12T06:40:51.614 DEBUG:teuthology.orchestra.run.trial028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.28 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-12T06:40:51.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:40:51.692+0000 7fc731693f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:40:51.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:40:51.692+0000 7fc731693f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:40:51.695 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:40:51.693+0000 7fc731693f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:40:51.695 INFO:teuthology.orchestra.run.trial028.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-12T06:40:51.696 INFO:teuthology.orchestra.run.trial028.stdout:monmaptool: generated fsid 0c019592-0202-4bc6-848f-fd37de9403a4 2026-02-12T06:40:51.696 INFO:teuthology.orchestra.run.trial028.stdout:setting min_mon_release = tentacle 2026-02-12T06:40:51.696 INFO:teuthology.orchestra.run.trial028.stdout:epoch 0 2026-02-12T06:40:51.696 INFO:teuthology.orchestra.run.trial028.stdout:fsid 0c019592-0202-4bc6-848f-fd37de9403a4 2026-02-12T06:40:51.696 INFO:teuthology.orchestra.run.trial028.stdout:last_changed 2026-02-12T06:40:51.694700+0000 2026-02-12T06:40:51.696 INFO:teuthology.orchestra.run.trial028.stdout:created 2026-02-12T06:40:51.694700+0000 2026-02-12T06:40:51.696 INFO:teuthology.orchestra.run.trial028.stdout:min_mon_release 20 (tentacle) 2026-02-12T06:40:51.696 INFO:teuthology.orchestra.run.trial028.stdout:election_strategy: 1 2026-02-12T06:40:51.696 INFO:teuthology.orchestra.run.trial028.stdout:0: [v2:10.20.193.28:3300/0,v1:10.20.193.28:6789/0] mon.a 2026-02-12T06:40:51.696 INFO:teuthology.orchestra.run.trial028.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-12T06:40:51.698 DEBUG:teuthology.orchestra.run.trial028:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-12T06:40:51.756 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 0c019592-0202-4bc6-848f-fd37de9403a4... 2026-02-12T06:40:51.758 DEBUG:teuthology.orchestra.run.trial028:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-12T06:40:51.839 INFO:teuthology.orchestra.run.trial028.stdout:[global] 2026-02-12T06:40:51.840 INFO:teuthology.orchestra.run.trial028.stdout: chdir = "" 2026-02-12T06:40:51.840 INFO:teuthology.orchestra.run.trial028.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-12T06:40:51.840 INFO:teuthology.orchestra.run.trial028.stdout: auth supported = none 2026-02-12T06:40:51.840 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.840 INFO:teuthology.orchestra.run.trial028.stdout: filestore xattr use omap = true 2026-02-12T06:40:51.840 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.840 INFO:teuthology.orchestra.run.trial028.stdout: mon clock drift allowed = 1.000 2026-02-12T06:40:51.840 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.840 INFO:teuthology.orchestra.run.trial028.stdout: osd crush chooseleaf type = 0 2026-02-12T06:40:51.840 INFO:teuthology.orchestra.run.trial028.stdout: auth debug = true 2026-02-12T06:40:51.841 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.841 INFO:teuthology.orchestra.run.trial028.stdout: ms die on old message = true 2026-02-12T06:40:51.841 INFO:teuthology.orchestra.run.trial028.stdout: ms die on bug = true 2026-02-12T06:40:51.841 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.841 INFO:teuthology.orchestra.run.trial028.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-12T06:40:51.841 INFO:teuthology.orchestra.run.trial028.stdout: mon pg warn max object skew = 0 2026-02-12T06:40:51.841 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.841 INFO:teuthology.orchestra.run.trial028.stdout: # disable pg_autoscaler by default for new pools 2026-02-12T06:40:51.841 INFO:teuthology.orchestra.run.trial028.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-12T06:40:51.841 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.841 INFO:teuthology.orchestra.run.trial028.stdout: osd pool default size = 2 2026-02-12T06:40:51.841 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.842 INFO:teuthology.orchestra.run.trial028.stdout: mon osd allow primary affinity = true 2026-02-12T06:40:51.842 INFO:teuthology.orchestra.run.trial028.stdout: mon osd allow pg remap = true 2026-02-12T06:40:51.842 INFO:teuthology.orchestra.run.trial028.stdout: mon warn on legacy crush tunables = false 2026-02-12T06:40:51.842 INFO:teuthology.orchestra.run.trial028.stdout: mon warn on crush straw calc version zero = false 2026-02-12T06:40:51.842 INFO:teuthology.orchestra.run.trial028.stdout: mon warn on no sortbitwise = false 2026-02-12T06:40:51.842 INFO:teuthology.orchestra.run.trial028.stdout: mon warn on osd down out interval zero = false 2026-02-12T06:40:51.842 INFO:teuthology.orchestra.run.trial028.stdout: mon warn on too few osds = false 2026-02-12T06:40:51.842 INFO:teuthology.orchestra.run.trial028.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-12T06:40:51.842 INFO:teuthology.orchestra.run.trial028.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-12T06:40:51.842 INFO:teuthology.orchestra.run.trial028.stdout: mon_allow_pool_size_one = true 2026-02-12T06:40:51.842 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.843 INFO:teuthology.orchestra.run.trial028.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-12T06:40:51.843 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.843 INFO:teuthology.orchestra.run.trial028.stdout: osd default data pool replay window = 5 2026-02-12T06:40:51.843 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.843 INFO:teuthology.orchestra.run.trial028.stdout: mon allow pool delete = true 2026-02-12T06:40:51.843 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.843 INFO:teuthology.orchestra.run.trial028.stdout: mon cluster log file level = debug 2026-02-12T06:40:51.843 INFO:teuthology.orchestra.run.trial028.stdout: debug asserts on shutdown = true 2026-02-12T06:40:51.843 INFO:teuthology.orchestra.run.trial028.stdout: mon health detail to clog = false 2026-02-12T06:40:51.843 INFO:teuthology.orchestra.run.trial028.stdout: mon host = 10.20.193.28 2026-02-12T06:40:51.843 INFO:teuthology.orchestra.run.trial028.stdout: auth client required = none 2026-02-12T06:40:51.843 INFO:teuthology.orchestra.run.trial028.stdout: auth cluster required = none 2026-02-12T06:40:51.843 INFO:teuthology.orchestra.run.trial028.stdout: auth service required = none 2026-02-12T06:40:51.844 INFO:teuthology.orchestra.run.trial028.stdout: debug asok = 0/0 2026-02-12T06:40:51.844 INFO:teuthology.orchestra.run.trial028.stdout: debug auth = 0/0 2026-02-12T06:40:51.844 INFO:teuthology.orchestra.run.trial028.stdout: debug bdev = 0/0 2026-02-12T06:40:51.844 INFO:teuthology.orchestra.run.trial028.stdout: debug bluefs = 0/0 2026-02-12T06:40:51.844 INFO:teuthology.orchestra.run.trial028.stdout: debug bluestore = 0/0 2026-02-12T06:40:51.844 INFO:teuthology.orchestra.run.trial028.stdout: debug buffer = 0/0 2026-02-12T06:40:51.844 INFO:teuthology.orchestra.run.trial028.stdout: debug civetweb = 0/0 2026-02-12T06:40:51.844 INFO:teuthology.orchestra.run.trial028.stdout: debug client = 0/0 2026-02-12T06:40:51.844 INFO:teuthology.orchestra.run.trial028.stdout: debug compressor = 0/0 2026-02-12T06:40:51.844 INFO:teuthology.orchestra.run.trial028.stdout: debug context = 0/0 2026-02-12T06:40:51.844 INFO:teuthology.orchestra.run.trial028.stdout: debug crush = 0/0 2026-02-12T06:40:51.844 INFO:teuthology.orchestra.run.trial028.stdout: debug crypto = 0/0 2026-02-12T06:40:51.845 INFO:teuthology.orchestra.run.trial028.stdout: debug dpdk = 0/0 2026-02-12T06:40:51.845 INFO:teuthology.orchestra.run.trial028.stdout: debug eventtrace = 0/0 2026-02-12T06:40:51.845 INFO:teuthology.orchestra.run.trial028.stdout: debug filer = 0/0 2026-02-12T06:40:51.845 INFO:teuthology.orchestra.run.trial028.stdout: debug filestore = 0/0 2026-02-12T06:40:51.845 INFO:teuthology.orchestra.run.trial028.stdout: debug finisher = 0/0 2026-02-12T06:40:51.845 INFO:teuthology.orchestra.run.trial028.stdout: debug fuse = 0/0 2026-02-12T06:40:51.845 INFO:teuthology.orchestra.run.trial028.stdout: debug heartbeatmap = 0/0 2026-02-12T06:40:51.845 INFO:teuthology.orchestra.run.trial028.stdout: debug javaclient = 0/0 2026-02-12T06:40:51.845 INFO:teuthology.orchestra.run.trial028.stdout: debug journal = 0/0 2026-02-12T06:40:51.845 INFO:teuthology.orchestra.run.trial028.stdout: debug journaler = 0/0 2026-02-12T06:40:51.845 INFO:teuthology.orchestra.run.trial028.stdout: debug kstore = 0/0 2026-02-12T06:40:51.845 INFO:teuthology.orchestra.run.trial028.stdout: debug leveldb = 0/0 2026-02-12T06:40:51.845 INFO:teuthology.orchestra.run.trial028.stdout: debug lockdep = 0/0 2026-02-12T06:40:51.846 INFO:teuthology.orchestra.run.trial028.stdout: debug mds = 0/0 2026-02-12T06:40:51.846 INFO:teuthology.orchestra.run.trial028.stdout: debug mds balancer = 0/0 2026-02-12T06:40:51.846 INFO:teuthology.orchestra.run.trial028.stdout: debug mds locker = 0/0 2026-02-12T06:40:51.846 INFO:teuthology.orchestra.run.trial028.stdout: debug mds log = 0/0 2026-02-12T06:40:51.846 INFO:teuthology.orchestra.run.trial028.stdout: debug mds log expire = 0/0 2026-02-12T06:40:51.846 INFO:teuthology.orchestra.run.trial028.stdout: debug mds migrator = 0/0 2026-02-12T06:40:51.846 INFO:teuthology.orchestra.run.trial028.stdout: debug memdb = 0/0 2026-02-12T06:40:51.846 INFO:teuthology.orchestra.run.trial028.stdout: debug mgr = 0/0 2026-02-12T06:40:51.846 INFO:teuthology.orchestra.run.trial028.stdout: debug mgrc = 0/0 2026-02-12T06:40:51.846 INFO:teuthology.orchestra.run.trial028.stdout: debug mon = 0/0 2026-02-12T06:40:51.846 INFO:teuthology.orchestra.run.trial028.stdout: debug monc = 0/0 2026-02-12T06:40:51.846 INFO:teuthology.orchestra.run.trial028.stdout: debug ms = 0/0 2026-02-12T06:40:51.846 INFO:teuthology.orchestra.run.trial028.stdout: debug objclass = 0/0 2026-02-12T06:40:51.847 INFO:teuthology.orchestra.run.trial028.stdout: debug objectcacher = 0/0 2026-02-12T06:40:51.847 INFO:teuthology.orchestra.run.trial028.stdout: debug objecter = 0/0 2026-02-12T06:40:51.847 INFO:teuthology.orchestra.run.trial028.stdout: debug optracker = 0/0 2026-02-12T06:40:51.847 INFO:teuthology.orchestra.run.trial028.stdout: debug osd = 0/0 2026-02-12T06:40:51.847 INFO:teuthology.orchestra.run.trial028.stdout: debug paxos = 0/0 2026-02-12T06:40:51.847 INFO:teuthology.orchestra.run.trial028.stdout: debug perfcounter = 0/0 2026-02-12T06:40:51.847 INFO:teuthology.orchestra.run.trial028.stdout: debug rados = 0/0 2026-02-12T06:40:51.847 INFO:teuthology.orchestra.run.trial028.stdout: debug rbd = 0/0 2026-02-12T06:40:51.847 INFO:teuthology.orchestra.run.trial028.stdout: debug rbd mirror = 0/0 2026-02-12T06:40:51.847 INFO:teuthology.orchestra.run.trial028.stdout: debug rbd replay = 0/0 2026-02-12T06:40:51.847 INFO:teuthology.orchestra.run.trial028.stdout: debug refs = 0/0 2026-02-12T06:40:51.847 INFO:teuthology.orchestra.run.trial028.stdout: debug rgw = 0/0 2026-02-12T06:40:51.847 INFO:teuthology.orchestra.run.trial028.stdout: debug rgw lifecycle = 0/0 2026-02-12T06:40:51.848 INFO:teuthology.orchestra.run.trial028.stdout: debug rgw notification = 0/0 2026-02-12T06:40:51.848 INFO:teuthology.orchestra.run.trial028.stdout: debug rgw sync = 0/0 2026-02-12T06:40:51.848 INFO:teuthology.orchestra.run.trial028.stdout: debug rocksdb = 0/0 2026-02-12T06:40:51.848 INFO:teuthology.orchestra.run.trial028.stdout: debug striper = 0/0 2026-02-12T06:40:51.848 INFO:teuthology.orchestra.run.trial028.stdout: debug throttle = 0/0 2026-02-12T06:40:51.848 INFO:teuthology.orchestra.run.trial028.stdout: debug timer = 0/0 2026-02-12T06:40:51.848 INFO:teuthology.orchestra.run.trial028.stdout: debug tp = 0/0 2026-02-12T06:40:51.848 INFO:teuthology.orchestra.run.trial028.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-12T06:40:51.848 INFO:teuthology.orchestra.run.trial028.stdout: ms client mode = crc 2026-02-12T06:40:51.848 INFO:teuthology.orchestra.run.trial028.stdout: ms cluster mode = crc 2026-02-12T06:40:51.848 INFO:teuthology.orchestra.run.trial028.stdout: ms mon client mode = crc 2026-02-12T06:40:51.848 INFO:teuthology.orchestra.run.trial028.stdout: ms mon cluster mode = crc 2026-02-12T06:40:51.848 INFO:teuthology.orchestra.run.trial028.stdout: ms mon service mode = crc 2026-02-12T06:40:51.849 INFO:teuthology.orchestra.run.trial028.stdout: ms service mode = crc 2026-02-12T06:40:51.849 INFO:teuthology.orchestra.run.trial028.stdout: fsid = 0c019592-0202-4bc6-848f-fd37de9403a4 2026-02-12T06:40:51.849 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.849 INFO:teuthology.orchestra.run.trial028.stdout:[osd] 2026-02-12T06:40:51.849 INFO:teuthology.orchestra.run.trial028.stdout: osd journal size = 100 2026-02-12T06:40:51.849 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.849 INFO:teuthology.orchestra.run.trial028.stdout: osd scrub load threshold = 5.0 2026-02-12T06:40:51.849 INFO:teuthology.orchestra.run.trial028.stdout: osd scrub max interval = 600 2026-02-12T06:40:51.849 INFO:teuthology.orchestra.run.trial028.stdout: osd mclock profile = high_recovery_ops 2026-02-12T06:40:51.849 INFO:teuthology.orchestra.run.trial028.stdout: osd mclock skip benchmark = true 2026-02-12T06:40:51.849 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.849 INFO:teuthology.orchestra.run.trial028.stdout: osd recover clone overlap = true 2026-02-12T06:40:51.850 INFO:teuthology.orchestra.run.trial028.stdout: osd recovery max chunk = 1048576 2026-02-12T06:40:51.850 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.850 INFO:teuthology.orchestra.run.trial028.stdout: osd debug shutdown = true 2026-02-12T06:40:51.850 INFO:teuthology.orchestra.run.trial028.stdout: osd debug op order = true 2026-02-12T06:40:51.850 INFO:teuthology.orchestra.run.trial028.stdout: osd debug verify stray on activate = true 2026-02-12T06:40:51.850 INFO:teuthology.orchestra.run.trial028.stdout: osd debug trim objects = true 2026-02-12T06:40:51.850 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.850 INFO:teuthology.orchestra.run.trial028.stdout: osd open classes on start = true 2026-02-12T06:40:51.850 INFO:teuthology.orchestra.run.trial028.stdout: osd debug pg log writeout = true 2026-02-12T06:40:51.850 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.850 INFO:teuthology.orchestra.run.trial028.stdout: osd deep scrub update digest min age = 30 2026-02-12T06:40:51.850 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.850 INFO:teuthology.orchestra.run.trial028.stdout: osd map max advance = 10 2026-02-12T06:40:51.851 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.851 INFO:teuthology.orchestra.run.trial028.stdout: journal zero on create = true 2026-02-12T06:40:51.851 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.851 INFO:teuthology.orchestra.run.trial028.stdout: filestore ondisk finisher threads = 3 2026-02-12T06:40:51.851 INFO:teuthology.orchestra.run.trial028.stdout: filestore apply finisher threads = 3 2026-02-12T06:40:51.851 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.851 INFO:teuthology.orchestra.run.trial028.stdout: bdev debug aio = true 2026-02-12T06:40:51.851 INFO:teuthology.orchestra.run.trial028.stdout: osd debug misdirected ops = true 2026-02-12T06:40:51.851 INFO:teuthology.orchestra.run.trial028.stdout: debug monc = 20 2026-02-12T06:40:51.851 INFO:teuthology.orchestra.run.trial028.stdout: crimson cpu num = 4 2026-02-12T06:40:51.851 INFO:teuthology.orchestra.run.trial028.stdout: crimson osd obc lru size = 10 2026-02-12T06:40:51.851 INFO:teuthology.orchestra.run.trial028.stdout: debug filestore = 0/0 2026-02-12T06:40:51.851 INFO:teuthology.orchestra.run.trial028.stdout: debug journal = 0/0 2026-02-12T06:40:51.852 INFO:teuthology.orchestra.run.trial028.stdout: debug ms = 0/0 2026-02-12T06:40:51.852 INFO:teuthology.orchestra.run.trial028.stdout: debug osd = 0/0 2026-02-12T06:40:51.852 INFO:teuthology.orchestra.run.trial028.stdout: debug seastore = 20 2026-02-12T06:40:51.852 INFO:teuthology.orchestra.run.trial028.stdout: debug seastore backref = 20 2026-02-12T06:40:51.852 INFO:teuthology.orchestra.run.trial028.stdout: debug seastore cache = 20 2026-02-12T06:40:51.852 INFO:teuthology.orchestra.run.trial028.stdout: debug seastore cleaner = 20 2026-02-12T06:40:51.852 INFO:teuthology.orchestra.run.trial028.stdout: debug seastore device = 20 2026-02-12T06:40:51.852 INFO:teuthology.orchestra.run.trial028.stdout: debug seastore epm = 20 2026-02-12T06:40:51.852 INFO:teuthology.orchestra.run.trial028.stdout: debug seastore fixedkv tree = 20 2026-02-12T06:40:51.852 INFO:teuthology.orchestra.run.trial028.stdout: debug seastore journal = 20 2026-02-12T06:40:51.852 INFO:teuthology.orchestra.run.trial028.stdout: debug seastore lba = 20 2026-02-12T06:40:51.852 INFO:teuthology.orchestra.run.trial028.stdout: debug seastore odata = 20 2026-02-12T06:40:51.852 INFO:teuthology.orchestra.run.trial028.stdout: debug seastore omap = 20 2026-02-12T06:40:51.852 INFO:teuthology.orchestra.run.trial028.stdout: debug seastore onode = 20 2026-02-12T06:40:51.853 INFO:teuthology.orchestra.run.trial028.stdout: debug seastore t = 20 2026-02-12T06:40:51.853 INFO:teuthology.orchestra.run.trial028.stdout: debug seastore tm = 20 2026-02-12T06:40:51.853 INFO:teuthology.orchestra.run.trial028.stdout: osd objectstore = seastore 2026-02-12T06:40:51.853 INFO:teuthology.orchestra.run.trial028.stdout: osd shutdown pgref assert = True 2026-02-12T06:40:51.853 INFO:teuthology.orchestra.run.trial028.stdout: seastore cachepin size pershard = 64M 2026-02-12T06:40:51.853 INFO:teuthology.orchestra.run.trial028.stdout: seastore main device type = RANDOM_BLOCK_SSD 2026-02-12T06:40:51.853 INFO:teuthology.orchestra.run.trial028.stdout: seastore max concurrent transactions = 8 2026-02-12T06:40:51.853 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.853 INFO:teuthology.orchestra.run.trial028.stdout:[mgr] 2026-02-12T06:40:51.853 INFO:teuthology.orchestra.run.trial028.stdout: debug ms = 1 2026-02-12T06:40:51.853 INFO:teuthology.orchestra.run.trial028.stdout: debug mgr = 20 2026-02-12T06:40:51.853 INFO:teuthology.orchestra.run.trial028.stdout: debug mon = 20 2026-02-12T06:40:51.853 INFO:teuthology.orchestra.run.trial028.stdout: debug auth = 20 2026-02-12T06:40:51.854 INFO:teuthology.orchestra.run.trial028.stdout: mon reweight min pgs per osd = 4 2026-02-12T06:40:51.854 INFO:teuthology.orchestra.run.trial028.stdout: mon reweight min bytes per osd = 10 2026-02-12T06:40:51.854 INFO:teuthology.orchestra.run.trial028.stdout: mgr/telemetry/nag = false 2026-02-12T06:40:51.854 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.854 INFO:teuthology.orchestra.run.trial028.stdout:[mon] 2026-02-12T06:40:51.854 INFO:teuthology.orchestra.run.trial028.stdout: debug ms = 0/0 2026-02-12T06:40:51.854 INFO:teuthology.orchestra.run.trial028.stdout: debug mon = 0/0 2026-02-12T06:40:51.854 INFO:teuthology.orchestra.run.trial028.stdout: debug paxos = 0/0 2026-02-12T06:40:51.854 INFO:teuthology.orchestra.run.trial028.stdout: debug auth = 20 2026-02-12T06:40:51.854 INFO:teuthology.orchestra.run.trial028.stdout: mon data avail warn = 5 2026-02-12T06:40:51.854 INFO:teuthology.orchestra.run.trial028.stdout: mon mgr mkfs grace = 240 2026-02-12T06:40:51.854 INFO:teuthology.orchestra.run.trial028.stdout: mon reweight min pgs per osd = 4 2026-02-12T06:40:51.855 INFO:teuthology.orchestra.run.trial028.stdout: mon osd reporter subtree level = osd 2026-02-12T06:40:51.855 INFO:teuthology.orchestra.run.trial028.stdout: mon osd prime pg temp = true 2026-02-12T06:40:51.855 INFO:teuthology.orchestra.run.trial028.stdout: mon reweight min bytes per osd = 10 2026-02-12T06:40:51.855 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.855 INFO:teuthology.orchestra.run.trial028.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-12T06:40:51.855 INFO:teuthology.orchestra.run.trial028.stdout: auth mon ticket ttl = 660 # 11m 2026-02-12T06:40:51.855 INFO:teuthology.orchestra.run.trial028.stdout: auth service ticket ttl = 240 # 4m 2026-02-12T06:40:51.855 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.855 INFO:teuthology.orchestra.run.trial028.stdout: # don't complain about insecure global_id in the test suite 2026-02-12T06:40:51.855 INFO:teuthology.orchestra.run.trial028.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-12T06:40:51.855 INFO:teuthology.orchestra.run.trial028.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-12T06:40:51.855 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.855 INFO:teuthology.orchestra.run.trial028.stdout: # 1m isn't quite enough 2026-02-12T06:40:51.856 INFO:teuthology.orchestra.run.trial028.stdout: mon_down_mkfs_grace = 2m 2026-02-12T06:40:51.856 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.856 INFO:teuthology.orchestra.run.trial028.stdout: mon_warn_on_filestore_osds = false 2026-02-12T06:40:51.856 INFO:teuthology.orchestra.run.trial028.stdout: osd pool default crimson = True 2026-02-12T06:40:51.856 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:40:51.856 INFO:teuthology.orchestra.run.trial028.stdout:[client] 2026-02-12T06:40:51.856 INFO:teuthology.orchestra.run.trial028.stdout: rgw cache enabled = true 2026-02-12T06:40:51.856 INFO:teuthology.orchestra.run.trial028.stdout: rgw enable ops log = true 2026-02-12T06:40:51.856 INFO:teuthology.orchestra.run.trial028.stdout: rgw enable usage log = true 2026-02-12T06:40:51.856 INFO:teuthology.orchestra.run.trial028.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-12T06:40:51.856 INFO:teuthology.orchestra.run.trial028.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-12T06:40:51.856 INFO:teuthology.orchestra.run.trial028.stdout:[mon.a] 2026-02-12T06:40:51.858 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-12T06:40:51.858 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-12T06:40:51.953 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-12T06:40:51.953 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:40:51.953 DEBUG:teuthology.orchestra.run.trial028:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-12T06:40:51.970 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:40:51.970 DEBUG:teuthology.orchestra.run.trial028:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-12T06:40:52.026 INFO:tasks.ceph:Sending monmap to node ubuntu@trial028.front.sepia.ceph.com 2026-02-12T06:40:52.027 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:40:52.027 DEBUG:teuthology.orchestra.run.trial028:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-12T06:40:52.027 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-12T06:40:52.100 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:40:52.100 DEBUG:teuthology.orchestra.run.trial028:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-12T06:40:52.157 INFO:tasks.ceph:Setting up mon nodes... 2026-02-12T06:40:52.157 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-12T06:40:52.157 DEBUG:teuthology.orchestra.run.trial028:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-12T06:40:52.248 INFO:teuthology.orchestra.run.trial028.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-12T06:40:52.251 INFO:tasks.ceph:Setting up mds nodes... 2026-02-12T06:40:52.251 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-12T06:40:52.251 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-12T06:40:52.292 INFO:teuthology.orchestra.run.trial028.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-12T06:40:52.305 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-12T06:40:52.305 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial028.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-12T06:40:52.306 DEBUG:teuthology.orchestra.run.trial028:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-12T06:40:52.373 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-12T06:40:52.373 INFO:tasks.ceph:role: osd.0 2026-02-12T06:40:52.373 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial028.front.sepia.ceph.com 2026-02-12T06:40:52.373 DEBUG:teuthology.orchestra.run.trial028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-12T06:40:52.438 INFO:teuthology.orchestra.run.trial028.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-12T06:40:52.438 INFO:teuthology.orchestra.run.trial028.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-12T06:40:52.438 INFO:teuthology.orchestra.run.trial028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-12T06:40:52.439 INFO:teuthology.orchestra.run.trial028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-12T06:40:52.439 INFO:teuthology.orchestra.run.trial028.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-12T06:40:52.439 INFO:teuthology.orchestra.run.trial028.stdout: = sunit=0 swidth=0 blks 2026-02-12T06:40:52.439 INFO:teuthology.orchestra.run.trial028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-12T06:40:52.439 INFO:teuthology.orchestra.run.trial028.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-12T06:40:52.439 INFO:teuthology.orchestra.run.trial028.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-12T06:40:52.439 INFO:teuthology.orchestra.run.trial028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-12T06:40:53.012 INFO:teuthology.orchestra.run.trial028.stdout:Discarding blocks...Done. 2026-02-12T06:40:53.015 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial028.front.sepia.ceph.com -o noatime 2026-02-12T06:40:53.015 DEBUG:teuthology.orchestra.run.trial028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-12T06:40:53.089 DEBUG:teuthology.orchestra.run.trial028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-12T06:40:53.159 DEBUG:teuthology.orchestra.run.trial028:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-12T06:40:53.223 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-12T06:40:53.223 INFO:tasks.ceph:role: osd.1 2026-02-12T06:40:53.223 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial028.front.sepia.ceph.com 2026-02-12T06:40:53.223 DEBUG:teuthology.orchestra.run.trial028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-12T06:40:53.289 INFO:teuthology.orchestra.run.trial028.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-12T06:40:53.289 INFO:teuthology.orchestra.run.trial028.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-12T06:40:53.289 INFO:teuthology.orchestra.run.trial028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-12T06:40:53.289 INFO:teuthology.orchestra.run.trial028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-12T06:40:53.289 INFO:teuthology.orchestra.run.trial028.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-12T06:40:53.289 INFO:teuthology.orchestra.run.trial028.stdout: = sunit=0 swidth=0 blks 2026-02-12T06:40:53.290 INFO:teuthology.orchestra.run.trial028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-12T06:40:53.290 INFO:teuthology.orchestra.run.trial028.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-12T06:40:53.290 INFO:teuthology.orchestra.run.trial028.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-12T06:40:53.290 INFO:teuthology.orchestra.run.trial028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-12T06:40:53.847 INFO:teuthology.orchestra.run.trial028.stdout:Discarding blocks...Done. 2026-02-12T06:40:53.849 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial028.front.sepia.ceph.com -o noatime 2026-02-12T06:40:53.849 DEBUG:teuthology.orchestra.run.trial028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-12T06:40:53.916 DEBUG:teuthology.orchestra.run.trial028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-12T06:40:53.984 DEBUG:teuthology.orchestra.run.trial028:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-12T06:40:54.049 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-12T06:40:54.049 INFO:tasks.ceph:role: osd.2 2026-02-12T06:40:54.049 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial028.front.sepia.ceph.com 2026-02-12T06:40:54.049 DEBUG:teuthology.orchestra.run.trial028:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-12T06:40:54.114 INFO:teuthology.orchestra.run.trial028.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-12T06:40:54.114 INFO:teuthology.orchestra.run.trial028.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-12T06:40:54.114 INFO:teuthology.orchestra.run.trial028.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-12T06:40:54.114 INFO:teuthology.orchestra.run.trial028.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-12T06:40:54.115 INFO:teuthology.orchestra.run.trial028.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-12T06:40:54.115 INFO:teuthology.orchestra.run.trial028.stdout: = sunit=0 swidth=0 blks 2026-02-12T06:40:54.115 INFO:teuthology.orchestra.run.trial028.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-12T06:40:54.115 INFO:teuthology.orchestra.run.trial028.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-12T06:40:54.115 INFO:teuthology.orchestra.run.trial028.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-12T06:40:54.115 INFO:teuthology.orchestra.run.trial028.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-12T06:40:54.674 INFO:teuthology.orchestra.run.trial028.stdout:Discarding blocks...Done. 2026-02-12T06:40:54.676 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial028.front.sepia.ceph.com -o noatime 2026-02-12T06:40:54.677 DEBUG:teuthology.orchestra.run.trial028:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-12T06:40:54.745 DEBUG:teuthology.orchestra.run.trial028:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-12T06:40:54.811 DEBUG:teuthology.orchestra.run.trial028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-12T06:40:54.961 INFO:teuthology.orchestra.run.trial028.stderr:INFO  osd - OSD::main: parsing early config 2026-02-12T06:40:55.012 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:40:55,012 seastar - Reactor backend: linux-aio 2026-02-12T06:40:55.028 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:40:55,028 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T06:40:55.028 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:40:55,028 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T06:40:55.030 INFO:teuthology.orchestra.run.trial028.stderr:==21297==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T06:40:55.090 INFO:teuthology.orchestra.run.trial028.stderr:DEBUG 2026-02-12 06:40:55,089 [shard 0:main] seastar - reactor::drain 2026-02-12T06:40:57.179 INFO:teuthology.orchestra.run.trial028.stderr:----------------------------------------------------- 2026-02-12T06:40:57.180 INFO:teuthology.orchestra.run.trial028.stderr:Suppressions used: 2026-02-12T06:40:57.180 INFO:teuthology.orchestra.run.trial028.stderr: count bytes template 2026-02-12T06:40:57.180 INFO:teuthology.orchestra.run.trial028.stderr: 1 8 MallocExtension::Register 2026-02-12T06:40:57.180 INFO:teuthology.orchestra.run.trial028.stderr:----------------------------------------------------- 2026-02-12T06:40:57.180 INFO:teuthology.orchestra.run.trial028.stderr: 2026-02-12T06:40:57.198 INFO:teuthology.orchestra.run.trial028.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-12T06:40:57.198 INFO:teuthology.orchestra.run.trial028.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-12T06:40:57.239 INFO:teuthology.orchestra.run.trial028.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-12T06:40:57.239 INFO:teuthology.orchestra.run.trial028.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-12T06:40:57.246 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:40:57,246 seastar - Reactor backend: linux-aio 2026-02-12T06:40:57.258 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:40:57,257 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T06:40:57.258 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:40:57,257 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T06:40:57.261 INFO:teuthology.orchestra.run.trial028.stderr:==21295==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T06:40:57.261 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:40:57,261 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-12T06:40:57.399 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f 0x9d5ea2a 0x9d68edb 0x78a0be9 0x7bb2209 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:40:57.399 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7fc 0x9d3b001 0x9d3b111 0x7bb2246 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:40:57.399 INFO:teuthology.orchestra.run.trial028.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff96c1f796 0xffffffff96c1fb70 0xffffffff96a9525e 0xffffffff96de6239 0xffffffff96dc97a5 0xffffffff96dd3b7b 0xffffffff96dd3fdb 0xffffffff96a910d2 0xffffffff9774d451 0xffffffff97800ba2 2026-02-12T06:40:57.400 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f 0x1092b93 0xf2fdae 0x9c70283 0x9c7074d 0x9c709c0 0x9c70c76 0x9c71238 0x9c796df 0x9c79dbf 0x9d6290c 0x9d66fb7 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:40:57.549 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65f2 /lib64/libasan.so.8+0x151b7 0x9cbbb5d 0x9d27716 0x9d27f10 0x9d36fa6 0x9d39a31 0x9d39c42 0x9d39cee 0x9d61e0d 0x9d66fb7 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:40:57.549 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1197c0 /lib64/libasan.so.8+0xfd269 /lib64/libasan.so.8+0xee879 0x9b44d0b 0x9bc24a6 0x9bc25bc 0x9bc2698 0x9bc2d54 0x9bc2ebb 0x9bc32bf 0x9bc357c 0x9d65185 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:40:57.549 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119594 /lib64/libasan.so.8+0xfd269 /lib64/libasan.so.8+0xef999 0xbb7f5c 0xbb7fd6 0xbb80ef 0x9d45a65 0x9d622cf 0x9d66fb7 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:40:59.658 INFO:teuthology.orchestra.run.trial028.stderr:----------------------------------------------------- 2026-02-12T06:40:59.658 INFO:teuthology.orchestra.run.trial028.stderr:Suppressions used: 2026-02-12T06:40:59.658 INFO:teuthology.orchestra.run.trial028.stderr: count bytes template 2026-02-12T06:40:59.658 INFO:teuthology.orchestra.run.trial028.stderr: 1 8 MallocExtension::Register 2026-02-12T06:40:59.658 INFO:teuthology.orchestra.run.trial028.stderr:----------------------------------------------------- 2026-02-12T06:40:59.658 INFO:teuthology.orchestra.run.trial028.stderr: 2026-02-12T06:40:59.690 DEBUG:teuthology.orchestra.run.trial028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-12T06:40:59.715 DEBUG:teuthology.orchestra.run.trial028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-12T06:40:59.871 INFO:teuthology.orchestra.run.trial028.stderr:INFO  osd - OSD::main: parsing early config 2026-02-12T06:40:59.932 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:40:59,931 seastar - Reactor backend: linux-aio 2026-02-12T06:40:59.942 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:40:59,942 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T06:40:59.943 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:40:59,942 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T06:40:59.944 INFO:teuthology.orchestra.run.trial028.stderr:==21363==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T06:41:00.003 INFO:teuthology.orchestra.run.trial028.stderr:DEBUG 2026-02-12 06:41:00,003 [shard 0:main] seastar - reactor::drain 2026-02-12T06:41:02.089 INFO:teuthology.orchestra.run.trial028.stderr:----------------------------------------------------- 2026-02-12T06:41:02.089 INFO:teuthology.orchestra.run.trial028.stderr:Suppressions used: 2026-02-12T06:41:02.089 INFO:teuthology.orchestra.run.trial028.stderr: count bytes template 2026-02-12T06:41:02.089 INFO:teuthology.orchestra.run.trial028.stderr: 1 8 MallocExtension::Register 2026-02-12T06:41:02.089 INFO:teuthology.orchestra.run.trial028.stderr:----------------------------------------------------- 2026-02-12T06:41:02.089 INFO:teuthology.orchestra.run.trial028.stderr: 2026-02-12T06:41:02.108 INFO:teuthology.orchestra.run.trial028.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-12T06:41:02.108 INFO:teuthology.orchestra.run.trial028.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-12T06:41:02.150 INFO:teuthology.orchestra.run.trial028.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-12T06:41:02.150 INFO:teuthology.orchestra.run.trial028.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-12T06:41:02.158 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:41:02,157 seastar - Reactor backend: linux-aio 2026-02-12T06:41:02.167 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:41:02,167 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T06:41:02.168 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:41:02,167 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T06:41:02.171 INFO:teuthology.orchestra.run.trial028.stderr:==21361==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T06:41:02.171 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:41:02,171 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-12T06:41:02.310 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f 0x4d43c72 0x9c7ef5d 0x9d45227 0x9d453ef 0x9d45550 0x9d45a44 0x9d622cf 0x9d66fb7 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:41:02.310 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191c88 /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7fc 0x9d3b001 0x9d3b111 0x7bb284b 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:41:02.310 INFO:teuthology.orchestra.run.trial028.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff971280f7 0xffffffff96df7fa6 0xffffffff96dfb960 0xffffffff96dfccc1 0xffffffff96e24923 0xffffffff96e24ae5 0xffffffff96dcb89e 0xffffffff96dd3b7b 0xffffffff96dd3fdb 0xffffffff96a910d2 0xffffffff9774d451 0xffffffff97800ba2 2026-02-12T06:41:02.310 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc08 /lib64/libasan.so.8+0x1006bc /lib64/libasan.so.8+0xc6619 /lib64/libasan.so.8+0x157e7 0x1092cf7 0xf2fdae 0x9c70450 0x9c7074d 0x9c709c0 0x9c70c76 0x9c71725 0x9c71c23 0x9c72253 0x9c723bd 0x9d5272d 0x9d65530 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:41:02.456 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x158c9 0x1092cf7 0xf2fdae 0x9c70283 0x9c7074d 0x9c709c0 0x9c70c76 0x9c71238 0x9c796df 0x9c79dbf 0x9d6290c 0x9d66fb7 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:41:02.456 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc6631 /lib64/libasan.so.8+0x157e7 0x1092cf7 0xf2fdae 0x9c70283 0x9c7074d 0x9c709c0 0x9c719c0 0x9c79a43 0x9c79dbf 0x9d6290c 0x9d66fb7 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:41:02.456 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f 0x316b49a 0x9c7070e 0x9c709c0 0x9c70c76 0x9c71238 0x9c796df 0x9c79dbf 0x9d6290c 0x9d66fb7 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:41:04.628 INFO:teuthology.orchestra.run.trial028.stderr:----------------------------------------------------- 2026-02-12T06:41:04.628 INFO:teuthology.orchestra.run.trial028.stderr:Suppressions used: 2026-02-12T06:41:04.628 INFO:teuthology.orchestra.run.trial028.stderr: count bytes template 2026-02-12T06:41:04.628 INFO:teuthology.orchestra.run.trial028.stderr: 1 8 MallocExtension::Register 2026-02-12T06:41:04.628 INFO:teuthology.orchestra.run.trial028.stderr:----------------------------------------------------- 2026-02-12T06:41:04.628 INFO:teuthology.orchestra.run.trial028.stderr: 2026-02-12T06:41:04.660 DEBUG:teuthology.orchestra.run.trial028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-12T06:41:04.686 DEBUG:teuthology.orchestra.run.trial028:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-12T06:41:04.837 INFO:teuthology.orchestra.run.trial028.stderr:INFO  osd - OSD::main: parsing early config 2026-02-12T06:41:04.895 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:41:04,894 seastar - Reactor backend: linux-aio 2026-02-12T06:41:04.906 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:41:04,906 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T06:41:04.906 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:41:04,906 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T06:41:04.909 INFO:teuthology.orchestra.run.trial028.stderr:==21429==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T06:41:04.968 INFO:teuthology.orchestra.run.trial028.stderr:DEBUG 2026-02-12 06:41:04,968 [shard 0:main] seastar - reactor::drain 2026-02-12T06:41:07.073 INFO:teuthology.orchestra.run.trial028.stderr:----------------------------------------------------- 2026-02-12T06:41:07.073 INFO:teuthology.orchestra.run.trial028.stderr:Suppressions used: 2026-02-12T06:41:07.073 INFO:teuthology.orchestra.run.trial028.stderr: count bytes template 2026-02-12T06:41:07.074 INFO:teuthology.orchestra.run.trial028.stderr: 1 8 MallocExtension::Register 2026-02-12T06:41:07.074 INFO:teuthology.orchestra.run.trial028.stderr:----------------------------------------------------- 2026-02-12T06:41:07.074 INFO:teuthology.orchestra.run.trial028.stderr: 2026-02-12T06:41:07.093 INFO:teuthology.orchestra.run.trial028.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-12T06:41:07.093 INFO:teuthology.orchestra.run.trial028.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-12T06:41:07.135 INFO:teuthology.orchestra.run.trial028.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-12T06:41:07.135 INFO:teuthology.orchestra.run.trial028.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-12T06:41:07.141 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:41:07,141 seastar - Reactor backend: linux-aio 2026-02-12T06:41:07.150 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:41:07,150 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T06:41:07.150 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:41:07,150 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T06:41:07.154 INFO:teuthology.orchestra.run.trial028.stderr:==21427==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T06:41:07.154 INFO:teuthology.orchestra.run.trial028.stderr:INFO 2026-02-12 06:41:07,154 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-12T06:41:07.293 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f 0x316b450 0x9c706c6 0x9c709c0 0x9c70c76 0x9c71725 0x9c71c23 0x9c72253 0x9c723bd 0x9d5272d 0x9d65530 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:41:07.293 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f 0x316b53e 0x9c7072e 0x9c709c0 0x9c70c76 0x9c71238 0x9c796df 0x9c79dbf 0x9d6290c 0x9d66fb7 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:41:07.293 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65f2 /lib64/libasan.so.8+0x157e7 0xc96a72 0xca7fdd 0x9d6342f 0x9d66fb7 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f93ace 0x6f978cf 0x6f9d479 0x74115d6 0x7433c3e 0x7283c37 0x7283d2f 0x7354fe2 0x735509b 0x7355252 0x7433cbf 0x7283dac 0x7283e31 0x72afa7b 0x741c466 0x741c4eb 0x741c5c9 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:41:07.438 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f 0xcb120d 0x1092bd4 0xf2fdae 0x9c70283 0x9c7074d 0x9c709c0 0x9c70c76 0x9c71725 0x9c71c23 0x9c72253 0x9c723bd 0x9d5272d 0x9d65530 0x9d27585 0x9d26585 0x7bb28d7 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:41:07.438 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1585b 0x9cef41b 0x9cef647 0x9d681d3 0x9d68320 0x9d683b3 0x9d691f8 0x78a0be9 0x7bb2209 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:41:07.438 INFO:teuthology.orchestra.run.trial028.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e4796c 0x9e47c55 0x9ea85f3 0x9ea8de6 0x9ea9054 0x9ea919c 0x9ea965b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x11978f /lib64/libasan.so.8+0xfd269 /lib64/libasan.so.8+0xedcd5 0xe1e15b 0x9bc2044 0x9bc2188 0x9bc226d 0x9bc24c9 0x9bc25bc 0x9bc2698 0x9bc2914 0x9bc2ebb 0x9bc32bf 0x9bc357c 0x9bc43c4 0x9d3b018 0x9d3b111 0x7bb2246 0x7bbc3a0 0x7a3ca55 0x7a7cdbd 0x7113237 0x6f3791e 0x6f39c86 0x747c276 0x729b1dd 0x729b2df 0x729b3d7 0x73541ae 0x7354267 0x735441a 0x7354552 0x73545cd 0x7354652 0x73547ad 0x741bee3 0x747c2ff 0x7282672 0x729b554 0x73a9088 0x74287fc 0x7428877 0x74288f2 0x74ec4a2 0x75da55c 0x9f024bb 0x9e56790 0x9f2ff2b 0x9f9870e 0x9fb319e 0x9fb381a 0x26ed857 0xa5c1720 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T06:41:09.625 INFO:teuthology.orchestra.run.trial028.stderr:----------------------------------------------------- 2026-02-12T06:41:09.625 INFO:teuthology.orchestra.run.trial028.stderr:Suppressions used: 2026-02-12T06:41:09.625 INFO:teuthology.orchestra.run.trial028.stderr: count bytes template 2026-02-12T06:41:09.625 INFO:teuthology.orchestra.run.trial028.stderr: 1 8 MallocExtension::Register 2026-02-12T06:41:09.625 INFO:teuthology.orchestra.run.trial028.stderr:----------------------------------------------------- 2026-02-12T06:41:09.625 INFO:teuthology.orchestra.run.trial028.stderr: 2026-02-12T06:41:09.656 DEBUG:teuthology.orchestra.run.trial028:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-12T06:41:09.682 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-12T06:41:09.683 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:41:09.683 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-12T06:41:09.747 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:41:09.748 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-12T06:41:09.812 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:41:09.812 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-12T06:41:09.876 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:41:09.876 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-12T06:41:09.940 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:41:09.940 DEBUG:teuthology.orchestra.run.trial028:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-12T06:41:09.997 INFO:tasks.ceph:Adding keys to all mons... 2026-02-12T06:41:09.997 DEBUG:teuthology.orchestra.run.trial028:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-12T06:41:10.061 INFO:teuthology.orchestra.run.trial028.stdout:[mgr.x] 2026-02-12T06:41:10.061 INFO:teuthology.orchestra.run.trial028.stdout: key = AQD0dY1pTEnKDhAAL/CmNMZV4cJZE1VnkcO/Fw== 2026-02-12T06:41:10.061 INFO:teuthology.orchestra.run.trial028.stdout:[osd.0] 2026-02-12T06:41:10.061 INFO:teuthology.orchestra.run.trial028.stdout: key = AQD5dY1pa89cExAAvXyyO6oM5j425YWiziT5bg== 2026-02-12T06:41:10.061 INFO:teuthology.orchestra.run.trial028.stdout:[osd.1] 2026-02-12T06:41:10.061 INFO:teuthology.orchestra.run.trial028.stdout: key = AQD+dY1p4CIDDhAA7kAbGjrn5OtwGnCaB6Q5IA== 2026-02-12T06:41:10.061 INFO:teuthology.orchestra.run.trial028.stdout:[osd.2] 2026-02-12T06:41:10.061 INFO:teuthology.orchestra.run.trial028.stdout: key = AQADdo1pRNMBDRAAYv+oKJcCS+hmEi0IqmAg6A== 2026-02-12T06:41:10.061 INFO:teuthology.orchestra.run.trial028.stdout:[client.0] 2026-02-12T06:41:10.061 INFO:teuthology.orchestra.run.trial028.stdout: key = AQD0dY1pkApuERAAmSIJEUryMTlgAH7LQDnoUg== 2026-02-12T06:41:10.062 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-12T06:41:10.151 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-12T06:41:10.200 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-12T06:41:10.248 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-12T06:41:10.298 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-12T06:41:10.348 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-12T06:41:10.348 DEBUG:teuthology.orchestra.run.trial028:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-12T06:41:10.372 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-12T06:41:10.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:10.458+0000 7f3985a38c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:10.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:10.458+0000 7f3985a38c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:10.466 DEBUG:teuthology.orchestra.run.trial028:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-12T06:41:10.492 DEBUG:teuthology.orchestra.run.trial028:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-12T06:41:10.550 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-12T06:41:10.550 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-12T06:41:10.551 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-12T06:41:10.592 INFO:tasks.ceph.mon.a:Started 2026-02-12T06:41:10.592 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-12T06:41:10.593 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-12T06:41:10.593 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-12T06:41:10.595 INFO:tasks.ceph.mgr.x:Started 2026-02-12T06:41:10.595 DEBUG:tasks.ceph:set 0 configs 2026-02-12T06:41:10.596 DEBUG:teuthology.orchestra.run.trial028:> sudo ceph --cluster ceph config dump 2026-02-12T06:41:10.658 INFO:tasks.ceph.mon.a.trial028.stderr:2026-02-12T06:41:10.656+0000 7fecf513bc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:10.658 INFO:tasks.ceph.mon.a.trial028.stderr:2026-02-12T06:41:10.656+0000 7fecf513bc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:10.658 INFO:tasks.ceph.mon.a.trial028.stderr:2026-02-12T06:41:10.656+0000 7fecf513bc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:10.659 INFO:tasks.ceph.mgr.x.trial028.stderr:2026-02-12T06:41:10.657+0000 7fbfc67c2fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:10.674 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:10.671+0000 7fd81a4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:10.675 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:10.674+0000 7fd81a4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:10.706 INFO:teuthology.orchestra.run.trial028.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-12T06:41:10.714 INFO:tasks.ceph:Setting crush tunables to default 2026-02-12T06:41:10.714 DEBUG:teuthology.orchestra.run.trial028:> sudo ceph --cluster ceph osd crush tunables default 2026-02-12T06:41:10.791 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:10.787+0000 7fae1e7c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:10.792 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:10.791+0000 7fae1e7c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:10.825 INFO:teuthology.orchestra.run.trial028.stderr:adjusted tunables profile to default 2026-02-12T06:41:10.833 INFO:tasks.ceph:check_enable_crimson: True 2026-02-12T06:41:10.834 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-12T06:41:10.834 DEBUG:teuthology.orchestra.run.trial028:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-12T06:41:10.861 INFO:tasks.ceph.mgr.x.trial028.stderr:2026-02-12T06:41:10.860+0000 7fbfc67c2fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:10.861 INFO:tasks.ceph.mgr.x.trial028.stderr:2026-02-12T06:41:10.860+0000 7fbfc67c2fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:10.908 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:10.905+0000 7f040f90b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:10.910 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:10.909+0000 7f040f90b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:10.948 INFO:tasks.ceph.mon.a.trial028.stderr:2026-02-12T06:41:10.947+0000 7feced3f1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-12T06:41:10.948 INFO:tasks.ceph.mon.a.trial028.stderr:Please be aware that this feature is experimental, untested, 2026-02-12T06:41:10.948 INFO:tasks.ceph.mon.a.trial028.stderr:unsupported, and may result in data corruption, data loss, 2026-02-12T06:41:10.948 INFO:tasks.ceph.mon.a.trial028.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-12T06:41:10.948 INFO:tasks.ceph.mon.a.trial028.stderr:feature with important data. 2026-02-12T06:41:10.948 INFO:tasks.ceph.mon.a.trial028.stderr: 2026-02-12T06:41:11.705 INFO:tasks.ceph.mgr.x.trial028.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-12T06:41:11.705 INFO:tasks.ceph.mgr.x.trial028.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-12T06:41:11.705 INFO:tasks.ceph.mgr.x.trial028.stderr: from numpy import show_config as show_numpy_config 2026-02-12T06:41:11.835 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-12T06:41:11.835 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:41:11.835 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-12T06:41:11.900 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:41:11.900 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-12T06:41:11.964 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:41:11.965 DEBUG:teuthology.orchestra.run.trial028:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-12T06:41:12.029 DEBUG:teuthology.orchestra.run.trial028:> sudo ceph --cluster ceph osd new 622d5584-9bbf-455b-b6c4-1180809d938f 0 2026-02-12T06:41:12.145 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:12.141+0000 7f3e1725a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:12.146 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:12.146+0000 7f3e15fd1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:12.184 INFO:teuthology.orchestra.run.trial028.stdout:0 2026-02-12T06:41:12.192 DEBUG:teuthology.orchestra.run.trial028:> sudo ceph --cluster ceph osd new 8a0ee4e2-299b-4e65-bda8-30b314b98136 1 2026-02-12T06:41:12.267 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:12.262+0000 7fb7ad88b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:12.268 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:12.267+0000 7fb7ad88b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:12.306 INFO:teuthology.orchestra.run.trial028.stdout:1 2026-02-12T06:41:12.314 DEBUG:teuthology.orchestra.run.trial028:> sudo ceph --cluster ceph osd new e10308ed-a0bd-408b-b259-97e8ff7a7c3a 2 2026-02-12T06:41:12.387 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:12.383+0000 7f767e0cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:12.388 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:12.387+0000 7f767e0cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:12.427 INFO:teuthology.orchestra.run.trial028.stdout:2 2026-02-12T06:41:12.436 DEBUG:teuthology.orchestra.run.trial028:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-12T06:41:12.463 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-12T06:41:12.464 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-12T06:41:12.506 INFO:tasks.ceph.osd.0:Started 2026-02-12T06:41:12.506 DEBUG:teuthology.orchestra.run.trial028:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-12T06:41:12.536 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-12T06:41:12.536 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-12T06:41:12.578 INFO:tasks.ceph.osd.1:Started 2026-02-12T06:41:12.579 DEBUG:teuthology.orchestra.run.trial028:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-12T06:41:12.606 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-12T06:41:12.606 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-12T06:41:12.648 INFO:tasks.ceph.osd.2:Started 2026-02-12T06:41:12.649 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-12T06:41:12.728 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:12.725+0000 7ffaba70b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:12.729 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:12.728+0000 7ffaba70b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:12.763 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:41:12.763 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":6,"fsid":"0c019592-0202-4bc6-848f-fd37de9403a4","created":"2026-02-12T06:41:10.667979+0000","modified":"2026-02-12T06:41:12.426893+0000","last_up_change":"0.000000","last_in_change":"2026-02-12T06:41:12.426893+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"622d5584-9bbf-455b-b6c4-1180809d938f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8a0ee4e2-299b-4e65-bda8-30b314b98136","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e10308ed-a0bd-408b-b259-97e8ff7a7c3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T06:41:12.772 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-12T06:41:12.772 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-12T06:41:13.074 DEBUG:teuthology.orchestra.run.trial028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-12T06:41:13.141 INFO:teuthology.misc.health.trial028.stderr:2026-02-12T06:41:13.139+0000 7f10c5851640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:13.142 INFO:teuthology.misc.health.trial028.stderr:2026-02-12T06:41:13.141+0000 7f10c5851640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:13.174 INFO:teuthology.misc.health.trial028.stdout: 2026-02-12T06:41:13.174 INFO:teuthology.misc.health.trial028.stdout:{"epoch":6,"fsid":"0c019592-0202-4bc6-848f-fd37de9403a4","created":"2026-02-12T06:41:10.667979+0000","modified":"2026-02-12T06:41:12.426893+0000","last_up_change":"0.000000","last_in_change":"2026-02-12T06:41:12.426893+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"622d5584-9bbf-455b-b6c4-1180809d938f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8a0ee4e2-299b-4e65-bda8-30b314b98136","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e10308ed-a0bd-408b-b259-97e8ff7a7c3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T06:41:13.181 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-12T06:41:14.108 INFO:tasks.ceph.mgr.x.trial028.stderr:2026-02-12T06:41:14.107+0000 7fbee172a640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-12T06:41:19.483 DEBUG:teuthology.orchestra.run.trial028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-12T06:41:19.552 INFO:teuthology.misc.health.trial028.stderr:2026-02-12T06:41:19.549+0000 7fe135ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:19.553 INFO:teuthology.misc.health.trial028.stderr:2026-02-12T06:41:19.552+0000 7fe135ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:19.661 INFO:teuthology.misc.health.trial028.stdout: 2026-02-12T06:41:19.661 INFO:teuthology.misc.health.trial028.stdout:{"epoch":7,"fsid":"0c019592-0202-4bc6-848f-fd37de9403a4","created":"2026-02-12T06:41:10.667979+0000","modified":"2026-02-12T06:41:18.835279+0000","last_up_change":"0.000000","last_in_change":"2026-02-12T06:41:12.426893+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"622d5584-9bbf-455b-b6c4-1180809d938f","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8a0ee4e2-299b-4e65-bda8-30b314b98136","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"e10308ed-a0bd-408b-b259-97e8ff7a7c3a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T06:41:19.669 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-12T06:41:25.972 DEBUG:teuthology.orchestra.run.trial028:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-12T06:41:26.043 INFO:teuthology.misc.health.trial028.stderr:2026-02-12T06:41:26.040+0000 7f034dc5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:26.044 INFO:teuthology.misc.health.trial028.stderr:2026-02-12T06:41:26.043+0000 7f034dc5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:26.154 INFO:teuthology.misc.health.trial028.stdout: 2026-02-12T06:41:26.154 INFO:teuthology.misc.health.trial028.stdout:{"epoch":13,"fsid":"0c019592-0202-4bc6-848f-fd37de9403a4","created":"2026-02-12T06:41:10.667979+0000","modified":"2026-02-12T06:41:25.858708+0000","last_up_change":"2026-02-12T06:41:21.845021+0000","last_in_change":"2026-02-12T06:41:12.426893+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-12T06:41:23.121023+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"622d5584-9bbf-455b-b6c4-1180809d938f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":1959968590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6807","nonce":1959968590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":1959968590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6811","nonce":1959968590}]},"public_addr":"10.20.193.28:6802/1959968590","cluster_addr":"10.20.193.28:6807/1959968590","heartbeat_back_addr":"10.20.193.28:6808/1959968590","heartbeat_front_addr":"10.20.193.28:6811/1959968590","state":["exists","up"]},{"osd":1,"uuid":"8a0ee4e2-299b-4e65-bda8-30b314b98136","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6803","nonce":1898615961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":1898615961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":1898615961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6813","nonce":1898615961}]},"public_addr":"10.20.193.28:6803/1898615961","cluster_addr":"10.20.193.28:6806/1898615961","heartbeat_back_addr":"10.20.193.28:6812/1898615961","heartbeat_front_addr":"10.20.193.28:6813/1898615961","state":["exists","up"]},{"osd":2,"uuid":"e10308ed-a0bd-408b-b259-97e8ff7a7c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":1529878095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6805","nonce":1529878095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":1529878095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6809","nonce":1529878095}]},"public_addr":"10.20.193.28:6804/1529878095","cluster_addr":"10.20.193.28:6805/1529878095","heartbeat_back_addr":"10.20.193.28:6810/1529878095","heartbeat_front_addr":"10.20.193.28:6809/1529878095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T06:41:26.162 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-12T06:41:26.162 INFO:tasks.ceph:Creating RBD pool 2026-02-12T06:41:26.162 DEBUG:teuthology.orchestra.run.trial028:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-12T06:41:26.238 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:26.234+0000 7fbb5899c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:26.239 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:26.239+0000 7fbb5899c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:26.866 INFO:teuthology.orchestra.run.trial028.stderr:pool 'rbd' created 2026-02-12T06:41:26.880 DEBUG:teuthology.orchestra.run.trial028:> rbd --cluster ceph pool init rbd 2026-02-12T06:41:26.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:26.922+0000 7f54aae10240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:26.923 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:26.922+0000 7f54aae10240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:26.924 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:26.924+0000 7f54aae10240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:29.900 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-12T06:41:29.901 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-12T06:41:29.902 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-12T06:41:30.018 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.014+0000 7f63bbd0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:30.019 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.018+0000 7f63bbd0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:30.128 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:41:30.137 INFO:teuthology.orchestra.run.trial028.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-12T06:41:30.137 INFO:tasks.ceph_manager:config epoch is 1 2026-02-12T06:41:30.137 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-12T06:41:30.137 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-12T06:41:30.138 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-12T06:41:30.215 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.212+0000 7f3d3d5fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:30.217 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.216+0000 7f3d3d5fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:30.368 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:41:30.377 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6800","nonce":1922936277},{"type":"v1","addr":"10.20.193.28:6801","nonce":1922936277}]},"active_addr":"10.20.193.28:6801/1922936277","active_change":"2026-02-12T06:41:13.102691+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.28:0","nonce":3295695284}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.28:0","nonce":289414187}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.28:0","nonce":3604264829}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.28:0","nonce":3257990851}]}]} 2026-02-12T06:41:30.381 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-12T06:41:30.381 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-12T06:41:30.382 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-12T06:41:30.458 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.455+0000 7f498f57b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:30.459 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.458+0000 7f498f57b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:30.570 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:41:30.570 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":17,"fsid":"0c019592-0202-4bc6-848f-fd37de9403a4","created":"2026-02-12T06:41:10.667979+0000","modified":"2026-02-12T06:41:29.872849+0000","last_up_change":"2026-02-12T06:41:21.845021+0000","last_in_change":"2026-02-12T06:41:12.426893+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-12T06:41:23.121023+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-12T06:41:26.351963+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"622d5584-9bbf-455b-b6c4-1180809d938f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":1959968590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6807","nonce":1959968590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":1959968590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6811","nonce":1959968590}]},"public_addr":"10.20.193.28:6802/1959968590","cluster_addr":"10.20.193.28:6807/1959968590","heartbeat_back_addr":"10.20.193.28:6808/1959968590","heartbeat_front_addr":"10.20.193.28:6811/1959968590","state":["exists","up"]},{"osd":1,"uuid":"8a0ee4e2-299b-4e65-bda8-30b314b98136","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6803","nonce":1898615961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":1898615961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":1898615961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6813","nonce":1898615961}]},"public_addr":"10.20.193.28:6803/1898615961","cluster_addr":"10.20.193.28:6806/1898615961","heartbeat_back_addr":"10.20.193.28:6812/1898615961","heartbeat_front_addr":"10.20.193.28:6813/1898615961","state":["exists","up"]},{"osd":2,"uuid":"e10308ed-a0bd-408b-b259-97e8ff7a7c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":1529878095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6805","nonce":1529878095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":1529878095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6809","nonce":1529878095}]},"public_addr":"10.20.193.28:6804/1529878095","cluster_addr":"10.20.193.28:6805/1529878095","heartbeat_back_addr":"10.20.193.28:6810/1529878095","heartbeat_front_addr":"10.20.193.28:6809/1529878095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T06:41:30.579 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-12T06:41:30.580 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-12T06:41:30.655 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.652+0000 7f2ca72c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:30.656 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.656+0000 7f2ca72c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:30.765 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:41:30.765 INFO:teuthology.orchestra.run.trial028.stdout:{"epoch":17,"fsid":"0c019592-0202-4bc6-848f-fd37de9403a4","created":"2026-02-12T06:41:10.667979+0000","modified":"2026-02-12T06:41:29.872849+0000","last_up_change":"2026-02-12T06:41:21.845021+0000","last_in_change":"2026-02-12T06:41:12.426893+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-12T06:41:23.121023+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-12T06:41:26.351963+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"622d5584-9bbf-455b-b6c4-1180809d938f","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6802","nonce":1959968590}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6807","nonce":1959968590}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6808","nonce":1959968590}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6811","nonce":1959968590}]},"public_addr":"10.20.193.28:6802/1959968590","cluster_addr":"10.20.193.28:6807/1959968590","heartbeat_back_addr":"10.20.193.28:6808/1959968590","heartbeat_front_addr":"10.20.193.28:6811/1959968590","state":["exists","up"]},{"osd":1,"uuid":"8a0ee4e2-299b-4e65-bda8-30b314b98136","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6803","nonce":1898615961}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6806","nonce":1898615961}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6812","nonce":1898615961}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6813","nonce":1898615961}]},"public_addr":"10.20.193.28:6803/1898615961","cluster_addr":"10.20.193.28:6806/1898615961","heartbeat_back_addr":"10.20.193.28:6812/1898615961","heartbeat_front_addr":"10.20.193.28:6813/1898615961","state":["exists","up"]},{"osd":2,"uuid":"e10308ed-a0bd-408b-b259-97e8ff7a7c3a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6804","nonce":1529878095}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6805","nonce":1529878095}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6810","nonce":1529878095}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.28:6809","nonce":1529878095}]},"public_addr":"10.20.193.28:6804/1529878095","cluster_addr":"10.20.193.28:6805/1529878095","heartbeat_back_addr":"10.20.193.28:6810/1529878095","heartbeat_front_addr":"10.20.193.28:6809/1529878095","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T06:41:30.775 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-12T06:41:30.776 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-12T06:41:30.777 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-12T06:41:30.858 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.855+0000 7f55cdd8c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:30.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.855+0000 7f0555c8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:30.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.856+0000 7f6b9fba4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:30.859 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.859+0000 7f55cdd8c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:30.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.859+0000 7f0555c8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:30.860 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.859+0000 7f6b9fba4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:30.876 INFO:teuthology.orchestra.run.trial028.stdout:42949672968 2026-02-12T06:41:30.877 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-12T06:41:30.878 INFO:teuthology.orchestra.run.trial028.stdout:38654705674 2026-02-12T06:41:30.878 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-12T06:41:30.880 INFO:teuthology.orchestra.run.trial028.stdout:42949672969 2026-02-12T06:41:30.881 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-12T06:41:30.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.994+0000 7fc91aa5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:30.998 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.994+0000 7efc8028b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:30.999 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.994+0000 7fe06865a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:31.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.998+0000 7fc91aa5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:31.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.998+0000 7efc7f002640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:31.000 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:30.998+0000 7fe06865a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:31.110 INFO:teuthology.orchestra.run.trial028.stdout:42949672968 2026-02-12T06:41:31.118 INFO:teuthology.orchestra.run.trial028.stdout:42949672969 2026-02-12T06:41:31.118 INFO:teuthology.orchestra.run.trial028.stdout:38654705674 2026-02-12T06:41:31.121 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672968 got 42949672968 for osd.1 2026-02-12T06:41:31.122 DEBUG:teuthology.parallel:result is None 2026-02-12T06:41:31.130 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672969 for osd.0 2026-02-12T06:41:31.130 DEBUG:teuthology.parallel:result is None 2026-02-12T06:41:31.131 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705674 for osd.2 2026-02-12T06:41:31.131 DEBUG:teuthology.parallel:result is None 2026-02-12T06:41:31.131 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-12T06:41:31.132 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-12T06:41:31.245 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:31.242+0000 7f409c4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:31.246 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:31.245+0000 7f409c4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:31.359 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:41:31.359 INFO:teuthology.orchestra.run.trial028.stderr:dumped all 2026-02-12T06:41:31.368 INFO:teuthology.orchestra.run.trial028.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-12T06:41:31.106878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62917020,"kb_used_data":62917020,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94369332,"statfs":{"total":161061224448,"available":96634195968,"internally_reserved":0,"allocated":64427028480,"data_stored":64427028480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.238939"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.885664+0000","last_change":"2026-02-12T06:41:29.888596+0000","last_active":"2026-02-12T06:41:29.885664+0000","last_peered":"2026-02-12T06:41:29.885664+0000","last_clean":"2026-02-12T06:41:29.885664+0000","last_became_active":"2026-02-12T06:41:27.903056+0000","last_became_peered":"2026-02-12T06:41:27.903056+0000","last_unstale":"2026-02-12T06:41:29.885664+0000","last_undegraded":"2026-02-12T06:41:29.885664+0000","last_fullsized":"2026-02-12T06:41:29.885664+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_clean_scrub_stamp":"2026-02-12T06:41:26.863511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.887206+0000","last_change":"2026-02-12T06:41:29.889678+0000","last_active":"2026-02-12T06:41:29.887206+0000","last_peered":"2026-02-12T06:41:29.887206+0000","last_clean":"2026-02-12T06:41:29.887206+0000","last_became_active":"2026-02-12T06:41:27.902381+0000","last_became_peered":"2026-02-12T06:41:27.902381+0000","last_unstale":"2026-02-12T06:41:29.887206+0000","last_undegraded":"2026-02-12T06:41:29.887206+0000","last_fullsized":"2026-02-12T06:41:29.887206+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_clean_scrub_stamp":"2026-02-12T06:41:26.863511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.887142+0000","last_change":"2026-02-12T06:41:29.890433+0000","last_active":"2026-02-12T06:41:29.887142+0000","last_peered":"2026-02-12T06:41:29.887142+0000","last_clean":"2026-02-12T06:41:29.887142+0000","last_became_active":"2026-02-12T06:41:27.906650+0000","last_became_peered":"2026-02-12T06:41:27.906650+0000","last_unstale":"2026-02-12T06:41:29.887142+0000","last_undegraded":"2026-02-12T06:41:29.887142+0000","last_fullsized":"2026-02-12T06:41:29.887142+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_clean_scrub_stamp":"2026-02-12T06:41:26.863511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.887140+0000","last_change":"2026-02-12T06:41:29.889115+0000","last_active":"2026-02-12T06:41:29.887140+0000","last_peered":"2026-02-12T06:41:29.887140+0000","last_clean":"2026-02-12T06:41:29.887140+0000","last_became_active":"2026-02-12T06:41:27.909601+0000","last_became_peered":"2026-02-12T06:41:27.909601+0000","last_unstale":"2026-02-12T06:41:29.887140+0000","last_undegraded":"2026-02-12T06:41:29.887140+0000","last_fullsized":"2026-02-12T06:41:29.887140+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_clean_scrub_stamp":"2026-02-12T06:41:26.863511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.885508+0000","last_change":"2026-02-12T06:41:29.885508+0000","last_active":"2026-02-12T06:41:29.885508+0000","last_peered":"2026-02-12T06:41:29.885508+0000","last_clean":"2026-02-12T06:41:29.885508+0000","last_became_active":"2026-02-12T06:41:27.905933+0000","last_became_peered":"2026-02-12T06:41:27.905933+0000","last_unstale":"2026-02-12T06:41:29.885508+0000","last_undegraded":"2026-02-12T06:41:29.885508+0000","last_fullsized":"2026-02-12T06:41:29.885508+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_clean_scrub_stamp":"2026-02-12T06:41:26.863511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.884582+0000","last_change":"2026-02-12T06:41:29.886640+0000","last_active":"2026-02-12T06:41:29.884582+0000","last_peered":"2026-02-12T06:41:29.884582+0000","last_clean":"2026-02-12T06:41:29.884582+0000","last_became_active":"2026-02-12T06:41:27.905701+0000","last_became_peered":"2026-02-12T06:41:27.905701+0000","last_unstale":"2026-02-12T06:41:29.884582+0000","last_undegraded":"2026-02-12T06:41:29.884582+0000","last_fullsized":"2026-02-12T06:41:29.884582+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_clean_scrub_stamp":"2026-02-12T06:41:26.863511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.882974+0000","last_change":"2026-02-12T06:41:29.884922+0000","last_active":"2026-02-12T06:41:29.882974+0000","last_peered":"2026-02-12T06:41:29.882974+0000","last_clean":"2026-02-12T06:41:29.882974+0000","last_became_active":"2026-02-12T06:41:27.909739+0000","last_became_peered":"2026-02-12T06:41:27.909739+0000","last_unstale":"2026-02-12T06:41:29.882974+0000","last_undegraded":"2026-02-12T06:41:29.882974+0000","last_fullsized":"2026-02-12T06:41:29.882974+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_clean_scrub_stamp":"2026-02-12T06:41:26.863511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.887203+0000","last_change":"2026-02-12T06:41:29.890342+0000","last_active":"2026-02-12T06:41:29.887203+0000","last_peered":"2026-02-12T06:41:29.887203+0000","last_clean":"2026-02-12T06:41:29.887203+0000","last_became_active":"2026-02-12T06:41:27.902272+0000","last_became_peered":"2026-02-12T06:41:27.902272+0000","last_unstale":"2026-02-12T06:41:29.887203+0000","last_undegraded":"2026-02-12T06:41:29.887203+0000","last_fullsized":"2026-02-12T06:41:29.887203+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_clean_scrub_stamp":"2026-02-12T06:41:26.863511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.885510+0000","last_change":"2026-02-12T06:41:24.886610+0000","last_active":"2026-02-12T06:41:29.885510+0000","last_peered":"2026-02-12T06:41:29.885510+0000","last_clean":"2026-02-12T06:41:29.885510+0000","last_became_active":"2026-02-12T06:41:24.879710+0000","last_became_peered":"2026-02-12T06:41:24.879710+0000","last_unstale":"2026-02-12T06:41:29.885510+0000","last_undegraded":"2026-02-12T06:41:29.885510+0000","last_fullsized":"2026-02-12T06:41:29.885510+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:23.849593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:23.849593+0000","last_clean_scrub_stamp":"2026-02-12T06:41:23.849593+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972620,"kb_used_data":20972620,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456164,"statfs":{"total":53687074816,"available":32211111936,"internally_reserved":0,"allocated":21475962880,"data_stored":21475962880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972580,"kb_used_data":20972580,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456204,"statfs":{"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971820,"kb_used_data":20971820,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456964,"statfs":{"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-12T06:41:31.370 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-12T06:41:31.446 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:31.443+0000 7feedccc2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:31.447 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:31.446+0000 7feedccc2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:31.556 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:41:31.557 INFO:teuthology.orchestra.run.trial028.stderr:dumped all 2026-02-12T06:41:31.565 INFO:teuthology.orchestra.run.trial028.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-12T06:41:31.106878+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62917020,"kb_used_data":62917020,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94369332,"statfs":{"total":161061224448,"available":96634195968,"internally_reserved":0,"allocated":64427028480,"data_stored":64427028480,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.238939"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.885664+0000","last_change":"2026-02-12T06:41:29.888596+0000","last_active":"2026-02-12T06:41:29.885664+0000","last_peered":"2026-02-12T06:41:29.885664+0000","last_clean":"2026-02-12T06:41:29.885664+0000","last_became_active":"2026-02-12T06:41:27.903056+0000","last_became_peered":"2026-02-12T06:41:27.903056+0000","last_unstale":"2026-02-12T06:41:29.885664+0000","last_undegraded":"2026-02-12T06:41:29.885664+0000","last_fullsized":"2026-02-12T06:41:29.885664+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_clean_scrub_stamp":"2026-02-12T06:41:26.863511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.887206+0000","last_change":"2026-02-12T06:41:29.889678+0000","last_active":"2026-02-12T06:41:29.887206+0000","last_peered":"2026-02-12T06:41:29.887206+0000","last_clean":"2026-02-12T06:41:29.887206+0000","last_became_active":"2026-02-12T06:41:27.902381+0000","last_became_peered":"2026-02-12T06:41:27.902381+0000","last_unstale":"2026-02-12T06:41:29.887206+0000","last_undegraded":"2026-02-12T06:41:29.887206+0000","last_fullsized":"2026-02-12T06:41:29.887206+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_clean_scrub_stamp":"2026-02-12T06:41:26.863511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.887142+0000","last_change":"2026-02-12T06:41:29.890433+0000","last_active":"2026-02-12T06:41:29.887142+0000","last_peered":"2026-02-12T06:41:29.887142+0000","last_clean":"2026-02-12T06:41:29.887142+0000","last_became_active":"2026-02-12T06:41:27.906650+0000","last_became_peered":"2026-02-12T06:41:27.906650+0000","last_unstale":"2026-02-12T06:41:29.887142+0000","last_undegraded":"2026-02-12T06:41:29.887142+0000","last_fullsized":"2026-02-12T06:41:29.887142+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_clean_scrub_stamp":"2026-02-12T06:41:26.863511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.887140+0000","last_change":"2026-02-12T06:41:29.889115+0000","last_active":"2026-02-12T06:41:29.887140+0000","last_peered":"2026-02-12T06:41:29.887140+0000","last_clean":"2026-02-12T06:41:29.887140+0000","last_became_active":"2026-02-12T06:41:27.909601+0000","last_became_peered":"2026-02-12T06:41:27.909601+0000","last_unstale":"2026-02-12T06:41:29.887140+0000","last_undegraded":"2026-02-12T06:41:29.887140+0000","last_fullsized":"2026-02-12T06:41:29.887140+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_clean_scrub_stamp":"2026-02-12T06:41:26.863511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.885508+0000","last_change":"2026-02-12T06:41:29.885508+0000","last_active":"2026-02-12T06:41:29.885508+0000","last_peered":"2026-02-12T06:41:29.885508+0000","last_clean":"2026-02-12T06:41:29.885508+0000","last_became_active":"2026-02-12T06:41:27.905933+0000","last_became_peered":"2026-02-12T06:41:27.905933+0000","last_unstale":"2026-02-12T06:41:29.885508+0000","last_undegraded":"2026-02-12T06:41:29.885508+0000","last_fullsized":"2026-02-12T06:41:29.885508+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_clean_scrub_stamp":"2026-02-12T06:41:26.863511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.884582+0000","last_change":"2026-02-12T06:41:29.886640+0000","last_active":"2026-02-12T06:41:29.884582+0000","last_peered":"2026-02-12T06:41:29.884582+0000","last_clean":"2026-02-12T06:41:29.884582+0000","last_became_active":"2026-02-12T06:41:27.905701+0000","last_became_peered":"2026-02-12T06:41:27.905701+0000","last_unstale":"2026-02-12T06:41:29.884582+0000","last_undegraded":"2026-02-12T06:41:29.884582+0000","last_fullsized":"2026-02-12T06:41:29.884582+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_clean_scrub_stamp":"2026-02-12T06:41:26.863511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.882974+0000","last_change":"2026-02-12T06:41:29.884922+0000","last_active":"2026-02-12T06:41:29.882974+0000","last_peered":"2026-02-12T06:41:29.882974+0000","last_clean":"2026-02-12T06:41:29.882974+0000","last_became_active":"2026-02-12T06:41:27.909739+0000","last_became_peered":"2026-02-12T06:41:27.909739+0000","last_unstale":"2026-02-12T06:41:29.882974+0000","last_undegraded":"2026-02-12T06:41:29.882974+0000","last_fullsized":"2026-02-12T06:41:29.882974+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_clean_scrub_stamp":"2026-02-12T06:41:26.863511+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.887203+0000","last_change":"2026-02-12T06:41:29.890342+0000","last_active":"2026-02-12T06:41:29.887203+0000","last_peered":"2026-02-12T06:41:29.887203+0000","last_clean":"2026-02-12T06:41:29.887203+0000","last_became_active":"2026-02-12T06:41:27.902272+0000","last_became_peered":"2026-02-12T06:41:27.902272+0000","last_unstale":"2026-02-12T06:41:29.887203+0000","last_undegraded":"2026-02-12T06:41:29.887203+0000","last_fullsized":"2026-02-12T06:41:29.887203+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:26.863511+0000","last_clean_scrub_stamp":"2026-02-12T06:41:26.863511+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-12T06:41:29.885510+0000","last_change":"2026-02-12T06:41:24.886610+0000","last_active":"2026-02-12T06:41:29.885510+0000","last_peered":"2026-02-12T06:41:29.885510+0000","last_clean":"2026-02-12T06:41:29.885510+0000","last_became_active":"2026-02-12T06:41:24.879710+0000","last_became_peered":"2026-02-12T06:41:24.879710+0000","last_unstale":"2026-02-12T06:41:29.885510+0000","last_undegraded":"2026-02-12T06:41:29.885510+0000","last_fullsized":"2026-02-12T06:41:29.885510+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T06:41:23.849593+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T06:41:23.849593+0000","last_clean_scrub_stamp":"2026-02-12T06:41:23.849593+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972620,"kb_used_data":20972620,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456164,"statfs":{"total":53687074816,"available":32211111936,"internally_reserved":0,"allocated":21475962880,"data_stored":21475962880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972580,"kb_used_data":20972580,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456204,"statfs":{"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971820,"kb_used_data":20971820,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456964,"statfs":{"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-12T06:41:31.567 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-12T06:41:31.567 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-12T06:41:31.567 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-12T06:41:31.568 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-12T06:41:31.643 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:31.641+0000 7fdf50f7b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:31.644 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:31.643+0000 7fdf50f7b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:31.780 INFO:teuthology.orchestra.run.trial028.stdout: 2026-02-12T06:41:31.780 INFO:teuthology.orchestra.run.trial028.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-12T06:41:31.790 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-12T06:41:31.790 INFO:teuthology.run_tasks:Running task ssh_keys... 2026-02-12T06:41:31.893 DEBUG:teuthology.orchestra.run.trial028:> echo 'StrictHostKeyChecking no 2026-02-12T06:41:31.893 DEBUG:teuthology.orchestra.run.trial028:> ' > /home/ubuntu/.ssh/config && echo 'UserKnownHostsFile ' /dev/null >> /home/ubuntu/.ssh/config && chmod 600 /home/ubuntu/.ssh/config 2026-02-12T06:41:31.909 INFO:teuthology.task.ssh_keys:generated public key AAAAB3NzaC1yc2EAAAADAQABAAABAQC+TIpNzdrD0Ls3/1DLEeSdutVn5n5HKenDHYr6iW8yARGIHdhAAjfgpNg3jNXL9yJe1OLMWFrqRCAicxDrHRDbN09yo35hlKKS4WTiTA4LzzlcAEK2teC7iAvocO52t0n4cgrcqJw/kIYlQH0i6+Bzsp1Q5/W1Mm39jnlo7C79YHH29fspFR4MxWTeIFzQpw85IeZQa3UFGIhcz57HLqYGS6Jo3WVb2vR4q5AERe+qlOETb60F/O2hdCdROx1qzuOwWODL2JWYmUexNbo54LON98yELJW846OLHnBoea1y1iz+6tMXuGgOTKwaGFgq7achXXpsdoFItQmYD9ptmNn1 2026-02-12T06:41:31.909 INFO:teuthology.task.ssh_keys:pushing keys to trial028.front.sepia.ceph.com for ubuntu 2026-02-12T06:41:31.909 DEBUG:teuthology.orchestra.run.trial028:> rm -f -- /home/ubuntu/.ssh/id_rsa 2026-02-12T06:41:31.964 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:41:31.964 DEBUG:teuthology.orchestra.run.trial028:> dd of=/home/ubuntu/.ssh/id_rsa 2026-02-12T06:41:31.964 DEBUG:teuthology.orchestra.run.trial028:> chmod 0500 /home/ubuntu/.ssh/id_rsa 2026-02-12T06:41:32.020 DEBUG:teuthology.orchestra.run.trial028:> rm -f -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-12T06:41:32.077 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:41:32.077 DEBUG:teuthology.orchestra.run.trial028:> dd of=/home/ubuntu/.ssh/id_rsa.pub 2026-02-12T06:41:32.133 DEBUG:teuthology.orchestra.run.trial028:> cp -v -a /home/ubuntu/.ssh/authorized_keys /home/ubuntu/.ssh/authorized_keys_2026-02-12_06:41:32:133780 2026-02-12T06:41:32.191 INFO:teuthology.orchestra.run.trial028.stdout:'/home/ubuntu/.ssh/authorized_keys' -> '/home/ubuntu/.ssh/authorized_keys_2026-02-12_06:41:32:133780' 2026-02-12T06:41:32.192 DEBUG:teuthology.orchestra.run.trial028:> set -ex 2026-02-12T06:41:32.192 DEBUG:teuthology.orchestra.run.trial028:> dd of=/home/ubuntu/.ssh/authorized_keys conv=notrunc oflag=append 2026-02-12T06:41:32.248 INFO:teuthology.run_tasks:Running task radosbench... 2026-02-12T06:41:32.268 INFO:tasks.radosbench:Beginning radosbench... 2026-02-12T06:41:32.268 INFO:tasks.radosbench:omap writes 2026-02-12T06:41:32.268 INFO:tasks.ceph.ceph_manager.ceph:creating pool_name unique_pool_0 2026-02-12T06:41:32.269 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool create unique_pool_0 16 2026-02-12T06:41:32.366 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:32.361+0000 7fb150f0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:32.367 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:32.366+0000 7fb150f0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:32.882 INFO:teuthology.orchestra.run.trial028.stderr:pool 'unique_pool_0' created 2026-02-12T06:41:32.894 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool application enable unique_pool_0 rados --yes-i-really-mean-it || true 2026-02-12T06:41:32.977 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:32.971+0000 7fbce6964640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:32.978 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:41:32.977+0000 7fbce6964640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:41:33.885 INFO:teuthology.orchestra.run.trial028.stderr:enabled application 'rados' on pool 'unique_pool_0' 2026-02-12T06:41:34.903 DEBUG:teuthology.orchestra.run.trial028:> /bin/sh -c 'adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage rados --no-log-to-stderr --name client.0 -t 16 -b 4096 --object-size 4096 -p unique_pool_0 bench 300 write --write-omap' 2026-02-12T06:41:34.907 DEBUG:teuthology.run_tasks:Unwinding manager radosbench 2026-02-12T06:41:34.924 INFO:tasks.radosbench:joining radosbench (timing out after 9300s) 2026-02-12T06:41:34.924 INFO:teuthology.orchestra.run:waiting for 9300 2026-02-12T06:41:34.952 INFO:tasks.radosbench.radosbench.0.trial028.stdout:hints = 1 2026-02-12T06:41:34.952 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Maintaining 16 concurrent writes of 4096 bytes to objects of size 4096 for up to 300 seconds or 0 objects 2026-02-12T06:41:34.952 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Object prefix: benchmark_data_trial028_23822 2026-02-12T06:41:34.952 INFO:tasks.radosbench.radosbench.0.trial028.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T06:41:34.953 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 0 0 0 0 0 0 - 0 2026-02-12T06:41:35.952 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 1 16 289 273 1.06631 1.06641 0.0635293 0.0546876 2026-02-12T06:41:36.952 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 2 16 584 568 1.10928 1.15234 0.0313997 0.0549231 2026-02-12T06:41:37.952 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 3 16 896 880 1.14574 1.21875 0.0270518 0.0534857 2026-02-12T06:41:38.952 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 4 16 1212 1196 1.16787 1.23438 0.0144918 0.0529508 2026-02-12T06:41:39.952 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 5 16 1532 1516 1.18428 1.25 0.0228263 0.0521853 2026-02-12T06:41:40.953 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 6 16 1866 1850 1.20433 1.30469 0.0256121 0.05172 2026-02-12T06:41:41.953 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 7 16 2193 2177 1.21474 1.27734 0.0421277 0.051097 2026-02-12T06:41:42.953 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 8 16 2493 2477 1.20937 1.17188 0.0443704 0.0515307 2026-02-12T06:41:43.953 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 9 16 2856 2840 1.23254 1.41797 0.0317744 0.0505379 2026-02-12T06:41:44.953 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 10 16 3192 3176 1.24052 1.3125 0.0498288 0.0502326 2026-02-12T06:41:45.953 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 11 16 3532 3516 1.24848 1.32812 0.0223592 0.0499165 2026-02-12T06:41:46.953 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 12 16 3880 3864 1.25771 1.35938 0.0368031 0.0495885 2026-02-12T06:41:47.953 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 13 16 4228 4212 1.26552 1.35938 0.0259117 0.0493205 2026-02-12T06:41:48.953 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 14 16 4552 4536 1.26552 1.26562 0.0720001 0.0492558 2026-02-12T06:41:49.953 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 15 16 4884 4868 1.26761 1.29688 0.0550112 0.0491972 2026-02-12T06:41:50.953 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 16 16 5224 5208 1.27138 1.32812 0.0334866 0.0490818 2026-02-12T06:41:51.953 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 17 16 5575 5559 1.27724 1.37109 0.088364 0.0488578 2026-02-12T06:41:52.954 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 18 16 5923 5907 1.2818 1.35938 0.0457504 0.0486869 2026-02-12T06:41:53.954 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 19 16 6276 6260 1.2869 1.37891 0.038055 0.0484767 2026-02-12T06:41:54.954 INFO:tasks.radosbench.radosbench.0.trial028.stdout:2026-02-12T06:41:54.953582+0000 min lat: 0.00991992 max lat: 0.511495 avg lat: 0.0483104 2026-02-12T06:41:54.954 INFO:tasks.radosbench.radosbench.0.trial028.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T06:41:54.954 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 20 16 6628 6612 1.2913 1.375 0.0790509 0.0483104 2026-02-12T06:41:55.954 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 21 16 6973 6957 1.29398 1.34766 0.0686109 0.0482098 2026-02-12T06:41:56.954 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 22 16 7316 7300 1.29606 1.33984 0.0480087 0.0481649 2026-02-12T06:41:57.954 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 23 16 7664 7648 1.29881 1.35938 0.0462776 0.048063 2026-02-12T06:41:58.954 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 24 16 8018 8002 1.3023 1.38281 0.0552012 0.0479309 2026-02-12T06:41:59.954 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 25 16 8375 8359 1.30599 1.39453 0.0507104 0.0477956 2026-02-12T06:42:00.954 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 26 16 8713 8697 1.30653 1.32031 0.0240487 0.0477676 2026-02-12T06:42:01.954 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 27 16 9042 9026 1.30574 1.28516 0.0471171 0.0478131 2026-02-12T06:42:02.954 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 28 16 9360 9344 1.30347 1.24219 0.0229155 0.0479092 2026-02-12T06:42:03.954 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 29 16 9708 9692 1.30539 1.35938 0.102366 0.0478243 2026-02-12T06:42:04.954 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 30 16 10062 10046 1.30797 1.38281 0.0584431 0.0477407 2026-02-12T06:42:05.954 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 31 16 10392 10376 1.30735 1.28906 0.0563217 0.0477502 2026-02-12T06:42:06.955 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 32 16 10726 10710 1.30727 1.30469 0.0299333 0.0477734 2026-02-12T06:42:07.955 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 33 16 11061 11045 1.3073 1.30859 0.0649104 0.0477634 2026-02-12T06:42:08.955 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 34 16 11401 11385 1.30791 1.32812 0.0480878 0.0477424 2026-02-12T06:42:09.955 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 35 16 11753 11737 1.30983 1.375 0.0769109 0.0476714 2026-02-12T06:42:10.955 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 36 16 12078 12062 1.30871 1.26953 0.0906188 0.0477118 2026-02-12T06:42:11.955 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 37 16 12403 12387 1.30764 1.26953 0.0369959 0.0477533 2026-02-12T06:42:12.955 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 38 16 12743 12727 1.30818 1.32812 0.0309377 0.0477438 2026-02-12T06:42:13.955 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 39 16 13075 13059 1.30789 1.29688 0.0522884 0.0477489 2026-02-12T06:42:14.955 INFO:tasks.radosbench.radosbench.0.trial028.stdout:2026-02-12T06:42:14.955191+0000 min lat: 0.00991992 max lat: 0.511495 avg lat: 0.0477676 2026-02-12T06:42:14.956 INFO:tasks.radosbench.radosbench.0.trial028.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T06:42:14.956 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 40 16 13403 13387 1.30722 1.28125 0.129649 0.0477676 2026-02-12T06:42:15.955 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 41 16 13729 13713 1.30639 1.27344 0.0493737 0.0477952 2026-02-12T06:42:16.956 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 42 16 14050 14034 1.30514 1.25391 0.0419355 0.047852 2026-02-12T06:42:17.956 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 43 16 14374 14358 1.30422 1.26562 0.0866534 0.047891 2026-02-12T06:42:18.955 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 44 16 14696 14680 1.30316 1.25781 0.103652 0.0479224 2026-02-12T06:42:19.955 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 45 16 14993 14977 1.29998 1.16016 0.0733894 0.0480315 2026-02-12T06:42:20.956 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 46 16 15300 15284 1.29779 1.19922 0.0231745 0.0481119 2026-02-12T06:42:21.956 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 47 16 15634 15618 1.29793 1.30469 0.0462883 0.0481182 2026-02-12T06:42:22.956 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 48 16 15926 15910 1.29465 1.14062 0.0421468 0.0482331 2026-02-12T06:42:23.956 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 49 16 16237 16221 1.29302 1.21484 0.0988601 0.0483021 2026-02-12T06:42:24.956 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 50 16 16543 16527 1.29107 1.19531 0.0440164 0.0483879 2026-02-12T06:42:25.956 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 51 16 16844 16828 1.28881 1.17578 0.0865155 0.0484493 2026-02-12T06:42:26.956 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 52 16 17151 17135 1.28708 1.19922 0.0483011 0.0485219 2026-02-12T06:42:27.957 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 53 16 17445 17429 1.28446 1.14844 0.0740679 0.0486271 2026-02-12T06:42:28.956 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 54 16 17758 17742 1.28332 1.22266 0.0712074 0.0486777 2026-02-12T06:42:29.957 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 55 16 18071 18055 1.28221 1.22266 0.0689836 0.0487112 2026-02-12T06:42:30.957 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 56 16 18370 18354 1.28017 1.16797 0.025428 0.0487806 2026-02-12T06:42:31.957 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 57 16 18683 18667 1.27916 1.22266 0.0381168 0.0488187 2026-02-12T06:42:32.957 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 58 16 18977 18961 1.2769 1.14844 0.0364543 0.048919 2026-02-12T06:42:33.957 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 59 16 19281 19265 1.27539 1.1875 0.0577914 0.0489738 2026-02-12T06:42:34.957 INFO:tasks.radosbench.radosbench.0.trial028.stdout:2026-02-12T06:42:34.956787+0000 min lat: 0.00991992 max lat: 0.511495 avg lat: 0.049021 2026-02-12T06:42:34.957 INFO:tasks.radosbench.radosbench.0.trial028.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T06:42:34.958 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 60 16 19586 19570 1.27399 1.19141 0.0435178 0.049021 2026-02-12T06:42:35.957 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 61 16 19893 19877 1.27276 1.19922 0.0287845 0.0490772 2026-02-12T06:42:36.957 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 62 16 20202 20186 1.2717 1.20703 0.0488895 0.0491098 2026-02-12T06:42:37.957 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 63 16 20502 20486 1.27011 1.17188 0.041504 0.0491657 2026-02-12T06:42:38.957 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 64 16 20805 20789 1.26876 1.18359 0.02359 0.0492335 2026-02-12T06:42:39.957 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 65 16 21119 21103 1.26811 1.22656 0.0269844 0.0492675 2026-02-12T06:42:40.957 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 66 16 21403 21387 1.2657 1.10938 0.043159 0.0493426 2026-02-12T06:42:41.958 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 67 16 21689 21673 1.26348 1.11719 0.0608001 0.049441 2026-02-12T06:42:42.958 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 68 16 21981 21965 1.26168 1.14062 0.115189 0.0495151 2026-02-12T06:42:43.958 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 69 16 22276 22260 1.26009 1.15234 0.0369999 0.0495725 2026-02-12T06:42:44.958 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 70 16 22570 22554 1.25849 1.14844 0.0437656 0.0496411 2026-02-12T06:42:45.958 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 71 16 22877 22861 1.25766 1.19922 0.035392 0.0496729 2026-02-12T06:42:46.958 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 72 16 23173 23157 1.25625 1.15625 0.0333973 0.0497287 2026-02-12T06:42:47.958 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 73 16 23472 23456 1.25504 1.16797 0.0364712 0.0497748 2026-02-12T06:42:48.958 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 74 16 23749 23733 1.2527 1.08203 0.0317334 0.0498631 2026-02-12T06:42:49.958 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 75 16 24030 24014 1.25063 1.09766 0.102406 0.0499511 2026-02-12T06:42:50.958 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 76 16 24320 24304 1.24908 1.13281 0.0689622 0.0500113 2026-02-12T06:42:51.959 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 77 16 24599 24583 1.24701 1.08984 0.0404506 0.0500965 2026-02-12T06:42:52.959 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 78 16 24854 24838 1.24379 0.996094 0.0514633 0.0502247 2026-02-12T06:42:53.958 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 79 16 25139 25123 1.24214 1.11328 0.0395184 0.0502916 2026-02-12T06:42:54.959 INFO:tasks.radosbench.radosbench.0.trial028.stdout:2026-02-12T06:42:54.958338+0000 min lat: 0.00991992 max lat: 0.511495 avg lat: 0.050349 2026-02-12T06:42:54.959 INFO:tasks.radosbench.radosbench.0.trial028.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T06:42:54.959 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 80 16 25426 25410 1.24062 1.12109 0.0456101 0.050349 2026-02-12T06:42:55.959 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 81 16 25722 25706 1.23958 1.15625 0.0699163 0.0503983 2026-02-12T06:42:56.959 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 82 16 26010 25994 1.23818 1.125 0.058776 0.0504557 2026-02-12T06:42:57.959 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 83 16 26295 26279 1.23668 1.11328 0.0595927 0.0505165 2026-02-12T06:42:58.959 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 84 16 26595 26579 1.2359 1.17188 0.030692 0.050551 2026-02-12T06:42:59.959 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 85 16 26882 26866 1.23455 1.12109 0.0534487 0.0506033 2026-02-12T06:43:00.960 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 86 16 27172 27156 1.23337 1.13281 0.0204718 0.0506554 2026-02-12T06:43:01.960 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 87 16 27448 27432 1.23158 1.07812 0.0326133 0.0507145 2026-02-12T06:43:02.960 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 88 16 27723 27707 1.22979 1.07422 0.0493646 0.0508018 2026-02-12T06:43:03.960 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 89 16 27996 27980 1.22795 1.06641 0.048916 0.0508644 2026-02-12T06:43:04.960 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 90 16 28279 28263 1.22659 1.10547 0.110132 0.0509295 2026-02-12T06:43:05.960 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 91 16 28574 28558 1.22577 1.15234 0.028557 0.0509624 2026-02-12T06:43:06.960 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 92 16 28841 28825 1.22379 1.04297 0.0954765 0.051049 2026-02-12T06:43:07.960 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 93 16 29116 29100 1.22218 1.07422 0.038057 0.051113 2026-02-12T06:43:08.960 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 94 16 29406 29390 1.22122 1.13281 0.040611 0.0511602 2026-02-12T06:43:09.960 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 95 16 29680 29664 1.21964 1.07031 0.0581078 0.0512215 2026-02-12T06:43:10.960 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 96 16 29954 29938 1.21808 1.07031 0.0314979 0.0512775 2026-02-12T06:43:11.960 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 97 16 30232 30216 1.21672 1.08594 0.0641165 0.0513429 2026-02-12T06:43:12.960 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 98 15 30515 30500 1.21562 1.10938 0.0541642 0.0513972 2026-02-12T06:43:13.961 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 99 16 30782 30766 1.21384 1.03906 0.0340336 0.0514691 2026-02-12T06:43:14.961 INFO:tasks.radosbench.radosbench.0.trial028.stdout:2026-02-12T06:43:14.960288+0000 min lat: 0.00991992 max lat: 0.511495 avg lat: 0.0515241 2026-02-12T06:43:14.961 INFO:tasks.radosbench.radosbench.0.trial028.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T06:43:14.961 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 100 16 31053 31037 1.21228 1.05859 0.0616084 0.0515241 2026-02-12T06:43:15.961 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 101 16 31334 31318 1.21115 1.09766 0.0618741 0.0515788 2026-02-12T06:43:16.961 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 102 16 31614 31598 1.20999 1.09375 0.049719 0.0516309 2026-02-12T06:43:17.961 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 103 16 31891 31875 1.20875 1.08203 0.05444 0.0516836 2026-02-12T06:43:18.961 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 104 16 32154 32138 1.20701 1.02734 0.0779091 0.0517585 2026-02-12T06:43:19.961 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 105 16 32422 32406 1.20548 1.04688 0.0765562 0.0518292 2026-02-12T06:43:20.961 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 106 16 32697 32681 1.20424 1.07422 0.0560039 0.051881 2026-02-12T06:43:21.961 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 107 16 32951 32935 1.20226 0.992188 0.0467294 0.0519574 2026-02-12T06:43:22.961 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 108 16 33210 33194 1.20049 1.01172 0.0517831 0.0520469 2026-02-12T06:43:23.961 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 109 16 33457 33441 1.19833 0.964844 0.0712964 0.0521322 2026-02-12T06:43:24.962 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 110 16 33707 33691 1.19631 0.976562 0.0268926 0.0522264 2026-02-12T06:43:25.962 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 111 16 33979 33963 1.19511 1.0625 0.0425374 0.0522769 2026-02-12T06:43:26.962 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 112 16 34243 34227 1.19364 1.03125 0.105032 0.052331 2026-02-12T06:43:27.962 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 113 16 34504 34488 1.1921 1.01953 0.0401648 0.0524102 2026-02-12T06:43:28.962 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 114 16 34776 34760 1.19097 1.0625 0.0629747 0.0524605 2026-02-12T06:43:29.962 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 115 16 35030 35014 1.18924 0.992188 0.117518 0.0525254 2026-02-12T06:43:30.962 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 116 16 35302 35286 1.18814 1.0625 0.064893 0.0525859 2026-02-12T06:43:31.962 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 117 16 35564 35548 1.18673 1.02344 0.0483048 0.0526447 2026-02-12T06:43:32.962 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 118 16 35829 35813 1.18545 1.03516 0.0627733 0.0527032 2026-02-12T06:43:33.962 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 119 16 36102 36086 1.18445 1.06641 0.0494381 0.0527473 2026-02-12T06:43:34.962 INFO:tasks.radosbench.radosbench.0.trial028.stdout:2026-02-12T06:43:34.961855+0000 min lat: 0.00991992 max lat: 0.511495 avg lat: 0.0528234 2026-02-12T06:43:34.962 INFO:tasks.radosbench.radosbench.0.trial028.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T06:43:34.963 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 120 16 36347 36331 1.18255 0.957031 0.0364304 0.0528234 2026-02-12T06:43:35.962 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 121 15 36607 36592 1.1812 1.01953 0.0995932 0.052893 2026-02-12T06:43:36.962 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 122 16 36875 36859 1.18007 1.04297 0.0921842 0.0529412 2026-02-12T06:43:37.963 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 123 16 37141 37125 1.17892 1.03906 0.0485586 0.0529905 2026-02-12T06:43:38.963 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 124 16 37413 37397 1.17798 1.0625 0.0575052 0.0530419 2026-02-12T06:43:39.963 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 125 16 37666 37650 1.17647 0.988281 0.0942598 0.0531027 2026-02-12T06:43:40.963 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 126 16 37924 37908 1.17513 1.00781 0.127508 0.0531637 2026-02-12T06:43:41.963 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 127 16 38195 38179 1.17421 1.05859 0.0481776 0.0532068 2026-02-12T06:43:42.963 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 128 16 38453 38437 1.17291 1.00781 0.027713 0.0532716 2026-02-12T06:43:43.963 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 129 16 38710 38694 1.1716 1.00391 0.0314617 0.0533225 2026-02-12T06:43:44.963 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 130 16 38968 38952 1.17034 1.00781 0.0555242 0.0533864 2026-02-12T06:43:45.963 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 131 16 39233 39217 1.1693 1.03516 0.0544253 0.0534332 2026-02-12T06:43:46.963 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 132 16 39491 39475 1.16808 1.00781 0.0628265 0.0534856 2026-02-12T06:43:47.963 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 133 16 39730 39714 1.16632 0.933594 0.100927 0.0535681 2026-02-12T06:43:48.963 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 134 16 39987 39971 1.1651 1.00391 0.0717214 0.0536221 2026-02-12T06:43:49.964 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 135 16 40242 40226 1.16385 0.996094 0.053355 0.0536792 2026-02-12T06:43:50.963 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 136 16 40501 40485 1.16273 1.01172 0.0430838 0.0537334 2026-02-12T06:43:51.963 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 137 16 40762 40746 1.16169 1.01953 0.0568898 0.0537826 2026-02-12T06:43:52.964 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 138 16 41025 41009 1.16071 1.02734 0.0663677 0.0538233 2026-02-12T06:43:53.964 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 139 15 41279 41264 1.15953 0.996094 0.0556207 0.0538841 2026-02-12T06:43:54.964 INFO:tasks.radosbench.radosbench.0.trial028.stdout:2026-02-12T06:43:54.963430+0000 min lat: 0.00991992 max lat: 0.511495 avg lat: 0.0539279 2026-02-12T06:43:54.964 INFO:tasks.radosbench.radosbench.0.trial028.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T06:43:54.964 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 140 16 41543 41527 1.15858 1.02734 0.0839897 0.0539279 2026-02-12T06:43:55.964 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 141 16 41793 41777 1.15729 0.976562 0.0891553 0.0539916 2026-02-12T06:43:56.964 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 142 16 42046 42030 1.1561 0.988281 0.0510257 0.0540469 2026-02-12T06:43:57.964 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 143 16 42301 42285 1.15498 0.996094 0.0679056 0.0540936 2026-02-12T06:43:58.964 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 144 16 42543 42527 1.15352 0.945312 0.186221 0.0541611 2026-02-12T06:43:59.964 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 145 16 42795 42779 1.15236 0.984375 0.073111 0.054217 2026-02-12T06:44:00.964 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 146 16 43039 43023 1.15099 0.953125 0.18894 0.0542781 2026-02-12T06:44:01.964 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 147 15 43293 43278 1.14994 0.996094 0.0517422 0.0543329 2026-02-12T06:44:02.964 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 148 16 43544 43528 1.14877 0.976562 0.0832891 0.0543884 2026-02-12T06:44:03.965 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 149 16 43791 43775 1.14753 0.964844 0.0486053 0.0544502 2026-02-12T06:44:04.964 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 150 16 44038 44022 1.14631 0.964844 0.0831578 0.0545068 2026-02-12T06:44:05.965 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 151 16 44292 44276 1.14529 0.992188 0.0363707 0.0545526 2026-02-12T06:44:06.965 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 152 16 44539 44523 1.1441 0.964844 0.0587036 0.0546099 2026-02-12T06:44:07.965 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 153 16 44783 44767 1.14286 0.953125 0.0718656 0.0546709 2026-02-12T06:44:08.965 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 154 16 45044 45028 1.14205 1.01953 0.122694 0.0547111 2026-02-12T06:44:09.965 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 155 16 45294 45278 1.14099 0.976562 0.0872749 0.054763 2026-02-12T06:44:10.965 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 156 16 45548 45532 1.14003 0.992188 0.0545356 0.0547994 2026-02-12T06:44:11.965 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 157 16 45801 45785 1.13906 0.988281 0.0392115 0.0548573 2026-02-12T06:44:12.965 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 158 16 46045 46029 1.13789 0.953125 0.0791047 0.0549048 2026-02-12T06:44:13.965 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 159 16 46298 46282 1.13695 0.988281 0.0507263 0.0549562 2026-02-12T06:44:14.966 INFO:tasks.radosbench.radosbench.0.trial028.stdout:2026-02-12T06:44:14.964981+0000 min lat: 0.00991992 max lat: 0.511495 avg lat: 0.0550084 2026-02-12T06:44:14.966 INFO:tasks.radosbench.radosbench.0.trial028.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T06:44:14.966 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 160 16 46544 46528 1.13585 0.960938 0.103314 0.0550084 2026-02-12T06:44:15.966 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 161 16 46791 46775 1.13478 0.964844 0.0381741 0.0550603 2026-02-12T06:44:16.966 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 162 16 47032 47016 1.13359 0.941406 0.0933503 0.0551185 2026-02-12T06:44:17.966 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 163 16 47275 47259 1.13246 0.949219 0.0742733 0.0551774 2026-02-12T06:44:18.966 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 164 16 47518 47502 1.13134 0.949219 0.0379046 0.0552307 2026-02-12T06:44:19.966 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 165 16 47765 47749 1.13033 0.964844 0.0742817 0.0552776 2026-02-12T06:44:20.966 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 166 16 48002 47986 1.1291 0.925781 0.0475804 0.0553391 2026-02-12T06:44:21.966 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 167 16 48246 48230 1.12804 0.953125 0.0621726 0.0553895 2026-02-12T06:44:22.966 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 168 16 48488 48472 1.12695 0.945312 0.144078 0.0554442 2026-02-12T06:44:23.966 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 169 16 48725 48709 1.12576 0.925781 0.0476478 0.0554999 2026-02-12T06:44:24.966 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 170 16 48970 48954 1.12477 0.957031 0.117445 0.0555522 2026-02-12T06:44:25.966 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 171 16 49210 49194 1.12368 0.9375 0.0778297 0.0556068 2026-02-12T06:44:26.966 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 172 16 49456 49440 1.12273 0.960938 0.0593412 0.0556541 2026-02-12T06:44:27.966 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 173 16 49693 49677 1.12159 0.925781 0.0275716 0.0557016 2026-02-12T06:44:28.967 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 174 16 49925 49909 1.12035 0.90625 0.120237 0.0557698 2026-02-12T06:44:29.967 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 175 16 50151 50135 1.11899 0.882812 0.0994444 0.055832 2026-02-12T06:44:30.967 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 176 16 50394 50378 1.11803 0.949219 0.0863529 0.0558912 2026-02-12T06:44:31.967 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 177 16 50641 50625 1.11716 0.964844 0.0556271 0.0559287 2026-02-12T06:44:32.967 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 178 16 50868 50852 1.11587 0.886719 0.0419109 0.0559984 2026-02-12T06:44:33.967 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 179 16 51111 51095 1.11494 0.949219 0.052375 0.0560423 2026-02-12T06:44:34.967 INFO:tasks.radosbench.radosbench.0.trial028.stdout:2026-02-12T06:44:34.966538+0000 min lat: 0.00991992 max lat: 0.511495 avg lat: 0.0561134 2026-02-12T06:44:34.967 INFO:tasks.radosbench.radosbench.0.trial028.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T06:44:34.967 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 180 16 51330 51314 1.1135 0.855469 0.0578572 0.0561134 2026-02-12T06:44:35.967 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 181 16 51563 51547 1.11237 0.910156 0.0497792 0.0561682 2026-02-12T06:44:36.967 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 182 16 51807 51791 1.1115 0.953125 0.0595664 0.0562166 2026-02-12T06:44:37.967 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 183 16 52047 52031 1.11054 0.9375 0.0568741 0.0562648 2026-02-12T06:44:38.967 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 184 16 52273 52257 1.10931 0.882812 0.0647083 0.0563261 2026-02-12T06:44:39.967 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 185 16 52507 52491 1.10825 0.914062 0.0530006 0.0563806 2026-02-12T06:44:40.967 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 186 16 52742 52726 1.10723 0.917969 0.103532 0.0564314 2026-02-12T06:44:41.968 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 187 16 52976 52960 1.10619 0.914062 0.0885847 0.0564859 2026-02-12T06:44:42.968 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 188 16 53204 53188 1.10505 0.890625 0.0492115 0.0565433 2026-02-12T06:44:43.968 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 189 16 53432 53416 1.10391 0.890625 0.0520286 0.0565956 2026-02-12T06:44:44.968 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 190 16 53662 53646 1.10283 0.898438 0.0344994 0.0566602 2026-02-12T06:44:45.968 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 191 16 53905 53889 1.10203 0.949219 0.0330747 0.0566957 2026-02-12T06:44:46.968 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 192 16 54140 54124 1.10107 0.917969 0.047995 0.0567475 2026-02-12T06:44:47.968 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 193 16 54376 54360 1.10014 0.921875 0.0397332 0.0567967 2026-02-12T06:44:48.968 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 194 16 54605 54589 1.09908 0.894531 0.0435961 0.0568503 2026-02-12T06:44:49.968 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 195 16 54837 54821 1.09809 0.90625 0.0685213 0.0569043 2026-02-12T06:44:50.968 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 196 16 55055 55039 1.09683 0.851562 0.120769 0.0569672 2026-02-12T06:44:51.968 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 197 16 55277 55261 1.09566 0.867188 0.092825 0.0570294 2026-02-12T06:44:52.968 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 198 16 55513 55497 1.09479 0.921875 0.0523072 0.057075 2026-02-12T06:44:53.968 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 199 16 55746 55730 1.09386 0.910156 0.0552805 0.057119 2026-02-12T06:44:54.969 INFO:tasks.radosbench.radosbench.0.trial028.stdout:2026-02-12T06:44:54.968066+0000 min lat: 0.00991992 max lat: 0.511495 avg lat: 0.0571641 2026-02-12T06:44:54.969 INFO:tasks.radosbench.radosbench.0.trial028.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T06:44:54.969 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 200 16 55990 55974 1.09315 0.953125 0.0436841 0.0571641 2026-02-12T06:44:55.969 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 201 16 56214 56198 1.09207 0.875 0.0504122 0.0572142 2026-02-12T06:44:56.969 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 202 16 56427 56411 1.09078 0.832031 0.0386415 0.0572725 2026-02-12T06:44:57.969 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 203 16 56662 56646 1.08993 0.917969 0.0791989 0.0573279 2026-02-12T06:44:58.969 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 204 16 56903 56887 1.0892 0.941406 0.0410511 0.0573713 2026-02-12T06:44:59.969 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 205 16 57125 57109 1.08812 0.867188 0.0606137 0.0574215 2026-02-12T06:45:00.969 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 206 16 57338 57322 1.08687 0.832031 0.0876467 0.0574862 2026-02-12T06:45:01.969 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 207 16 57559 57543 1.08579 0.863281 0.0570364 0.0575387 2026-02-12T06:45:02.969 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 208 16 57793 57777 1.08497 0.914062 0.0690696 0.0575909 2026-02-12T06:45:03.969 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 209 16 58020 58004 1.08402 0.886719 0.117967 0.0576366 2026-02-12T06:45:04.969 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 210 16 58249 58233 1.08312 0.894531 0.0446495 0.0576916 2026-02-12T06:45:05.969 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 211 16 58470 58454 1.08207 0.863281 0.074062 0.057746 2026-02-12T06:45:06.970 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 212 16 58686 58670 1.08095 0.84375 0.112888 0.0578034 2026-02-12T06:45:07.970 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 213 16 58912 58896 1.08002 0.882812 0.043914 0.0578528 2026-02-12T06:45:08.970 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 214 16 59145 59129 1.07922 0.910156 0.0925183 0.0579028 2026-02-12T06:45:09.970 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 215 16 59375 59359 1.07838 0.898438 0.0581722 0.0579411 2026-02-12T06:45:10.970 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 216 16 59601 59585 1.07748 0.882812 0.0376861 0.0579928 2026-02-12T06:45:11.970 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 217 16 59836 59820 1.07674 0.917969 0.0702616 0.0580328 2026-02-12T06:45:12.970 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 218 16 60064 60048 1.07589 0.890625 0.0595841 0.0580807 2026-02-12T06:45:13.970 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 219 16 60270 60254 1.07465 0.804688 0.111144 0.0581443 2026-02-12T06:45:14.970 INFO:tasks.radosbench.radosbench.0.trial028.stdout:2026-02-12T06:45:14.969625+0000 min lat: 0.00991992 max lat: 0.511495 avg lat: 0.058185 2026-02-12T06:45:14.970 INFO:tasks.radosbench.radosbench.0.trial028.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T06:45:14.971 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 220 16 60501 60485 1.07387 0.902344 0.0272065 0.058185 2026-02-12T06:45:15.970 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 221 16 60726 60710 1.07298 0.878906 0.0579591 0.058233 2026-02-12T06:45:16.970 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 222 16 60961 60945 1.07229 0.917969 0.0682007 0.0582758 2026-02-12T06:45:17.970 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 223 16 61179 61163 1.0713 0.851562 0.0604679 0.0583244 2026-02-12T06:45:18.970 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 224 16 61394 61378 1.07026 0.839844 0.0383179 0.0583814 2026-02-12T06:45:19.971 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 225 16 61618 61602 1.06939 0.875 0.0441246 0.0584303 2026-02-12T06:45:20.971 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 226 16 61835 61819 1.06841 0.847656 0.0598152 0.058487 2026-02-12T06:45:21.971 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 227 16 62054 62038 1.06747 0.855469 0.0613078 0.0585379 2026-02-12T06:45:22.971 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 228 16 62278 62262 1.06663 0.875 0.0482549 0.0585786 2026-02-12T06:45:23.971 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 229 16 62500 62484 1.06576 0.867188 0.102612 0.0586267 2026-02-12T06:45:24.971 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 230 16 62720 62704 1.06486 0.859375 0.0537144 0.0586775 2026-02-12T06:45:25.971 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 231 16 62948 62932 1.06411 0.890625 0.0425995 0.0587216 2026-02-12T06:45:26.971 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 232 16 63174 63158 1.06332 0.882812 0.0571228 0.0587636 2026-02-12T06:45:27.971 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 233 16 63383 63367 1.06226 0.816406 0.057414 0.0588159 2026-02-12T06:45:28.971 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 234 16 63602 63586 1.06138 0.855469 0.117142 0.058871 2026-02-12T06:45:29.971 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 235 16 63824 63808 1.06055 0.867188 0.0581582 0.0589195 2026-02-12T06:45:30.971 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 236 16 64040 64024 1.05963 0.84375 0.0311839 0.058962 2026-02-12T06:45:31.971 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 237 16 64253 64237 1.05867 0.832031 0.0891852 0.0590259 2026-02-12T06:45:32.971 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 238 16 64466 64450 1.05772 0.832031 0.0949553 0.0590736 2026-02-12T06:45:33.971 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 239 16 64676 64660 1.05673 0.820312 0.0985398 0.0591275 2026-02-12T06:45:34.972 INFO:tasks.radosbench.radosbench.0.trial028.stdout:2026-02-12T06:45:34.971125+0000 min lat: 0.00991992 max lat: 0.511495 avg lat: 0.0591847 2026-02-12T06:45:34.972 INFO:tasks.radosbench.radosbench.0.trial028.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T06:45:34.972 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 240 16 64889 64873 1.05579 0.832031 0.0600086 0.0591847 2026-02-12T06:45:35.972 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 241 16 65117 65101 1.05511 0.890625 0.0711778 0.0592215 2026-02-12T06:45:36.972 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 242 16 65334 65318 1.05425 0.847656 0.0556672 0.0592692 2026-02-12T06:45:37.972 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 243 16 65542 65526 1.05325 0.8125 0.0724649 0.0593271 2026-02-12T06:45:38.972 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 244 16 65767 65751 1.05254 0.878906 0.0634535 0.059369 2026-02-12T06:45:39.972 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 245 16 65980 65964 1.05164 0.832031 0.054247 0.0594174 2026-02-12T06:45:40.972 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 246 16 66199 66183 1.05084 0.855469 0.0423939 0.059462 2026-02-12T06:45:41.972 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 247 16 66399 66383 1.04975 0.78125 0.0882394 0.059523 2026-02-12T06:45:42.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 248 16 66602 66586 1.04871 0.792969 0.0480127 0.0595842 2026-02-12T06:45:43.972 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 249 16 66823 66807 1.04797 0.863281 0.068927 0.0596256 2026-02-12T06:45:44.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 250 16 67025 67009 1.04693 0.789062 0.0306894 0.0596836 2026-02-12T06:45:45.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 251 16 67242 67226 1.04614 0.847656 0.0680132 0.0597339 2026-02-12T06:45:46.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 252 16 67463 67447 1.04541 0.863281 0.0562411 0.0597715 2026-02-12T06:45:47.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 253 16 67674 67658 1.04454 0.824219 0.0620398 0.0598254 2026-02-12T06:45:48.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 254 16 67890 67874 1.04375 0.84375 0.0682621 0.0598642 2026-02-12T06:45:49.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 255 16 68103 68087 1.04292 0.832031 0.11744 0.059917 2026-02-12T06:45:50.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 256 16 68317 68301 1.04211 0.835938 0.152952 0.0599593 2026-02-12T06:45:51.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 257 16 68527 68511 1.04124 0.820312 0.0668473 0.0600101 2026-02-12T06:45:52.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 258 16 68737 68721 1.04039 0.820312 0.200257 0.0600565 2026-02-12T06:45:53.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 259 16 68956 68940 1.03967 0.855469 0.0514413 0.0601025 2026-02-12T06:45:54.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout:2026-02-12T06:45:54.972693+0000 min lat: 0.00991992 max lat: 0.511495 avg lat: 0.0601457 2026-02-12T06:45:54.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T06:45:54.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 260 16 69174 69158 1.03895 0.851562 0.0521313 0.0601457 2026-02-12T06:45:55.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 261 16 69374 69358 1.03796 0.78125 0.0789452 0.0601991 2026-02-12T06:45:56.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 262 16 69582 69566 1.0371 0.8125 0.0667227 0.0602477 2026-02-12T06:45:57.973 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 263 16 69784 69768 1.03616 0.789062 0.110288 0.0602989 2026-02-12T06:45:58.974 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 264 16 70000 69984 1.03543 0.84375 0.127076 0.060346 2026-02-12T06:45:59.974 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 265 16 70213 70197 1.03466 0.832031 0.0985593 0.06039 2026-02-12T06:46:00.974 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 266 16 70411 70395 1.03368 0.773438 0.078655 0.0604448 2026-02-12T06:46:01.974 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 267 16 70618 70602 1.03284 0.808594 0.0979775 0.0604978 2026-02-12T06:46:02.974 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 268 16 70813 70797 1.03182 0.761719 0.0529802 0.0605564 2026-02-12T06:46:03.974 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 269 16 71021 71005 1.03101 0.8125 0.147245 0.0606069 2026-02-12T06:46:04.974 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 270 16 71231 71215 1.03023 0.820312 0.0454552 0.0606548 2026-02-12T06:46:05.974 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 271 16 71453 71437 1.02963 0.867188 0.0750628 0.0606911 2026-02-12T06:46:06.974 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 272 16 71654 71638 1.02873 0.785156 0.0795122 0.0607445 2026-02-12T06:46:07.974 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 273 16 71860 71844 1.02791 0.804688 0.0459653 0.0607895 2026-02-12T06:46:08.974 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 274 16 72062 72046 1.02703 0.789062 0.048363 0.0608425 2026-02-12T06:46:09.974 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 275 16 72278 72262 1.02637 0.84375 0.0460592 0.0608835 2026-02-12T06:46:10.975 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 276 16 72476 72460 1.02545 0.773438 0.0953805 0.0609345 2026-02-12T06:46:11.975 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 277 16 72678 72662 1.0246 0.789062 0.0433514 0.0609874 2026-02-12T06:46:12.975 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 278 16 72889 72873 1.02388 0.824219 0.0375228 0.0610285 2026-02-12T06:46:13.974 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 279 16 73108 73092 1.02327 0.855469 0.0603963 0.0610667 2026-02-12T06:46:14.975 INFO:tasks.radosbench.radosbench.0.trial028.stdout:2026-02-12T06:46:14.974231+0000 min lat: 0.00991992 max lat: 0.511495 avg lat: 0.0611114 2026-02-12T06:46:14.975 INFO:tasks.radosbench.radosbench.0.trial028.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T06:46:14.975 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 280 16 73312 73296 1.02246 0.796875 0.0333287 0.0611114 2026-02-12T06:46:15.975 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 281 16 73509 73493 1.02156 0.769531 0.0602997 0.0611633 2026-02-12T06:46:16.975 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 282 16 73720 73704 1.02086 0.824219 0.120517 0.061209 2026-02-12T06:46:17.975 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 283 16 73922 73906 1.02004 0.789062 0.068154 0.0612592 2026-02-12T06:46:18.975 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 284 16 74127 74111 1.01927 0.800781 0.0890613 0.061304 2026-02-12T06:46:19.975 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 285 16 74319 74303 1.01833 0.75 0.099591 0.0613647 2026-02-12T06:46:20.975 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 286 16 74514 74498 1.01743 0.761719 0.0759813 0.0614148 2026-02-12T06:46:21.975 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 287 16 74716 74700 1.01663 0.789062 0.0922712 0.0614612 2026-02-12T06:46:22.975 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 288 16 74927 74911 1.01596 0.824219 0.0833458 0.0615058 2026-02-12T06:46:23.976 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 289 16 75107 75091 1.01488 0.703125 0.0737864 0.0615674 2026-02-12T06:46:24.976 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 290 16 75316 75300 1.0142 0.816406 0.108634 0.061612 2026-02-12T06:46:25.976 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 291 16 75524 75508 1.0135 0.8125 0.0315041 0.0616521 2026-02-12T06:46:26.976 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 292 16 75722 75706 1.01268 0.773438 0.17771 0.0617003 2026-02-12T06:46:27.976 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 293 16 75931 75915 1.01201 0.816406 0.054688 0.0617473 2026-02-12T06:46:28.976 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 294 16 76126 76110 1.01116 0.761719 0.0565391 0.0617998 2026-02-12T06:46:29.976 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 295 16 76337 76321 1.01053 0.824219 0.0585596 0.0618373 2026-02-12T06:46:30.976 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 296 16 76544 76528 1.00984 0.808594 0.0999805 0.061878 2026-02-12T06:46:31.976 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 297 16 76740 76724 1.00902 0.765625 0.0520514 0.0619231 2026-02-12T06:46:32.976 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 298 16 76942 76926 1.00828 0.789062 0.0948819 0.0619704 2026-02-12T06:46:33.976 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 299 16 77140 77124 1.0075 0.773438 0.0744324 0.0620223 2026-02-12T06:46:34.976 INFO:tasks.radosbench.radosbench.0.trial028.stdout:2026-02-12T06:46:34.975740+0000 min lat: 0.00991992 max lat: 0.511495 avg lat: 0.0620671 2026-02-12T06:46:34.977 INFO:tasks.radosbench.radosbench.0.trial028.stdout: sec Cur ops started finished avg MB/s cur MB/s last lat(s) avg lat(s) 2026-02-12T06:46:34.977 INFO:tasks.radosbench.radosbench.0.trial028.stdout: 300 10 77330 77320 1.00669 0.765625 0.190997 0.0620671 2026-02-12T06:46:35.977 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Total time run: 300.086 2026-02-12T06:46:35.977 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Total writes made: 77330 2026-02-12T06:46:35.977 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Write size: 4096 2026-02-12T06:46:35.977 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Object size: 4096 2026-02-12T06:46:35.977 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Bandwidth (MB/sec): 1.00661 2026-02-12T06:46:35.977 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Stddev Bandwidth: 0.172222 2026-02-12T06:46:35.977 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Max bandwidth (MB/sec): 1.41797 2026-02-12T06:46:35.978 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Min bandwidth (MB/sec): 0.703125 2026-02-12T06:46:35.978 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Average IOPS: 257 2026-02-12T06:46:35.978 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Stddev IOPS: 44.0887 2026-02-12T06:46:35.978 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Max IOPS: 363 2026-02-12T06:46:35.978 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Min IOPS: 180 2026-02-12T06:46:35.978 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Average Latency(s): 0.0620775 2026-02-12T06:46:35.978 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Stddev Latency(s): 0.0306714 2026-02-12T06:46:35.978 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Max latency(s): 0.511495 2026-02-12T06:46:35.978 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Min latency(s): 0.00991992 2026-02-12T06:46:36.005 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Cleaning up (deleting benchmark objects) 2026-02-12T06:54:21.706 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Removed 77330 objects 2026-02-12T06:54:21.707 INFO:tasks.radosbench.radosbench.0.trial028.stdout:Clean up completed and total clean up time :465.702 2026-02-12T06:54:23.657 INFO:tasks.ceph.ceph_manager.ceph:removing pool_name unique_pool_0 2026-02-12T06:54:23.658 DEBUG:teuthology.orchestra.run.trial028:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd pool rm unique_pool_0 unique_pool_0 --yes-i-really-really-mean-it 2026-02-12T06:54:23.740 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:54:23.735+0000 7f1feada4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:54:23.741 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:54:23.740+0000 7f1feada4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:54:23.914 INFO:teuthology.orchestra.run.trial028.stderr:pool 'unique_pool_0' removed 2026-02-12T06:54:23.926 DEBUG:teuthology.run_tasks:Unwinding manager ssh_keys 2026-02-12T06:54:23.941 INFO:teuthology.task.ssh_keys:Cleaning up SSH keys 2026-02-12T06:54:23.941 INFO:teuthology.task.ssh_keys:cleaning up keys added for testing 2026-02-12T06:54:23.942 INFO:teuthology.task.ssh_keys: cleaning up keys for user ubuntu on trial028.front.sepia.ceph.com 2026-02-12T06:54:23.942 DEBUG:teuthology.orchestra.run.trial028:> rm -- /home/ubuntu/.ssh/id_rsa 2026-02-12T06:54:23.963 DEBUG:teuthology.orchestra.run.trial028:> rm -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-12T06:54:24.018 DEBUG:teuthology.orchestra.run.trial028:> stat -c '"%a"' /home/ubuntu/.ssh/authorized_keys 2026-02-12T06:54:24.075 INFO:teuthology.orchestra.run.trial028.stdout:"600" 2026-02-12T06:54:24.075 DEBUG:teuthology.orchestra.run.trial028:> mv -- /home/ubuntu/.ssh/authorized_keys_2026-02-12_06:41:32:133780 /home/ubuntu/.ssh/authorized_keys 2026-02-12T06:54:24.142 DEBUG:teuthology.orchestra.run.trial028:> chmod 600 /home/ubuntu/.ssh/authorized_keys 2026-02-12T06:54:24.208 DEBUG:teuthology.orchestra.run.trial028:> rm /home/ubuntu/.ssh/config 2026-02-12T06:54:24.275 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-12T06:54:24.291 DEBUG:teuthology.orchestra.run.trial028:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-12T06:54:24.411 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:54:24.406+0000 7fb9fe6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:54:24.413 INFO:teuthology.orchestra.run.trial028.stderr:2026-02-12T06:54:24.412+0000 7fb9fe6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T06:54:24.552 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-12T06:54:24.552 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-12T06:54:24.552 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-12T06:54:24.552 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T06:54:24.652 INFO:tasks.ceph.osd.0:Stopped 2026-02-12T06:54:24.652 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-12T06:54:24.652 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T06:54:24.742 INFO:tasks.ceph.osd.1:Stopped 2026-02-12T06:54:24.743 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-12T06:54:24.743 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T06:54:24.811 INFO:tasks.ceph.osd.2:Stopped 2026-02-12T06:54:24.811 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-12T06:54:24.811 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-12T06:54:24.811 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T06:54:24.847 INFO:tasks.ceph.mgr.x:Stopped 2026-02-12T06:54:24.847 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-12T06:54:24.848 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-12T06:54:24.848 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T06:54:24.896 INFO:tasks.ceph.mon.a:Stopped 2026-02-12T06:54:24.896 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-12T06:54:24.897 DEBUG:teuthology.orchestra.run.trial028:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v '\(PG_' | grep -E -v '\(OSD_' | grep -E -v '\(OBJECT_' | grep -E -v 'overall HEALTH' | head -n 1 2026-02-12T06:54:24.963 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial028.front.sepia.ceph.com 2026-02-12T06:54:24.963 DEBUG:teuthology.orchestra.run.trial028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-12T06:54:25.064 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial028.front.sepia.ceph.com 2026-02-12T06:54:25.064 DEBUG:teuthology.orchestra.run.trial028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-12T06:54:25.097 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial028.front.sepia.ceph.com 2026-02-12T06:54:25.097 DEBUG:teuthology.orchestra.run.trial028:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-12T06:54:25.166 INFO:tasks.ceph:Archiving mon data... 2026-02-12T06:54:25.168 DEBUG:teuthology.misc:Transferring archived files from trial028:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/moagrawa-2026-02-12_06:23:35-crimson-rados-main-distro-debug-trial/45989/data/mon.a.tgz 2026-02-12T06:54:25.168 DEBUG:teuthology.orchestra.run.trial028:> mktemp 2026-02-12T06:54:25.223 INFO:teuthology.orchestra.run.trial028.stdout:/tmp/tmp.egO8oY4VnH 2026-02-12T06:54:25.223 DEBUG:teuthology.orchestra.run.trial028:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.egO8oY4VnH 2026-02-12T06:54:25.314 DEBUG:teuthology.orchestra.run.trial028:> sudo chmod 0666 /tmp/tmp.egO8oY4VnH 2026-02-12T06:54:25.395 DEBUG:teuthology.orchestra.remote:trial028:/tmp/tmp.egO8oY4VnH is 201KB 2026-02-12T06:54:25.446 DEBUG:teuthology.orchestra.run.trial028:> rm -fr /tmp/tmp.egO8oY4VnH 2026-02-12T06:54:25.461 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-12T06:54:25.461 DEBUG:teuthology.orchestra.run.trial028:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-12T06:54:25.581 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-12T06:54:25.582 INFO:tasks.ceph:Archiving crash dumps... 2026-02-12T06:54:25.584 DEBUG:teuthology.misc:Transferring archived files from trial028:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/moagrawa-2026-02-12_06:23:35-crimson-rados-main-distro-debug-trial/45989/remote/trial028/crash 2026-02-12T06:54:25.586 DEBUG:teuthology.orchestra.run.trial028:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-12T06:54:25.646 INFO:tasks.ceph:Compressing logs... 2026-02-12T06:54:25.646 DEBUG:teuthology.orchestra.run.trial028:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-12T06:54:25.712 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-12T06:54:25.712 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23094.log 2026-02-12T06:54:25.712 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-12T06:54:25.712 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.23094.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-12T06:54:25.713 INFO:teuthology.orchestra.run.trial028.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.23094.log.gz 2026-02-12T06:54:25.713 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.20662.log 2026-02-12T06:54:25.721 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22857.log 2026-02-12T06:54:25.721 INFO:teuthology.orchestra.run.trial028.stderr: 91.6% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-12T06:54:25.721 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph.tmp-client.admin.20662.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21951.log 2026-02-12T06:54:25.721 INFO:teuthology.orchestra.run.trial028.stderr: 66.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.20662.log.gz 2026-02-12T06:54:25.721 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.22857.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22857.log.gz 2026-02-12T06:54:25.721 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22277.log 2026-02-12T06:54:25.722 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.21951.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.21951.log.gz 2026-02-12T06:54:25.722 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22046.log 2026-02-12T06:54:25.722 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5/var/log/ceph/ceph-client.admin.22277.log: --verbose -- /var/log/ceph/ceph-client.admin.23798.log 2026-02-12T06:54:25.722 INFO:teuthology.orchestra.run.trial028.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22277.log.gz 2026-02-12T06:54:25.722 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.22046.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23045.log 2026-02-12T06:54:25.722 INFO:teuthology.orchestra.run.trial028.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22046.log.gz 2026-02-12T06:54:25.722 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23342.log 2026-02-12T06:54:25.722 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.23798.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.23798.log.gz 2026-02-12T06:54:25.722 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.23045.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.23045.log.gz 2026-02-12T06:54:25.722 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22093.log 2026-02-12T06:54:25.722 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.23342.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.23342.log.gz 2026-02-12T06:54:25.722 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22574.log 2026-02-12T06:54:25.722 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.22093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23197.log 2026-02-12T06:54:25.722 INFO:teuthology.orchestra.run.trial028.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22093.log.gz 2026-02-12T06:54:25.722 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23437.log 2026-02-12T06:54:25.723 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.22574.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.22574.log.gz 2026-02-12T06:54:25.723 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.23197.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.23197.log.gz 2026-02-12T06:54:25.723 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23196.log 2026-02-12T06:54:25.723 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.23437.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23994.log 2026-02-12T06:54:25.723 INFO:teuthology.orchestra.run.trial028.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.23437.log.gz 2026-02-12T06:54:25.723 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.23196.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23344.log 2026-02-12T06:54:25.723 INFO:teuthology.orchestra.run.trial028.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.23196.log.gz 2026-02-12T06:54:25.723 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.23994.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23535.log 2026-02-12T06:54:25.723 INFO:teuthology.orchestra.run.trial028.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.23994.log.gz 2026-02-12T06:54:25.723 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.23344.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22787.log 2026-02-12T06:54:25.723 INFO:teuthology.orchestra.run.trial028.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.23344.log.gz 2026-02-12T06:54:25.723 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.23535.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.23535.log.gz 2026-02-12T06:54:25.723 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-12T06:54:25.723 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.22787.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23486.log 2026-02-12T06:54:25.723 INFO:teuthology.orchestra.run.trial028.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22787.log.gz 2026-02-12T06:54:25.724 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23343.log 2026-02-12T06:54:25.724 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.23486.log: gzip 86.4% -- replaced with /var/log/ceph/ceph.audit.log.gz -5 2026-02-12T06:54:25.724 INFO:teuthology.orchestra.run.trial028.stderr: --verbose -- /var/log/ceph/ceph-client.admin.24179.log 2026-02-12T06:54:25.724 INFO:teuthology.orchestra.run.trial028.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.23486.log.gz 2026-02-12T06:54:25.724 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.23343.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-12T06:54:25.724 INFO:teuthology.orchestra.run.trial028.stderr: 47.2% -- replaced with /var/log/ceph/ceph-client.admin.23343.log.gz 2026-02-12T06:54:25.724 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.24179.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22882.log 2026-02-12T06:54:25.724 INFO:teuthology.orchestra.run.trial028.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.24179.log.gz 2026-02-12T06:54:25.724 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22230.log 2026-02-12T06:54:25.724 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.22882.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22324.log 2026-02-12T06:54:25.724 INFO:teuthology.orchestra.run.trial028.stderr: 67.2% -- replaced with /var/log/ceph/ceph-client.admin.22882.log.gz 2026-02-12T06:54:25.724 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.22230.log: 48.4% -- replaced with /var/log/ceph/ceph-client.admin.22230.log.gz 2026-02-12T06:54:25.724 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23746.log 2026-02-12T06:54:25.724 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-12T06:54:25.724 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.22324.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22324.log.gz 2026-02-12T06:54:25.724 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.23746.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.23746.log.gz 2026-02-12T06:54:25.725 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22720.log 2026-02-12T06:54:25.725 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23195.log 2026-02-12T06:54:25.725 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.22720.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.22720.log.gz 2026-02-12T06:54:25.725 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-12T06:54:25.725 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.0.23822.log 2026-02-12T06:54:25.725 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.23195.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.23195.log.gz 2026-02-12T06:54:25.725 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22996.log 2026-02-12T06:54:25.725 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.0.23822.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22947.log 2026-02-12T06:54:25.725 INFO:teuthology.orchestra.run.trial028.stderr: 66.4% -- replaced with /var/log/ceph/ceph-client.0.23822.log.gz 2026-02-12T06:54:25.725 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.22996.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22543.log 2026-02-12T06:54:25.725 INFO:teuthology.orchestra.run.trial028.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22996.log.gz 2026-02-12T06:54:25.725 INFO:teuthology.orchestra.run.trial028.stderr: 88.5% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-12T06:54:25.725 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.22947.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22947.log.gz 2026-02-12T06:54:25.725 INFO:teuthology.orchestra.run.trial028.stderr:/var/log/ceph/ceph-client.admin.22543.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22543.log.gz 2026-02-12T06:54:25.739 INFO:teuthology.orchestra.run.trial028.stderr: 92.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-12T06:55:05.057 INFO:teuthology.orchestra.run.trial028.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-12T06:55:05.848 INFO:teuthology.orchestra.run.trial028.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-12T06:55:13.371 INFO:teuthology.orchestra.run.trial028.stderr: 93.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-12T06:55:13.373 INFO:teuthology.orchestra.run.trial028.stderr: 2026-02-12T06:55:13.374 INFO:teuthology.orchestra.run.trial028.stderr:real 0m47.671s 2026-02-12T06:55:13.374 INFO:teuthology.orchestra.run.trial028.stderr:user 2m1.261s 2026-02-12T06:55:13.374 INFO:teuthology.orchestra.run.trial028.stderr:sys 0m5.658s 2026-02-12T06:55:13.374 INFO:tasks.ceph:Archiving logs... 2026-02-12T06:55:13.374 DEBUG:teuthology.misc:Transferring archived files from trial028:/var/log/ceph to /home/teuthworker/mnt/teuthology/moagrawa-2026-02-12_06:23:35-crimson-rados-main-distro-debug-trial/45989/remote/trial028/log 2026-02-12T06:55:13.376 DEBUG:teuthology.orchestra.run.trial028:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-12T06:55:22.679 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-12T06:55:22.749 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-12T06:55:22.760 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-12T06:55:22.761 DEBUG:teuthology.orchestra.run.trial028:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-12T06:55:22.787 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-12T06:55:22.798 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-12T06:55:22.798 DEBUG:teuthology.orchestra.run.trial028:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-12T06:55:22.842 INFO:teuthology.orchestra.run.trial028.stderr:bash: line 1: ntpq: command not found 2026-02-12T06:55:22.869 INFO:teuthology.orchestra.run.trial028.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-12T06:55:22.870 INFO:teuthology.orchestra.run.trial028.stdout:=============================================================================== 2026-02-12T06:55:22.870 INFO:teuthology.orchestra.run.trial028.stdout:^- 23.186.168.130 2 7 367 42 -2118us[-2118us] +/- 65ms 2026-02-12T06:55:22.870 INFO:teuthology.orchestra.run.trial028.stdout:^- 23.186.168.133 2 6 277 49 -2158us[-2158us] +/- 65ms 2026-02-12T06:55:22.870 INFO:teuthology.orchestra.run.trial028.stdout:^* 149.28.61.105.vultruserc> 3 6 377 56 -163us[ -163us] +/- 3857us 2026-02-12T06:55:22.870 INFO:teuthology.orchestra.run.trial028.stdout:^- kjsl-fmt2-net.fmt2.kjsl.> 2 6 377 50 +138us[ +138us] +/- 34ms 2026-02-12T06:55:22.870 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-12T06:55:22.881 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-12T06:55:22.881 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-12T06:55:22.896 DEBUG:teuthology.orchestra.run.trial028:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-12T06:55:22.953 DEBUG:teuthology.orchestra.run.trial028:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-12T06:55:22.977 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T06:55:22.977 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-12T06:55:22.989 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-12T06:55:23.003 INFO:teuthology.task.internal:Duration was 1161.474577 seconds 2026-02-12T06:55:23.003 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-12T06:55:23.015 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-12T06:55:23.015 DEBUG:teuthology.orchestra.run.trial028:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-12T06:55:23.055 INFO:teuthology.orchestra.run.trial028.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-12T06:55:23.451 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-12T06:55:23.451 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial028.front.sepia.ceph.com 2026-02-12T06:55:23.452 DEBUG:teuthology.orchestra.run.trial028:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-12T06:55:23.510 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-12T06:55:23.510 DEBUG:teuthology.orchestra.run.trial028:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-12T06:55:23.593 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-12T06:55:23.593 DEBUG:teuthology.orchestra.run.trial028:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-12T06:55:23.656 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T06:55:23.656 INFO:teuthology.orchestra.run.trial028.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T06:55:23.656 INFO:teuthology.orchestra.run.trial028.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-12T06:55:23.656 INFO:teuthology.orchestra.run.trial028.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-12T06:55:23.657 INFO:teuthology.orchestra.run.trial028.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-12T06:55:23.661 INFO:teuthology.orchestra.run.trial028.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 87.8% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-12T06:55:23.662 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-12T06:55:23.673 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-12T06:55:23.673 DEBUG:teuthology.orchestra.run.trial028:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-12T06:55:23.728 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-12T06:55:23.745 DEBUG:teuthology.orchestra.run.trial028:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-12T06:55:23.792 INFO:teuthology.orchestra.run.trial028.stdout:kernel.core_pattern = core 2026-02-12T06:55:23.823 DEBUG:teuthology.orchestra.run.trial028:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-12T06:55:23.878 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T06:55:23.879 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-12T06:55:23.890 INFO:teuthology.task.internal:Transferring archived files... 2026-02-12T06:55:23.890 DEBUG:teuthology.misc:Transferring archived files from trial028:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/moagrawa-2026-02-12_06:23:35-crimson-rados-main-distro-debug-trial/45989/remote/trial028 2026-02-12T06:55:23.890 DEBUG:teuthology.orchestra.run.trial028:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-12T06:55:23.951 INFO:teuthology.task.internal:Removing archive directory... 2026-02-12T06:55:23.951 DEBUG:teuthology.orchestra.run.trial028:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-12T06:55:24.009 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-12T06:55:24.022 INFO:teuthology.task.internal:Not uploading archives. 2026-02-12T06:55:24.022 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-12T06:55:24.040 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-12T06:55:24.040 DEBUG:teuthology.orchestra.run.trial028:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-12T06:55:24.065 INFO:teuthology.orchestra.run.trial028.stdout: 786525 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 12 06:55 /home/ubuntu/cephtest 2026-02-12T06:55:24.066 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-12T06:55:24.085 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-12T06:55:24.111 INFO:teuthology.run:Summary data: description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} perf_ignorelist settings/optimized workloads/radosbench_omap_write} duration: 1161.4745771884918 flavor: debug owner: scheduled_moagrawa@soko04.front.sepia.ceph.com success: true 2026-02-12T06:55:24.111 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-12T06:55:24.192 INFO:teuthology.run:pass