2026-02-25T08:03:00.085 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-25T08:03:00.102 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-25T08:03:00.182 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69108 branch: wip-logstoremanager description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} perf_ignorelist settings/optimized workloads/radosbench_4M_write} email: null first_in_suite: false flavor: debug job_id: '69108' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-logstoremanager ceph: conf: global: auth client required: none auth cluster required: none auth service required: none auth supported: none debug asok: 0/0 debug auth: 0/0 debug bdev: 0/0 debug bluefs: 0/0 debug bluestore: 0/0 debug buffer: 0/0 debug civetweb: 0/0 debug client: 0/0 debug compressor: 0/0 debug context: 0/0 debug crush: 0/0 debug crypto: 0/0 debug dpdk: 0/0 debug eventtrace: 0/0 debug filer: 0/0 debug filestore: 0/0 debug finisher: 0/0 debug fuse: 0/0 debug heartbeatmap: 0/0 debug javaclient: 0/0 debug journal: 0/0 debug journaler: 0/0 debug kstore: 0/0 debug leveldb: 0/0 debug lockdep: 0/0 debug mds: 0/0 debug mds balancer: 0/0 debug mds locker: 0/0 debug mds log: 0/0 debug mds log expire: 0/0 debug mds migrator: 0/0 debug memdb: 0/0 debug mgr: 0/0 debug mgrc: 0/0 debug mon: 0/0 debug monc: 0/0 debug ms: 0/0 debug objclass: 0/0 debug objectcacher: 0/0 debug objecter: 0/0 debug optracker: 0/0 debug osd: 0/0 debug paxos: 0/0 debug perfcounter: 0/0 debug rados: 0/0 debug rbd: 0/0 debug rbd mirror: 0/0 debug rbd replay: 0/0 debug refs: 0/0 debug rgw: 0/0 debug rgw lifecycle: 0/0 debug rgw notification: 0/0 debug rgw sync: 0/0 debug rocksdb: 0/0 debug striper: 0/0 debug throttle: 0/0 debug timer: 0/0 debug tp: 0/0 enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 0/0 debug ms: 0/0 debug paxos: 0/0 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug filestore: 0/0 debug journal: 0/0 debug ms: 0/0 debug osd: 0/0 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - \(PG_ - \(OSD_ - \(OBJECT_ - overall HEALTH sha1: 6aeb6975bc9a4790a399c9cc70dba9635d2a7819 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: flavor: debug sha1: 6aeb6975bc9a4790a399c9cc70dba9635d2a7819 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: c32469671c6b0222f786dd36c34eb702f780a11d owner: scheduled_matan@soko04.front.sepia.ceph.com priority: 70 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 869 sha1: 6aeb6975bc9a4790a399c9cc70dba9635d2a7819 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: c32469671c6b0222f786dd36c34eb702f780a11d targets: trial137.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH3WPWOa+mBftIaV7aiPNtRWdWFz/J2quv1MdtWj5IQ7tlOtV5xba1lj00Zq81vlQIk8DXonJt2cXQn0rKdnV88= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: osd: debug monc: 20 wait-for-scrub: false - ssh_keys: null - cbt: benchmarks: radosbench: concurrent_ops: 4 concurrent_procs: 2 op_size: - 4194304 pool_monitoring_list: - collectl pool_profile: replicated run_monitoring_list: - collectl time: 60 write_only: true cluster: iterations: 1 osds_per_node: 3 pool_profiles: replicated: pg_size: 256 pgp_size: 256 replication: replicated user: ubuntu monitoring_profiles: perf: args: stat -p {pid} -o {perf_dir}/perf_stat.{pid} nodes: - osds perf_cmd: perf pid_glob: crimson-osd.*.pid teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-25_07:55:35 tube: trial user: matan verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699 2026-02-25T08:03:00.182 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa; will attempt to use it 2026-02-25T08:03:00.183 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks 2026-02-25T08:03:00.183 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-25T08:03:00.185 INFO:teuthology.task.internal:Checking packages... 2026-02-25T08:03:00.185 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash '6aeb6975bc9a4790a399c9cc70dba9635d2a7819' 2026-02-25T08:03:00.185 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-25T08:03:00.185 INFO:teuthology.packaging:ref: None 2026-02-25T08:03:00.185 INFO:teuthology.packaging:tag: None 2026-02-25T08:03:00.185 INFO:teuthology.packaging:branch: wip-logstoremanager 2026-02-25T08:03:00.185 INFO:teuthology.packaging:sha1: 6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:03:00.185 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=wip-logstoremanager 2026-02-25T08:03:00.337 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5338.g6aeb6975 2026-02-25T08:03:00.338 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-25T08:03:00.349 INFO:teuthology.task.internal:no buildpackages task found 2026-02-25T08:03:00.349 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-25T08:03:00.363 INFO:teuthology.task.internal:Saving configuration 2026-02-25T08:03:00.377 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-25T08:03:00.389 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-25T08:03:00.465 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial137.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69108', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-25 07:58:02.040132', 'locked_by': 'scheduled_matan@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH3WPWOa+mBftIaV7aiPNtRWdWFz/J2quv1MdtWj5IQ7tlOtV5xba1lj00Zq81vlQIk8DXonJt2cXQn0rKdnV88='} 2026-02-25T08:03:00.465 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-25T08:03:00.475 INFO:teuthology.task.internal:roles: ubuntu@trial137.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-25T08:03:00.475 INFO:teuthology.run_tasks:Running task console_log... 2026-02-25T08:03:00.557 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7aca0e023920>, signals=[15]) 2026-02-25T08:03:00.557 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-25T08:03:00.569 INFO:teuthology.task.internal:Opening connections... 2026-02-25T08:03:00.569 DEBUG:teuthology.task.internal:connecting to ubuntu@trial137.front.sepia.ceph.com 2026-02-25T08:03:00.570 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:03:00.633 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-25T08:03:00.644 DEBUG:teuthology.orchestra.run.trial137:> uname -m 2026-02-25T08:03:00.708 INFO:teuthology.orchestra.run.trial137.stdout:x86_64 2026-02-25T08:03:00.708 DEBUG:teuthology.orchestra.run.trial137:> cat /etc/os-release 2026-02-25T08:03:00.763 INFO:teuthology.orchestra.run.trial137.stdout:NAME="CentOS Stream" 2026-02-25T08:03:00.763 INFO:teuthology.orchestra.run.trial137.stdout:VERSION="9" 2026-02-25T08:03:00.763 INFO:teuthology.orchestra.run.trial137.stdout:ID="centos" 2026-02-25T08:03:00.763 INFO:teuthology.orchestra.run.trial137.stdout:ID_LIKE="rhel fedora" 2026-02-25T08:03:00.763 INFO:teuthology.orchestra.run.trial137.stdout:VERSION_ID="9" 2026-02-25T08:03:00.763 INFO:teuthology.orchestra.run.trial137.stdout:PLATFORM_ID="platform:el9" 2026-02-25T08:03:00.763 INFO:teuthology.orchestra.run.trial137.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-25T08:03:00.763 INFO:teuthology.orchestra.run.trial137.stdout:ANSI_COLOR="0;31" 2026-02-25T08:03:00.763 INFO:teuthology.orchestra.run.trial137.stdout:LOGO="fedora-logo-icon" 2026-02-25T08:03:00.763 INFO:teuthology.orchestra.run.trial137.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-25T08:03:00.763 INFO:teuthology.orchestra.run.trial137.stdout:HOME_URL="https://centos.org/" 2026-02-25T08:03:00.763 INFO:teuthology.orchestra.run.trial137.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-25T08:03:00.763 INFO:teuthology.orchestra.run.trial137.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-25T08:03:00.763 INFO:teuthology.orchestra.run.trial137.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-25T08:03:00.764 INFO:teuthology.lock.ops:Updating trial137.front.sepia.ceph.com on lock server 2026-02-25T08:03:00.822 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-25T08:03:00.833 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-25T08:03:00.844 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-25T08:03:00.844 DEBUG:teuthology.orchestra.run.trial137:> test '!' -e /home/ubuntu/cephtest 2026-02-25T08:03:00.858 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-25T08:03:00.869 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-25T08:03:00.869 DEBUG:teuthology.orchestra.run.trial137:> test -z $(ls -A /var/lib/ceph) 2026-02-25T08:03:00.914 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-25T08:03:01.007 INFO:teuthology.run_tasks:Running task kernel... 2026-02-25T08:03:01.026 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-25T08:03:01.026 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-25T08:03:01.026 DEBUG:teuthology.orchestra.run.trial137:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-25T08:03:01.041 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:03:01.041 DEBUG:teuthology.orchestra.run.trial137:> uname -r 2026-02-25T08:03:01.097 INFO:teuthology.orchestra.run.trial137.stdout:5.14.0-681.el9.x86_64 2026-02-25T08:03:01.097 INFO:teuthology.task.kernel:Running kernel on trial137: 5.14.0-681.el9.x86_64 2026-02-25T08:03:01.097 DEBUG:teuthology.orchestra.run.trial137:> sudo yum install -y kernel 2026-02-25T08:03:06.863 INFO:teuthology.orchestra.run.trial137.stdout:CentOS Stream 9 - BaseOS 1.7 MB/s | 8.9 MB 00:05 2026-02-25T08:03:10.410 INFO:teuthology.orchestra.run.trial137.stdout:CentOS Stream 9 - AppStream 9.0 MB/s | 27 MB 00:03 2026-02-25T08:03:14.739 INFO:teuthology.orchestra.run.trial137.stdout:CentOS Stream 9 - CRB 4.2 MB/s | 8.0 MB 00:01 2026-02-25T08:03:16.867 INFO:teuthology.orchestra.run.trial137.stdout:CentOS Stream 9 - Extras packages 15 kB/s | 20 kB 00:01 2026-02-25T08:03:18.066 INFO:teuthology.orchestra.run.trial137.stdout:Extra Packages for Enterprise Linux 18 MB/s | 20 MB 00:01 2026-02-25T08:03:21.310 INFO:teuthology.orchestra.run.trial137.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-25T08:03:22.316 INFO:teuthology.orchestra.run.trial137.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-25T08:03:22.316 INFO:teuthology.orchestra.run.trial137.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-25T08:03:22.336 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:03:22.337 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:03:22.337 INFO:teuthology.orchestra.run.trial137.stdout: Package Architecture Version Repository Size 2026-02-25T08:03:22.337 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:03:22.337 INFO:teuthology.orchestra.run.trial137.stdout:Installing: 2026-02-25T08:03:22.338 INFO:teuthology.orchestra.run.trial137.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-25T08:03:22.338 INFO:teuthology.orchestra.run.trial137.stdout:Installing dependencies: 2026-02-25T08:03:22.338 INFO:teuthology.orchestra.run.trial137.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-25T08:03:22.338 INFO:teuthology.orchestra.run.trial137.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-25T08:03:22.338 INFO:teuthology.orchestra.run.trial137.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-25T08:03:22.338 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:03:22.338 INFO:teuthology.orchestra.run.trial137.stdout:Transaction Summary 2026-02-25T08:03:22.338 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:03:22.338 INFO:teuthology.orchestra.run.trial137.stdout:Install 4 Packages 2026-02-25T08:03:22.338 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:03:22.339 INFO:teuthology.orchestra.run.trial137.stdout:Total download size: 89 M 2026-02-25T08:03:22.339 INFO:teuthology.orchestra.run.trial137.stdout:Installed size: 135 M 2026-02-25T08:03:22.339 INFO:teuthology.orchestra.run.trial137.stdout:Downloading Packages: 2026-02-25T08:03:22.553 INFO:teuthology.orchestra.run.trial137.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 6.8 MB/s | 901 kB 00:00 2026-02-25T08:03:23.330 INFO:teuthology.orchestra.run.trial137.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 19 MB/s | 17 MB 00:00 2026-02-25T08:03:23.970 INFO:teuthology.orchestra.run.trial137.stdout:(3/4): kernel-modules-core-5.14.0-686.el9.x86_6 22 MB/s | 31 MB 00:01 2026-02-25T08:03:24.448 INFO:teuthology.orchestra.run.trial137.stdout:(4/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 20 MB/s | 40 MB 00:02 2026-02-25T08:03:24.448 INFO:teuthology.orchestra.run.trial137.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:03:24.449 INFO:teuthology.orchestra.run.trial137.stdout:Total 42 MB/s | 89 MB 00:02 2026-02-25T08:03:24.640 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction check 2026-02-25T08:03:24.851 INFO:teuthology.orchestra.run.trial137.stdout:Transaction check succeeded. 2026-02-25T08:03:24.851 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction test 2026-02-25T08:03:25.235 INFO:teuthology.orchestra.run.trial137.stdout:Transaction test succeeded. 2026-02-25T08:03:25.235 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction 2026-02-25T08:03:26.340 INFO:teuthology.orchestra.run.trial137.stdout: Preparing : 1/1 2026-02-25T08:03:26.630 INFO:teuthology.orchestra.run.trial137.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-25T08:03:26.652 INFO:teuthology.orchestra.run.trial137.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T08:03:29.211 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T08:03:29.244 INFO:teuthology.orchestra.run.trial137.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T08:03:32.348 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T08:03:32.354 INFO:teuthology.orchestra.run.trial137.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:03:35.625 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:03:53.036 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:03:53.058 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:03:53.246 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:03:53.246 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-25T08:03:53.246 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T08:03:53.246 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T08:03:53.303 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T08:03:53.303 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:03:53.303 INFO:teuthology.orchestra.run.trial137.stdout:Installed: 2026-02-25T08:03:53.303 INFO:teuthology.orchestra.run.trial137.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-25T08:03:53.303 INFO:teuthology.orchestra.run.trial137.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-25T08:03:53.304 INFO:teuthology.orchestra.run.trial137.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-25T08:03:53.304 INFO:teuthology.orchestra.run.trial137.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-25T08:03:53.304 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:03:53.304 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:03:53.365 DEBUG:teuthology.orchestra.run.trial137:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T08:03:53.396 INFO:teuthology.orchestra.run.trial137.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T08:03:53.396 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T08:03:53.396 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-25T08:03:53.397 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-25T08:03:53.397 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-25T08:03:53.397 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-25T08:03:53.397 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial137.front.sepia.ceph.com, path=None, version=distro) 2026-02-25T08:03:53.397 DEBUG:teuthology.orchestra.run.trial137:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T08:03:53.461 INFO:teuthology.orchestra.run.trial137.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T08:03:53.462 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T08:03:53.462 DEBUG:teuthology.orchestra.run.trial137:> sudo rpm -qi grub2-tools 2026-02-25T08:03:53.530 INFO:teuthology.orchestra.run.trial137.stdout:Name : grub2-tools 2026-02-25T08:03:53.530 INFO:teuthology.orchestra.run.trial137.stdout:Epoch : 1 2026-02-25T08:03:53.530 INFO:teuthology.orchestra.run.trial137.stdout:Version : 2.06 2026-02-25T08:03:53.530 INFO:teuthology.orchestra.run.trial137.stdout:Release : 120.el9 2026-02-25T08:03:53.531 INFO:teuthology.orchestra.run.trial137.stdout:Architecture: x86_64 2026-02-25T08:03:53.531 INFO:teuthology.orchestra.run.trial137.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-25T08:03:53.531 INFO:teuthology.orchestra.run.trial137.stdout:Group : Unspecified 2026-02-25T08:03:53.531 INFO:teuthology.orchestra.run.trial137.stdout:Size : 9108611 2026-02-25T08:03:53.531 INFO:teuthology.orchestra.run.trial137.stdout:License : GPLv3+ 2026-02-25T08:03:53.531 INFO:teuthology.orchestra.run.trial137.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-25T08:03:53.531 INFO:teuthology.orchestra.run.trial137.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-25T08:03:53.531 INFO:teuthology.orchestra.run.trial137.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-25T08:03:53.531 INFO:teuthology.orchestra.run.trial137.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-25T08:03:53.531 INFO:teuthology.orchestra.run.trial137.stdout:Packager : builder@centos.org 2026-02-25T08:03:53.531 INFO:teuthology.orchestra.run.trial137.stdout:Vendor : CentOS 2026-02-25T08:03:53.531 INFO:teuthology.orchestra.run.trial137.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-25T08:03:53.531 INFO:teuthology.orchestra.run.trial137.stdout:Summary : Support tools for GRUB. 2026-02-25T08:03:53.532 INFO:teuthology.orchestra.run.trial137.stdout:Description : 2026-02-25T08:03:53.532 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:03:53.532 INFO:teuthology.orchestra.run.trial137.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-25T08:03:53.532 INFO:teuthology.orchestra.run.trial137.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-25T08:03:53.532 INFO:teuthology.orchestra.run.trial137.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-25T08:03:53.532 INFO:teuthology.orchestra.run.trial137.stdout:hardware devices. 2026-02-25T08:03:53.532 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:03:53.532 INFO:teuthology.orchestra.run.trial137.stdout:This subpackage provides tools for support of all platforms. 2026-02-25T08:03:53.533 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-25T08:03:53.533 INFO:teuthology.task.kernel:Updating grub on trial137 to boot 5.14.0-686.el9.x86_64 2026-02-25T08:03:53.534 DEBUG:teuthology.orchestra.run.trial137:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-25T08:03:53.597 INFO:teuthology.orchestra.run.trial137.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-25T08:03:53.599 DEBUG:teuthology.orchestra.run.trial137:> sudo command -v grubby && echo yes || echo no 2026-02-25T08:03:53.662 INFO:teuthology.orchestra.run.trial137.stdout:/sbin/grubby 2026-02-25T08:03:53.663 INFO:teuthology.orchestra.run.trial137.stdout:yes 2026-02-25T08:03:53.664 DEBUG:teuthology.orchestra.run.trial137:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-25T08:03:53.727 INFO:teuthology.orchestra.run.trial137.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-25T08:03:53.728 DEBUG:teuthology.orchestra.run.trial137:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-25T08:03:53.801 DEBUG:teuthology.orchestra.run.trial137:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-25T08:03:54.006 INFO:teuthology.orchestra.run.trial137.stderr:Generating grub configuration file ... 2026-02-25T08:03:54.075 INFO:teuthology.orchestra.run.trial137.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-25T08:03:54.081 INFO:teuthology.orchestra.run.trial137.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-25T08:03:54.332 INFO:teuthology.orchestra.run.trial137.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-25T08:03:54.338 INFO:teuthology.orchestra.run.trial137.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-25T08:03:54.372 INFO:teuthology.orchestra.run.trial137.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-25T08:03:54.378 INFO:teuthology.orchestra.run.trial137.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-25T08:03:54.409 INFO:teuthology.orchestra.run.trial137.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-25T08:03:54.415 INFO:teuthology.orchestra.run.trial137.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-25T08:03:54.438 INFO:teuthology.orchestra.run.trial137.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-25T08:03:54.444 INFO:teuthology.orchestra.run.trial137.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-25T08:03:54.530 INFO:teuthology.orchestra.run.trial137.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-25T08:03:54.538 INFO:teuthology.orchestra.run.trial137.stderr:done 2026-02-25T08:03:54.540 DEBUG:teuthology.orchestra.run.trial137:> test -d /sys/firmware/efi 2026-02-25T08:03:54.555 DEBUG:teuthology.orchestra.run.trial137:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-25T08:03:54.622 INFO:teuthology.orchestra.run.trial137.stdout:/boot/efi/EFI/BOOT 2026-02-25T08:03:54.622 INFO:teuthology.orchestra.run.trial137.stdout:/boot/efi/EFI/centos 2026-02-25T08:03:54.623 DEBUG:teuthology.orchestra.run.trial137:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-25T08:03:54.688 INFO:teuthology.orchestra.run.trial137.stdout:no 2026-02-25T08:03:54.688 DEBUG:teuthology.orchestra.run.trial137:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-25T08:03:54.753 INFO:teuthology.orchestra.run.trial137.stdout:yes 2026-02-25T08:03:54.753 DEBUG:teuthology.orchestra.run.trial137:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-25T08:03:54.817 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial137 2026-02-25T08:03:54.817 DEBUG:teuthology.orchestra.run.trial137:> sudo bash -c ' 2026-02-25T08:03:54.817 DEBUG:teuthology.orchestra.run.trial137:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-25T08:03:54.817 DEBUG:teuthology.orchestra.run.trial137:> ' 2026-02-25T08:04:24.859 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-25T08:04:24.860 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial137.front.sepia.ceph.com' 2026-02-25T08:04:24.860 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:04:39.168 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.137 2026-02-25T08:04:48.176 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial137.front.sepia.ceph.com' 2026-02-25T08:04:48.177 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:05:24.064 DEBUG:teuthology.orchestra.run.trial137:> true 2026-02-25T08:05:24.212 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial137.front.sepia.ceph.com' 2026-02-25T08:05:24.212 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-686.el9.x86_64"... 2026-02-25T08:05:24.212 DEBUG:teuthology.orchestra.run.trial137:> uname -r 2026-02-25T08:05:24.267 INFO:teuthology.orchestra.run.trial137.stdout:5.14.0-686.el9.x86_64 2026-02-25T08:05:24.268 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-25T08:05:24.268 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-25T08:05:24.268 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-25T08:05:25.269 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-25T08:05:25.269 DEBUG:teuthology.orchestra.run.trial137:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-25T08:05:25.300 INFO:teuthology.orchestra.run.trial137.stdout:ttyS1 2026-02-25T08:05:25.324 DEBUG:teuthology.parallel:result is None 2026-02-25T08:05:25.324 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-25T08:05:25.334 INFO:teuthology.task.internal:Creating test directory... 2026-02-25T08:05:25.335 DEBUG:teuthology.orchestra.run.trial137:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-25T08:05:25.381 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-25T08:05:25.392 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-25T08:05:25.405 INFO:teuthology.task.internal:Creating archive directory... 2026-02-25T08:05:25.406 DEBUG:teuthology.orchestra.run.trial137:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-25T08:05:25.439 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-25T08:05:25.450 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-25T08:05:25.450 DEBUG:teuthology.orchestra.run.trial137:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-25T08:05:25.502 INFO:teuthology.orchestra.run.trial137.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T08:05:25.511 INFO:teuthology.orchestra.run.trial137.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T08:05:25.512 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-25T08:05:25.525 INFO:teuthology.task.internal:Configuring sudo... 2026-02-25T08:05:25.525 DEBUG:teuthology.orchestra.run.trial137:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-25T08:05:25.578 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-25T08:05:25.591 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-25T08:05:25.591 DEBUG:teuthology.orchestra.run.trial137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-25T08:05:25.634 DEBUG:teuthology.orchestra.run.trial137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T08:05:25.749 DEBUG:teuthology.orchestra.run.trial137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T08:05:25.772 DEBUG:teuthology.orchestra.run.trial137:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T08:05:25.896 DEBUG:teuthology.orchestra.run.trial137:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T08:05:25.921 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:05:25.921 DEBUG:teuthology.orchestra.run.trial137:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-25T08:05:25.984 DEBUG:teuthology.orchestra.run.trial137:> sudo service rsyslog restart 2026-02-25T08:05:26.052 INFO:teuthology.orchestra.run.trial137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T08:05:26.449 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-25T08:05:26.468 INFO:teuthology.task.internal:Starting timer... 2026-02-25T08:05:26.468 INFO:teuthology.run_tasks:Running task pcp... 2026-02-25T08:05:26.485 INFO:teuthology.run_tasks:Running task selinux... 2026-02-25T08:05:26.497 DEBUG:teuthology.orchestra.run.trial137:> sudo service auditd rotate 2026-02-25T08:05:26.525 INFO:teuthology.orchestra.run.trial137.stdout:Rotating logs: 2026-02-25T08:05:26.526 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-25T08:05:26.526 DEBUG:teuthology.orchestra.run.trial137:> /usr/sbin/getenforce 2026-02-25T08:05:26.583 INFO:teuthology.orchestra.run.trial137.stdout:Permissive 2026-02-25T08:05:26.583 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial137.front.sepia.ceph.com': 'permissive'} 2026-02-25T08:05:26.583 DEBUG:teuthology.orchestra.run.trial137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T08:05:26.647 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:05:26.647 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-25T08:05:26.647 DEBUG:teuthology.orchestra.run.trial137:> sudo /usr/sbin/setenforce permissive 2026-02-25T08:05:26.710 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-25T08:05:26.727 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-25T08:05:26.733 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-25T08:05:26.734 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial137.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-25T08:07:55.604 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial137.front.sepia.ceph.com')] 2026-02-25T08:07:55.605 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial137.front.sepia.ceph.com' 2026-02-25T08:07:55.606 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial137.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T08:07:55.677 DEBUG:teuthology.orchestra.run.trial137:> true 2026-02-25T08:07:55.754 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial137.front.sepia.ceph.com' 2026-02-25T08:07:55.754 INFO:teuthology.run_tasks:Running task clock... 2026-02-25T08:07:55.768 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-25T08:07:55.768 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-25T08:07:55.768 DEBUG:teuthology.orchestra.run.trial137:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T08:07:55.822 INFO:teuthology.orchestra.run.trial137.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-25T08:07:55.834 INFO:teuthology.orchestra.run.trial137.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-25T08:07:55.855 INFO:teuthology.orchestra.run.trial137.stderr:sudo: ntpd: command not found 2026-02-25T08:07:55.865 INFO:teuthology.orchestra.run.trial137.stdout:506 Cannot talk to daemon 2026-02-25T08:07:55.876 INFO:teuthology.orchestra.run.trial137.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-25T08:07:55.888 INFO:teuthology.orchestra.run.trial137.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-25T08:07:55.934 INFO:teuthology.orchestra.run.trial137.stderr:bash: line 1: ntpq: command not found 2026-02-25T08:07:55.936 INFO:teuthology.orchestra.run.trial137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T08:07:55.936 INFO:teuthology.orchestra.run.trial137.stdout:=============================================================================== 2026-02-25T08:07:55.937 INFO:teuthology.run_tasks:Running task install... 2026-02-25T08:07:55.947 DEBUG:teuthology.task.install:project ceph 2026-02-25T08:07:55.947 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}} 2026-02-25T08:07:55.947 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'} 2026-02-25T08:07:55.947 INFO:teuthology.task.install:Using flavor: debug 2026-02-25T08:07:55.950 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-25T08:07:55.951 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-25T08:07:55.951 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819', 'tag': None, 'wait_for_package': False} 2026-02-25T08:07:55.951 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:07:56.088 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/ 2026-02-25T08:07:56.088 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5338.g6aeb6975 2026-02-25T08:07:56.437 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T08:07:56.437 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:07:56.437 DEBUG:teuthology.orchestra.run.trial137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T08:07:56.463 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-25T08:07:56.463 DEBUG:teuthology.orchestra.run.trial137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T08:07:56.528 DEBUG:teuthology.orchestra.run.trial137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T08:07:56.600 DEBUG:teuthology.orchestra.run.trial137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T08:07:56.664 INFO:teuthology.orchestra.run.trial137.stdout:check_obsoletes = 1 2026-02-25T08:07:56.666 DEBUG:teuthology.orchestra.run.trial137:> sudo yum clean all 2026-02-25T08:07:56.813 INFO:teuthology.orchestra.run.trial137.stdout:41 files removed 2026-02-25T08:07:56.828 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-radosgw 2026-02-25T08:07:57.819 INFO:teuthology.orchestra.run.trial137.stdout:ceph packages for x86_64 111 kB/s | 94 kB 00:00 2026-02-25T08:07:58.488 INFO:teuthology.orchestra.run.trial137.stdout:ceph noarch packages 38 kB/s | 24 kB 00:00 2026-02-25T08:07:59.079 INFO:teuthology.orchestra.run.trial137.stdout:ceph source packages 3.7 kB/s | 2.1 kB 00:00 2026-02-25T08:08:02.587 INFO:teuthology.orchestra.run.trial137.stdout:CentOS Stream 9 - BaseOS 2.6 MB/s | 8.9 MB 00:03 2026-02-25T08:08:06.686 INFO:teuthology.orchestra.run.trial137.stdout:CentOS Stream 9 - AppStream 7.6 MB/s | 27 MB 00:03 2026-02-25T08:08:10.061 INFO:teuthology.orchestra.run.trial137.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 8.0 MB 00:00 2026-02-25T08:08:11.050 INFO:teuthology.orchestra.run.trial137.stdout:CentOS Stream 9 - Extras packages 88 kB/s | 20 kB 00:00 2026-02-25T08:08:11.938 INFO:teuthology.orchestra.run.trial137.stdout:Extra Packages for Enterprise Linux 25 MB/s | 20 MB 00:00 2026-02-25T08:08:15.258 INFO:teuthology.orchestra.run.trial137.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-25T08:08:16.477 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:08:16.478 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:08:16.478 INFO:teuthology.orchestra.run.trial137.stdout: Package Arch Version Repository Size 2026-02-25T08:08:16.478 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout:Installing: 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: ceph-radosgw x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 11 M 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout:Upgrading: 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: librados2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 3.3 M 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: librbd1 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 2.7 M 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout:Installing dependencies: 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: ceph-base x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 5.1 M 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: ceph-common x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 21 M 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: ceph-selinux x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 25 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: libcephfs2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 860 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: libradosstriper1 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 263 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: librgw2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 5.6 M 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: python3-ceph-argparse x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 45 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: python3-ceph-common x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 185 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: python3-cephfs x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 204 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: python3-rados x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 324 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: python3-rbd x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 306 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: python3-rgw x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 100 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout:Installing weak dependencies: 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: libcephfs-daemon x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 37 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 26 k 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-25T08:08:16.479 INFO:teuthology.orchestra.run.trial137.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-25T08:08:16.480 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:08:16.480 INFO:teuthology.orchestra.run.trial137.stdout:Transaction Summary 2026-02-25T08:08:16.480 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:08:16.480 INFO:teuthology.orchestra.run.trial137.stdout:Install 35 Packages 2026-02-25T08:08:16.480 INFO:teuthology.orchestra.run.trial137.stdout:Upgrade 2 Packages 2026-02-25T08:08:16.480 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:08:16.480 INFO:teuthology.orchestra.run.trial137.stdout:Total download size: 61 M 2026-02-25T08:08:16.480 INFO:teuthology.orchestra.run.trial137.stdout:Downloading Packages: 2026-02-25T08:08:19.894 INFO:teuthology.orchestra.run.trial137.stdout:(1/37): ceph-base-20.3.0-5338.g6aeb6975.el9.x86 1.6 MB/s | 5.1 MB 00:03 2026-02-25T08:08:19.957 INFO:teuthology.orchestra.run.trial137.stdout:(2/37): ceph-selinux-20.3.0-5338.g6aeb6975.el9. 401 kB/s | 25 kB 00:00 2026-02-25T08:08:20.024 INFO:teuthology.orchestra.run.trial137.stdout:(3/37): libcephfs-daemon-20.3.0-5338.g6aeb6975. 548 kB/s | 37 kB 00:00 2026-02-25T08:08:20.093 INFO:teuthology.orchestra.run.trial137.stdout:(4/37): libcephfs-proxy2-20.3.0-5338.g6aeb6975. 377 kB/s | 26 kB 00:00 2026-02-25T08:08:21.170 INFO:teuthology.orchestra.run.trial137.stdout:(5/37): libcephfs2-20.3.0-5338.g6aeb6975.el9.x8 799 kB/s | 860 kB 00:01 2026-02-25T08:08:21.511 INFO:teuthology.orchestra.run.trial137.stdout:(6/37): libradosstriper1-20.3.0-5338.g6aeb6975. 772 kB/s | 263 kB 00:00 2026-02-25T08:08:30.707 INFO:teuthology.orchestra.run.trial137.stdout:(7/37): librgw2-20.3.0-5338.g6aeb6975.el9.x86_6 627 kB/s | 5.6 MB 00:09 2026-02-25T08:08:30.839 INFO:teuthology.orchestra.run.trial137.stdout:(8/37): python3-ceph-argparse-20.3.0-5338.g6aeb 343 kB/s | 45 kB 00:00 2026-02-25T08:08:31.363 INFO:teuthology.orchestra.run.trial137.stdout:(9/37): python3-ceph-common-20.3.0-5338.g6aeb69 353 kB/s | 185 kB 00:00 2026-02-25T08:08:31.906 INFO:teuthology.orchestra.run.trial137.stdout:(10/37): ceph-radosgw-20.3.0-5338.g6aeb6975.el9 754 kB/s | 11 MB 00:15 2026-02-25T08:08:32.173 INFO:teuthology.orchestra.run.trial137.stdout:(11/37): python3-cephfs-20.3.0-5338.g6aeb6975.e 251 kB/s | 204 kB 00:00 2026-02-25T08:08:32.869 INFO:teuthology.orchestra.run.trial137.stdout:(12/37): python3-rados-20.3.0-5338.g6aeb6975.el 336 kB/s | 324 kB 00:00 2026-02-25T08:08:33.149 INFO:teuthology.orchestra.run.trial137.stdout:(13/37): python3-rgw-20.3.0-5338.g6aeb6975.el9. 358 kB/s | 100 kB 00:00 2026-02-25T08:08:33.208 INFO:teuthology.orchestra.run.trial137.stdout:(14/37): python3-rbd-20.3.0-5338.g6aeb6975.el9. 295 kB/s | 306 kB 00:01 2026-02-25T08:08:33.212 INFO:teuthology.orchestra.run.trial137.stdout:(15/37): mailcap-2.1.49-5.el9.noarch.rpm 517 kB/s | 33 kB 00:00 2026-02-25T08:08:33.267 INFO:teuthology.orchestra.run.trial137.stdout:(16/37): pciutils-3.7.0-7.el9.x86_64.rpm 1.7 MB/s | 93 kB 00:00 2026-02-25T08:08:33.366 INFO:teuthology.orchestra.run.trial137.stdout:(17/37): boost-program-options-1.75.0-13.el9.x8 1.0 MB/s | 104 kB 00:00 2026-02-25T08:08:33.375 INFO:teuthology.orchestra.run.trial137.stdout:(18/37): nvme-cli-2.16-1.el9.x86_64.rpm 6.9 MB/s | 1.2 MB 00:00 2026-02-25T08:08:33.424 INFO:teuthology.orchestra.run.trial137.stdout:(19/37): libnbd-1.20.3-4.el9.x86_64.rpm 2.8 MB/s | 164 kB 00:00 2026-02-25T08:08:33.439 INFO:teuthology.orchestra.run.trial137.stdout:(20/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.5 MB/s | 160 kB 00:00 2026-02-25T08:08:33.465 INFO:teuthology.orchestra.run.trial137.stdout:(21/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.1 MB/s | 45 kB 00:00 2026-02-25T08:08:33.505 INFO:teuthology.orchestra.run.trial137.stdout:(22/37): librdkafka-1.6.1-102.el9.x86_64.rpm 9.6 MB/s | 662 kB 00:00 2026-02-25T08:08:33.552 INFO:teuthology.orchestra.run.trial137.stdout:(23/37): qatlib-25.08.0-2.el9.x86_64.rpm 5.0 MB/s | 240 kB 00:00 2026-02-25T08:08:33.554 INFO:teuthology.orchestra.run.trial137.stdout:(24/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.2 MB/s | 292 kB 00:00 2026-02-25T08:08:33.595 INFO:teuthology.orchestra.run.trial137.stdout:(25/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 1.6 MB/s | 66 kB 00:00 2026-02-25T08:08:33.596 INFO:teuthology.orchestra.run.trial137.stdout:(26/37): qatlib-service-25.08.0-2.el9.x86_64.rp 840 kB/s | 37 kB 00:00 2026-02-25T08:08:33.819 INFO:teuthology.orchestra.run.trial137.stdout:(27/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.3 MB/s | 308 kB 00:00 2026-02-25T08:08:34.180 INFO:teuthology.orchestra.run.trial137.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 69 kB/s | 25 kB 00:00 2026-02-25T08:08:34.422 INFO:teuthology.orchestra.run.trial137.stdout:(29/37): libarrow-9.0.0-15.el9.x86_64.rpm 5.3 MB/s | 4.4 MB 00:00 2026-02-25T08:08:34.474 INFO:teuthology.orchestra.run.trial137.stdout:(30/37): liboath-2.6.12-1.el9.x86_64.rpm 166 kB/s | 49 kB 00:00 2026-02-25T08:08:34.487 INFO:teuthology.orchestra.run.trial137.stdout:(31/37): libunwind-1.6.2-1.el9.x86_64.rpm 1.0 MB/s | 67 kB 00:00 2026-02-25T08:08:34.595 INFO:teuthology.orchestra.run.trial137.stdout:(32/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 6.8 MB/s | 838 kB 00:00 2026-02-25T08:08:34.600 INFO:teuthology.orchestra.run.trial137.stdout:(33/37): re2-20211101-20.el9.x86_64.rpm 1.7 MB/s | 191 kB 00:00 2026-02-25T08:08:34.749 INFO:teuthology.orchestra.run.trial137.stdout:(34/37): thrift-0.15.0-4.el9.x86_64.rpm 10 MB/s | 1.6 MB 00:00 2026-02-25T08:08:39.323 INFO:teuthology.orchestra.run.trial137.stdout:(35/37): librbd1-20.3.0-5338.g6aeb6975.el9.x86_ 602 kB/s | 2.7 MB 00:04 2026-02-25T08:08:42.491 INFO:teuthology.orchestra.run.trial137.stdout:(36/37): librados2-20.3.0-5338.g6aeb6975.el9.x8 426 kB/s | 3.3 MB 00:07 2026-02-25T08:09:04.733 INFO:teuthology.orchestra.run.trial137.stdout:(37/37): ceph-common-20.3.0-5338.g6aeb6975.el9. 444 kB/s | 21 MB 00:48 2026-02-25T08:09:04.737 INFO:teuthology.orchestra.run.trial137.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:09:04.737 INFO:teuthology.orchestra.run.trial137.stdout:Total 1.3 MB/s | 61 MB 00:48 2026-02-25T08:09:04.830 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction check 2026-02-25T08:09:04.848 INFO:teuthology.orchestra.run.trial137.stdout:Transaction check succeeded. 2026-02-25T08:09:04.848 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction test 2026-02-25T08:09:05.094 INFO:teuthology.orchestra.run.trial137.stdout:Transaction test succeeded. 2026-02-25T08:09:05.094 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction 2026-02-25T08:09:05.385 INFO:teuthology.orchestra.run.trial137.stdout: Preparing : 1/1 2026-02-25T08:09:05.535 INFO:teuthology.orchestra.run.trial137.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-25T08:09:05.539 INFO:teuthology.orchestra.run.trial137.stdout: Upgrading : librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/39 2026-02-25T08:09:05.588 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/39 2026-02-25T08:09:05.588 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/39 2026-02-25T08:09:05.599 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/39 2026-02-25T08:09:05.601 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libcephfs-daemon-2:20.3.0-5338.g6aeb6975.el9.x86_6 4/39 2026-02-25T08:09:05.602 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_6 5/39 2026-02-25T08:09:05.629 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_6 5/39 2026-02-25T08:09:05.631 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 6/39 2026-02-25T08:09:05.640 INFO:teuthology.orchestra.run.trial137.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-25T08:09:05.643 INFO:teuthology.orchestra.run.trial137.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-25T08:09:05.646 INFO:teuthology.orchestra.run.trial137.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-25T08:09:05.650 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-25T08:09:05.661 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-ceph-argparse-2:20.3.0-5338.g6aeb6975.el9. 11/39 2026-02-25T08:09:05.674 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/39 2026-02-25T08:09:05.674 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_6 13/39 2026-02-25T08:09:05.698 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_6 13/39 2026-02-25T08:09:05.702 INFO:teuthology.orchestra.run.trial137.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-25T08:09:05.734 INFO:teuthology.orchestra.run.trial137.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-25T08:09:05.736 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-25T08:09:05.742 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-25T08:09:05.744 INFO:teuthology.orchestra.run.trial137.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-25T08:09:05.748 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-25T08:09:05.868 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T08:09:05.869 INFO:teuthology.orchestra.run.trial137.stdout: Upgrading : librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 21/39 2026-02-25T08:09:05.895 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 21/39 2026-02-25T08:09:05.898 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 22/39 2026-02-25T08:09:05.904 INFO:teuthology.orchestra.run.trial137.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-25T08:09:06.162 INFO:teuthology.orchestra.run.trial137.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-25T08:09:06.163 INFO:teuthology.orchestra.run.trial137.stdout: Installing : librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 25/39 2026-02-25T08:09:06.179 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 25/39 2026-02-25T08:09:06.183 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 26/39 2026-02-25T08:09:06.188 INFO:teuthology.orchestra.run.trial137.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-25T08:09:06.191 INFO:teuthology.orchestra.run.trial137.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-25T08:09:06.191 INFO:teuthology.orchestra.run.trial137.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T08:09:06.209 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-25T08:09:06.321 INFO:teuthology.orchestra.run.trial137.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-25T08:09:06.323 INFO:teuthology.orchestra.run.trial137.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T08:09:06.611 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-25T08:09:06.611 INFO:teuthology.orchestra.run.trial137.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-25T08:09:06.611 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:09:06.659 INFO:teuthology.orchestra.run.trial137.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-25T08:09:06.661 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-ceph-common-2:20.3.0-5338.g6aeb6975.el9.x8 33/39 2026-02-25T08:09:07.640 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 34/39 2026-02-25T08:09:07.642 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 34/39 2026-02-25T08:09:07.911 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 34/39 2026-02-25T08:09:07.914 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 35/39 2026-02-25T08:09:07.953 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 35/39 2026-02-25T08:09:07.954 INFO:teuthology.orchestra.run.trial137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-25T08:09:07.954 INFO:teuthology.orchestra.run.trial137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-25T08:09:07.954 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:09:07.957 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 36/39 2026-02-25T08:09:12.444 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 36/39 2026-02-25T08:09:12.444 INFO:teuthology.orchestra.run.trial137.stdout:skipping the directory /sys 2026-02-25T08:09:12.444 INFO:teuthology.orchestra.run.trial137.stdout:skipping the directory /proc 2026-02-25T08:09:12.444 INFO:teuthology.orchestra.run.trial137.stdout:skipping the directory /mnt 2026-02-25T08:09:12.445 INFO:teuthology.orchestra.run.trial137.stdout:skipping the directory /var/tmp 2026-02-25T08:09:12.445 INFO:teuthology.orchestra.run.trial137.stdout:skipping the directory /home 2026-02-25T08:09:12.445 INFO:teuthology.orchestra.run.trial137.stdout:skipping the directory /root 2026-02-25T08:09:12.445 INFO:teuthology.orchestra.run.trial137.stdout:skipping the directory /tmp 2026-02-25T08:09:12.445 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:09:12.955 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 37/39 2026-02-25T08:09:12.978 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 37/39 2026-02-25T08:09:12.978 INFO:teuthology.orchestra.run.trial137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:09:12.978 INFO:teuthology.orchestra.run.trial137.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-25T08:09:12.978 INFO:teuthology.orchestra.run.trial137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T08:09:12.978 INFO:teuthology.orchestra.run.trial137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T08:09:12.979 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:09:12.980 INFO:teuthology.orchestra.run.trial137.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T08:09:12.991 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-25T08:09:12.991 INFO:teuthology.orchestra.run.trial137.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T08:09:13.044 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T08:09:13.045 INFO:teuthology.orchestra.run.trial137.stdout:Creating group 'qat' with GID 992. 2026-02-25T08:09:13.045 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:09:13.340 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/39 2026-02-25T08:09:13.340 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/39 2026-02-25T08:09:13.340 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/39 2026-02-25T08:09:13.341 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 4/39 2026-02-25T08:09:13.341 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libcephfs-daemon-2:20.3.0-5338.g6aeb6975.el9.x86_6 5/39 2026-02-25T08:09:13.341 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_6 6/39 2026-02-25T08:09:13.341 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 7/39 2026-02-25T08:09:13.341 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_6 8/39 2026-02-25T08:09:13.341 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 9/39 2026-02-25T08:09:13.341 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5338.g6aeb6975.el9. 10/39 2026-02-25T08:09:13.341 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-ceph-common-2:20.3.0-5338.g6aeb6975.el9.x8 11/39 2026-02-25T08:09:13.341 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/39 2026-02-25T08:09:13.341 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 13/39 2026-02-25T08:09:13.341 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 14/39 2026-02-25T08:09:13.342 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 15/39 2026-02-25T08:09:13.342 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-25T08:09:13.342 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-25T08:09:13.342 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-25T08:09:13.342 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-25T08:09:13.342 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-25T08:09:13.342 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-25T08:09:13.342 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-25T08:09:13.342 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-25T08:09:13.342 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-25T08:09:13.342 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-25T08:09:13.342 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-25T08:09:13.343 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-25T08:09:13.343 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-25T08:09:13.343 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-25T08:09:13.343 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-25T08:09:13.343 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-25T08:09:13.343 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-25T08:09:13.343 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-25T08:09:13.343 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-25T08:09:13.343 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-25T08:09:13.343 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 36/39 2026-02-25T08:09:13.343 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-25T08:09:13.343 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 38/39 2026-02-25T08:09:13.407 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-25T08:09:13.408 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:09:13.408 INFO:teuthology.orchestra.run.trial137.stdout:Upgraded: 2026-02-25T08:09:13.408 INFO:teuthology.orchestra.run.trial137.stdout: librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.408 INFO:teuthology.orchestra.run.trial137.stdout: librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.408 INFO:teuthology.orchestra.run.trial137.stdout:Installed: 2026-02-25T08:09:13.408 INFO:teuthology.orchestra.run.trial137.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-25T08:09:13.409 INFO:teuthology.orchestra.run.trial137.stdout: ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.409 INFO:teuthology.orchestra.run.trial137.stdout: ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.409 INFO:teuthology.orchestra.run.trial137.stdout: ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.409 INFO:teuthology.orchestra.run.trial137.stdout: ceph-selinux-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.409 INFO:teuthology.orchestra.run.trial137.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-25T08:09:13.409 INFO:teuthology.orchestra.run.trial137.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-25T08:09:13.409 INFO:teuthology.orchestra.run.trial137.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-25T08:09:13.409 INFO:teuthology.orchestra.run.trial137.stdout: libcephfs-daemon-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.409 INFO:teuthology.orchestra.run.trial137.stdout: libcephfs-proxy2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.409 INFO:teuthology.orchestra.run.trial137.stdout: libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.409 INFO:teuthology.orchestra.run.trial137.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-25T08:09:13.410 INFO:teuthology.orchestra.run.trial137.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-25T08:09:13.410 INFO:teuthology.orchestra.run.trial137.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-25T08:09:13.410 INFO:teuthology.orchestra.run.trial137.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-25T08:09:13.410 INFO:teuthology.orchestra.run.trial137.stdout: libradosstriper1-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.410 INFO:teuthology.orchestra.run.trial137.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-25T08:09:13.410 INFO:teuthology.orchestra.run.trial137.stdout: librgw2-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.410 INFO:teuthology.orchestra.run.trial137.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-25T08:09:13.410 INFO:teuthology.orchestra.run.trial137.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-25T08:09:13.410 INFO:teuthology.orchestra.run.trial137.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-25T08:09:13.410 INFO:teuthology.orchestra.run.trial137.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-25T08:09:13.410 INFO:teuthology.orchestra.run.trial137.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-25T08:09:13.410 INFO:teuthology.orchestra.run.trial137.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-25T08:09:13.410 INFO:teuthology.orchestra.run.trial137.stdout: python3-ceph-argparse-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.411 INFO:teuthology.orchestra.run.trial137.stdout: python3-ceph-common-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.411 INFO:teuthology.orchestra.run.trial137.stdout: python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.411 INFO:teuthology.orchestra.run.trial137.stdout: python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.411 INFO:teuthology.orchestra.run.trial137.stdout: python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.411 INFO:teuthology.orchestra.run.trial137.stdout: python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:09:13.411 INFO:teuthology.orchestra.run.trial137.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-25T08:09:13.411 INFO:teuthology.orchestra.run.trial137.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-25T08:09:13.411 INFO:teuthology.orchestra.run.trial137.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-25T08:09:13.411 INFO:teuthology.orchestra.run.trial137.stdout: re2-1:20211101-20.el9.x86_64 2026-02-25T08:09:13.411 INFO:teuthology.orchestra.run.trial137.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-25T08:09:13.411 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:09:13.411 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:09:13.448 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-test 2026-02-25T08:09:13.762 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:58 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:09:13.845 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:09:13.846 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:09:13.846 INFO:teuthology.orchestra.run.trial137.stdout: Package Arch Version Repository Size 2026-02-25T08:09:13.846 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:09:13.846 INFO:teuthology.orchestra.run.trial137.stdout:Installing: 2026-02-25T08:09:13.846 INFO:teuthology.orchestra.run.trial137.stdout: ceph-test x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 127 M 2026-02-25T08:09:13.846 INFO:teuthology.orchestra.run.trial137.stdout:Installing dependencies: 2026-02-25T08:09:13.846 INFO:teuthology.orchestra.run.trial137.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-25T08:09:13.846 INFO:teuthology.orchestra.run.trial137.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-25T08:09:13.846 INFO:teuthology.orchestra.run.trial137.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-25T08:09:13.846 INFO:teuthology.orchestra.run.trial137.stdout: libcephsqlite x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 153 k 2026-02-25T08:09:13.847 INFO:teuthology.orchestra.run.trial137.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-25T08:09:13.847 INFO:teuthology.orchestra.run.trial137.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-25T08:09:13.847 INFO:teuthology.orchestra.run.trial137.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-25T08:09:13.847 INFO:teuthology.orchestra.run.trial137.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-25T08:09:13.847 INFO:teuthology.orchestra.run.trial137.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-25T08:09:13.847 INFO:teuthology.orchestra.run.trial137.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-25T08:09:13.847 INFO:teuthology.orchestra.run.trial137.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-25T08:09:13.847 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:09:13.847 INFO:teuthology.orchestra.run.trial137.stdout:Transaction Summary 2026-02-25T08:09:13.847 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:09:13.847 INFO:teuthology.orchestra.run.trial137.stdout:Install 12 Packages 2026-02-25T08:09:13.848 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:09:13.848 INFO:teuthology.orchestra.run.trial137.stdout:Total download size: 132 M 2026-02-25T08:09:13.848 INFO:teuthology.orchestra.run.trial137.stdout:Installed size: 988 M 2026-02-25T08:09:13.848 INFO:teuthology.orchestra.run.trial137.stdout:Downloading Packages: 2026-02-25T08:09:14.763 INFO:teuthology.orchestra.run.trial137.stdout:[MIRROR] hwloc-libs-2.4.1-6.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/hwloc-libs-2.4.1-6.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:15.033 INFO:teuthology.orchestra.run.trial137.stdout:(1/12): libcephsqlite-20.3.0-5338.g6aeb6975.el9 232 kB/s | 153 kB 00:00 2026-02-25T08:09:15.191 INFO:teuthology.orchestra.run.trial137.stdout:[MIRROR] hwloc-libs-2.4.1-6.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/hwloc-libs-2.4.1-6.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:15.278 INFO:teuthology.orchestra.run.trial137.stdout:(2/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 2.3 MB/s | 2.1 MB 00:00 2026-02-25T08:09:15.385 INFO:teuthology.orchestra.run.trial137.stdout:[MIRROR] libatomic-11.5.0-14.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libatomic-11.5.0-14.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:15.632 INFO:teuthology.orchestra.run.trial137.stdout:[MIRROR] lksctp-tools-1.0.19-2.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/lksctp-tools-1.0.19-2.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:15.743 INFO:teuthology.orchestra.run.trial137.stdout:[MIRROR] libatomic-11.5.0-14.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/libatomic-11.5.0-14.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:15.769 INFO:teuthology.orchestra.run.trial137.stdout:(3/12): libatomic-11.5.0-14.el9.x86_64.rpm 31 kB/s | 23 kB 00:00 2026-02-25T08:09:15.983 INFO:teuthology.orchestra.run.trial137.stdout:[MIRROR] lksctp-tools-1.0.19-2.el9.x86_64.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/lksctp-tools-1.0.19-2.el9.x86_64.rpm (IP: 140.211.166.134) 2026-02-25T08:09:16.009 INFO:teuthology.orchestra.run.trial137.stdout:(4/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 128 kB/s | 94 kB 00:00 2026-02-25T08:09:16.358 INFO:teuthology.orchestra.run.trial137.stdout:(5/12): libasan8-15.2.1-7.el9.x86_64.rpm 891 kB/s | 525 kB 00:00 2026-02-25T08:09:16.459 INFO:teuthology.orchestra.run.trial137.stdout:(6/12): libxslt-1.1.34-12.el9.x86_64.rpm 2.3 MB/s | 233 kB 00:00 2026-02-25T08:09:16.466 INFO:teuthology.orchestra.run.trial137.stdout:(7/12): libubsan-11.5.0-14.el9.x86_64.rpm 385 kB/s | 176 kB 00:00 2026-02-25T08:09:16.643 INFO:teuthology.orchestra.run.trial137.stdout:(8/12): protobuf-3.14.0-17.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2026-02-25T08:09:16.656 INFO:teuthology.orchestra.run.trial137.stdout:(9/12): socat-1.7.4.1-8.el9.x86_64.rpm 1.6 MB/s | 303 kB 00:00 2026-02-25T08:09:16.722 INFO:teuthology.orchestra.run.trial137.stdout:(10/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 804 kB/s | 64 kB 00:00 2026-02-25T08:09:17.092 INFO:teuthology.orchestra.run.trial137.stdout:(11/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 280 kB/s | 122 kB 00:00 2026-02-25T08:17:38.015 INFO:teuthology.orchestra.run.trial137.stdout:(12/12): ceph-test-20.3.0-5338.g6aeb6975.el9.x8 258 kB/s | 127 MB 08:23 2026-02-25T08:17:38.022 INFO:teuthology.orchestra.run.trial137.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:17:38.022 INFO:teuthology.orchestra.run.trial137.stdout:Total 267 kB/s | 132 MB 08:24 2026-02-25T08:17:38.087 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction check 2026-02-25T08:17:38.099 INFO:teuthology.orchestra.run.trial137.stdout:Transaction check succeeded. 2026-02-25T08:17:38.099 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction test 2026-02-25T08:17:38.534 INFO:teuthology.orchestra.run.trial137.stdout:Transaction test succeeded. 2026-02-25T08:17:38.534 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction 2026-02-25T08:17:38.995 INFO:teuthology.orchestra.run.trial137.stdout: Preparing : 1/1 2026-02-25T08:17:39.003 INFO:teuthology.orchestra.run.trial137.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-25T08:17:39.012 INFO:teuthology.orchestra.run.trial137.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-25T08:17:39.017 INFO:teuthology.orchestra.run.trial137.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-25T08:17:39.021 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-25T08:17:39.023 INFO:teuthology.orchestra.run.trial137.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-25T08:17:39.028 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-25T08:17:39.029 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-25T08:17:39.044 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-25T08:17:39.048 INFO:teuthology.orchestra.run.trial137.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-25T08:17:39.057 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-25T08:17:39.066 INFO:teuthology.orchestra.run.trial137.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-25T08:17:39.066 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 11/12 2026-02-25T08:17:46.012 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 11/12 2026-02-25T08:17:46.031 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/12 2026-02-25T08:17:46.803 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 12/12 2026-02-25T08:17:46.804 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/12 2026-02-25T08:17:46.804 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/12 2026-02-25T08:17:46.804 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-25T08:17:46.804 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-25T08:17:46.804 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-25T08:17:46.804 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-25T08:17:46.804 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-25T08:17:46.804 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-25T08:17:46.804 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-25T08:17:46.804 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-25T08:17:46.805 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-25T08:17:46.862 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-25T08:17:46.862 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:17:46.862 INFO:teuthology.orchestra.run.trial137.stdout:Installed: 2026-02-25T08:17:46.862 INFO:teuthology.orchestra.run.trial137.stdout: ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:17:46.862 INFO:teuthology.orchestra.run.trial137.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-25T08:17:46.863 INFO:teuthology.orchestra.run.trial137.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-25T08:17:46.863 INFO:teuthology.orchestra.run.trial137.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-25T08:17:46.863 INFO:teuthology.orchestra.run.trial137.stdout: libcephsqlite-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:17:46.863 INFO:teuthology.orchestra.run.trial137.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-25T08:17:46.863 INFO:teuthology.orchestra.run.trial137.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-25T08:17:46.863 INFO:teuthology.orchestra.run.trial137.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-25T08:17:46.863 INFO:teuthology.orchestra.run.trial137.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-25T08:17:46.863 INFO:teuthology.orchestra.run.trial137.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-25T08:17:46.863 INFO:teuthology.orchestra.run.trial137.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-25T08:17:46.863 INFO:teuthology.orchestra.run.trial137.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-25T08:17:46.863 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:17:46.863 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:17:46.903 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph 2026-02-25T08:17:47.222 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:09:32 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:17:47.309 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:17:47.312 INFO:teuthology.orchestra.run.trial137.stdout:======================================================================================= 2026-02-25T08:17:47.312 INFO:teuthology.orchestra.run.trial137.stdout: Package Arch Version Repository Size 2026-02-25T08:17:47.312 INFO:teuthology.orchestra.run.trial137.stdout:======================================================================================= 2026-02-25T08:17:47.312 INFO:teuthology.orchestra.run.trial137.stdout:Installing: 2026-02-25T08:17:47.312 INFO:teuthology.orchestra.run.trial137.stdout: ceph x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 6.5 k 2026-02-25T08:17:47.312 INFO:teuthology.orchestra.run.trial137.stdout:Installing dependencies: 2026-02-25T08:17:47.313 INFO:teuthology.orchestra.run.trial137.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-25T08:17:47.313 INFO:teuthology.orchestra.run.trial137.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 57 k 2026-02-25T08:17:47.313 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mds x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 2.0 M 2026-02-25T08:17:47.313 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mgr x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 937 k 2026-02-25T08:17:47.313 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 297 k 2026-02-25T08:17:47.313 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mon x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 4.9 M 2026-02-25T08:17:47.313 INFO:teuthology.orchestra.run.trial137.stdout: ceph-osd x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 163 k 2026-02-25T08:17:47.313 INFO:teuthology.orchestra.run.trial137.stdout: ceph-osd-classic x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 15 M 2026-02-25T08:17:47.313 INFO:teuthology.orchestra.run.trial137.stdout: ceph-osd-crimson x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 88 M 2026-02-25T08:17:47.313 INFO:teuthology.orchestra.run.trial137.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 17 k 2026-02-25T08:17:47.313 INFO:teuthology.orchestra.run.trial137.stdout: cephadm noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 1.0 M 2026-02-25T08:17:47.314 INFO:teuthology.orchestra.run.trial137.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-25T08:17:47.314 INFO:teuthology.orchestra.run.trial137.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-25T08:17:47.314 INFO:teuthology.orchestra.run.trial137.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-25T08:17:47.314 INFO:teuthology.orchestra.run.trial137.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-25T08:17:47.314 INFO:teuthology.orchestra.run.trial137.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-25T08:17:47.314 INFO:teuthology.orchestra.run.trial137.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-25T08:17:47.314 INFO:teuthology.orchestra.run.trial137.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-25T08:17:47.314 INFO:teuthology.orchestra.run.trial137.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-25T08:17:47.314 INFO:teuthology.orchestra.run.trial137.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-25T08:17:47.314 INFO:teuthology.orchestra.run.trial137.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-25T08:17:47.314 INFO:teuthology.orchestra.run.trial137.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-25T08:17:47.315 INFO:teuthology.orchestra.run.trial137.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-25T08:17:47.315 INFO:teuthology.orchestra.run.trial137.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-25T08:17:47.315 INFO:teuthology.orchestra.run.trial137.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-25T08:17:47.315 INFO:teuthology.orchestra.run.trial137.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-25T08:17:47.315 INFO:teuthology.orchestra.run.trial137.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-25T08:17:47.315 INFO:teuthology.orchestra.run.trial137.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-25T08:17:47.315 INFO:teuthology.orchestra.run.trial137.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-25T08:17:47.315 INFO:teuthology.orchestra.run.trial137.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-25T08:17:47.315 INFO:teuthology.orchestra.run.trial137.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-25T08:17:47.315 INFO:teuthology.orchestra.run.trial137.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-25T08:17:47.315 INFO:teuthology.orchestra.run.trial137.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-25T08:17:47.315 INFO:teuthology.orchestra.run.trial137.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-25T08:17:47.316 INFO:teuthology.orchestra.run.trial137.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-25T08:17:47.316 INFO:teuthology.orchestra.run.trial137.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-25T08:17:47.316 INFO:teuthology.orchestra.run.trial137.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-25T08:17:47.316 INFO:teuthology.orchestra.run.trial137.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-25T08:17:47.316 INFO:teuthology.orchestra.run.trial137.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-25T08:17:47.316 INFO:teuthology.orchestra.run.trial137.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-25T08:17:47.316 INFO:teuthology.orchestra.run.trial137.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-25T08:17:47.316 INFO:teuthology.orchestra.run.trial137.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-25T08:17:47.316 INFO:teuthology.orchestra.run.trial137.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-25T08:17:47.316 INFO:teuthology.orchestra.run.trial137.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-25T08:17:47.316 INFO:teuthology.orchestra.run.trial137.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-25T08:17:47.316 INFO:teuthology.orchestra.run.trial137.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-25T08:17:47.317 INFO:teuthology.orchestra.run.trial137.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-25T08:17:47.317 INFO:teuthology.orchestra.run.trial137.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-25T08:17:47.317 INFO:teuthology.orchestra.run.trial137.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-25T08:17:47.317 INFO:teuthology.orchestra.run.trial137.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-25T08:17:47.317 INFO:teuthology.orchestra.run.trial137.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-25T08:17:47.317 INFO:teuthology.orchestra.run.trial137.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-25T08:17:47.317 INFO:teuthology.orchestra.run.trial137.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-25T08:17:47.317 INFO:teuthology.orchestra.run.trial137.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-25T08:17:47.317 INFO:teuthology.orchestra.run.trial137.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-25T08:17:47.317 INFO:teuthology.orchestra.run.trial137.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-25T08:17:47.317 INFO:teuthology.orchestra.run.trial137.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-25T08:17:47.317 INFO:teuthology.orchestra.run.trial137.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-25T08:17:47.317 INFO:teuthology.orchestra.run.trial137.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-25T08:17:47.318 INFO:teuthology.orchestra.run.trial137.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-25T08:17:47.318 INFO:teuthology.orchestra.run.trial137.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-25T08:17:47.318 INFO:teuthology.orchestra.run.trial137.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-25T08:17:47.318 INFO:teuthology.orchestra.run.trial137.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-25T08:17:47.318 INFO:teuthology.orchestra.run.trial137.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-25T08:17:47.318 INFO:teuthology.orchestra.run.trial137.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-25T08:17:47.318 INFO:teuthology.orchestra.run.trial137.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-25T08:17:47.318 INFO:teuthology.orchestra.run.trial137.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-25T08:17:47.318 INFO:teuthology.orchestra.run.trial137.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-25T08:17:47.318 INFO:teuthology.orchestra.run.trial137.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-25T08:17:47.318 INFO:teuthology.orchestra.run.trial137.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-25T08:17:47.318 INFO:teuthology.orchestra.run.trial137.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-25T08:17:47.319 INFO:teuthology.orchestra.run.trial137.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-25T08:17:47.319 INFO:teuthology.orchestra.run.trial137.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-25T08:17:47.319 INFO:teuthology.orchestra.run.trial137.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-25T08:17:47.319 INFO:teuthology.orchestra.run.trial137.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-25T08:17:47.319 INFO:teuthology.orchestra.run.trial137.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-25T08:17:47.319 INFO:teuthology.orchestra.run.trial137.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-25T08:17:47.319 INFO:teuthology.orchestra.run.trial137.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-25T08:17:47.319 INFO:teuthology.orchestra.run.trial137.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-25T08:17:47.319 INFO:teuthology.orchestra.run.trial137.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-25T08:17:47.319 INFO:teuthology.orchestra.run.trial137.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-25T08:17:47.319 INFO:teuthology.orchestra.run.trial137.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-25T08:17:47.319 INFO:teuthology.orchestra.run.trial137.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-25T08:17:47.319 INFO:teuthology.orchestra.run.trial137.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-25T08:17:47.320 INFO:teuthology.orchestra.run.trial137.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-25T08:17:47.320 INFO:teuthology.orchestra.run.trial137.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-25T08:17:47.320 INFO:teuthology.orchestra.run.trial137.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-25T08:17:47.320 INFO:teuthology.orchestra.run.trial137.stdout:Installing weak dependencies: 2026-02-25T08:17:47.320 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 184 k 2026-02-25T08:17:47.320 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 51 M 2026-02-25T08:17:47.320 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 7.4 M 2026-02-25T08:17:47.320 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 22 k 2026-02-25T08:17:47.320 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mgr-rook noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 50 k 2026-02-25T08:17:47.320 INFO:teuthology.orchestra.run.trial137.stdout: ceph-volume noarch 2:20.3.0-5338.g6aeb6975.el9 ceph-noarch 301 k 2026-02-25T08:17:47.320 INFO:teuthology.orchestra.run.trial137.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-25T08:17:47.320 INFO:teuthology.orchestra.run.trial137.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-25T08:17:47.321 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:17:47.321 INFO:teuthology.orchestra.run.trial137.stdout:Transaction Summary 2026-02-25T08:17:47.321 INFO:teuthology.orchestra.run.trial137.stdout:======================================================================================= 2026-02-25T08:17:47.321 INFO:teuthology.orchestra.run.trial137.stdout:Install 96 Packages 2026-02-25T08:17:47.321 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:17:47.322 INFO:teuthology.orchestra.run.trial137.stdout:Total download size: 226 M 2026-02-25T08:17:47.322 INFO:teuthology.orchestra.run.trial137.stdout:Installed size: 1.5 G 2026-02-25T08:17:47.322 INFO:teuthology.orchestra.run.trial137.stdout:Downloading Packages: 2026-02-25T08:17:47.892 INFO:teuthology.orchestra.run.trial137.stdout:(1/96): ceph-20.3.0-5338.g6aeb6975.el9.x86_64.r 23 kB/s | 6.5 kB 00:00 2026-02-25T08:17:50.641 INFO:teuthology.orchestra.run.trial137.stdout:(2/96): ceph-mgr-20.3.0-5338.g6aeb6975.el9.x86_ 309 kB/s | 937 kB 00:03 2026-02-25T08:17:51.451 INFO:teuthology.orchestra.run.trial137.stdout:(3/96): ceph-osd-20.3.0-5338.g6aeb6975.el9.x86_ 202 kB/s | 163 kB 00:00 2026-02-25T08:17:52.164 INFO:teuthology.orchestra.run.trial137.stdout:(4/96): ceph-mds-20.3.0-5338.g6aeb6975.el9.x86_ 454 kB/s | 2.0 MB 00:04 2026-02-25T08:18:06.325 INFO:teuthology.orchestra.run.trial137.stdout:(5/96): ceph-mon-20.3.0-5338.g6aeb6975.el9.x86_ 270 kB/s | 4.9 MB 00:18 2026-02-25T08:18:06.749 INFO:teuthology.orchestra.run.trial137.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5338.g6a 135 kB/s | 57 kB 00:00 2026-02-25T08:18:07.797 INFO:teuthology.orchestra.run.trial137.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5338.g6aeb6975. 175 kB/s | 184 kB 00:01 2026-02-25T08:18:55.913 INFO:teuthology.orchestra.run.trial137.stdout:(8/96): ceph-osd-classic-20.3.0-5338.g6aeb6975. 235 kB/s | 15 MB 01:04 2026-02-25T08:19:28.701 INFO:teuthology.orchestra.run.trial137.stdout:(9/96): ceph-mgr-diskprediction-local-20.3.0-53 231 kB/s | 7.4 MB 00:32 2026-02-25T08:19:29.004 INFO:teuthology.orchestra.run.trial137.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5338.g6aeb69 74 kB/s | 22 kB 00:00 2026-02-25T08:19:31.090 INFO:teuthology.orchestra.run.trial137.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5338.g6ae 142 kB/s | 297 kB 00:02 2026-02-25T08:19:31.370 INFO:teuthology.orchestra.run.trial137.stdout:(12/96): ceph-mgr-rook-20.3.0-5338.g6aeb6975.el 180 kB/s | 50 kB 00:00 2026-02-25T08:19:31.440 INFO:teuthology.orchestra.run.trial137.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5338.g6a 248 kB/s | 17 kB 00:00 2026-02-25T08:19:32.773 INFO:teuthology.orchestra.run.trial137.stdout:(14/96): ceph-volume-20.3.0-5338.g6aeb6975.el9. 226 kB/s | 301 kB 00:01 2026-02-25T08:19:38.204 INFO:teuthology.orchestra.run.trial137.stdout:(15/96): cephadm-20.3.0-5338.g6aeb6975.el9.noar 192 kB/s | 1.0 MB 00:05 2026-02-25T08:19:38.321 INFO:teuthology.orchestra.run.trial137.stdout:(16/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 2.9 MB/s | 351 kB 00:00 2026-02-25T08:19:38.343 INFO:teuthology.orchestra.run.trial137.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.8 MB/s | 40 kB 00:00 2026-02-25T08:19:38.365 INFO:teuthology.orchestra.run.trial137.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 3.2 MB/s | 72 kB 00:00 2026-02-25T08:19:38.409 INFO:teuthology.orchestra.run.trial137.stdout:(19/96): libgfortran-11.5.0-14.el9.x86_64.rpm 18 MB/s | 794 kB 00:00 2026-02-25T08:19:38.434 INFO:teuthology.orchestra.run.trial137.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 7.3 MB/s | 184 kB 00:00 2026-02-25T08:19:38.458 INFO:teuthology.orchestra.run.trial137.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 10 MB/s | 253 kB 00:00 2026-02-25T08:19:38.523 INFO:teuthology.orchestra.run.trial137.stdout:(22/96): python3-cryptography-36.0.1-5.el9.x86_ 19 MB/s | 1.2 MB 00:00 2026-02-25T08:19:38.547 INFO:teuthology.orchestra.run.trial137.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 4.4 MB/s | 106 kB 00:00 2026-02-25T08:19:38.571 INFO:teuthology.orchestra.run.trial137.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 5.6 MB/s | 135 kB 00:00 2026-02-25T08:19:38.594 INFO:teuthology.orchestra.run.trial137.stdout:(25/96): python3-pyparsing-2.4.7-9.el9.noarch.r 6.2 MB/s | 150 kB 00:00 2026-02-25T08:19:38.619 INFO:teuthology.orchestra.run.trial137.stdout:(26/96): python3-requests-2.25.1-10.el9.noarch. 5.1 MB/s | 126 kB 00:00 2026-02-25T08:19:38.643 INFO:teuthology.orchestra.run.trial137.stdout:(27/96): python3-urllib3-1.26.5-7.el9.noarch.rp 8.8 MB/s | 218 kB 00:00 2026-02-25T08:19:38.667 INFO:teuthology.orchestra.run.trial137.stdout:(28/96): unzip-6.0-59.el9.x86_64.rpm 7.3 MB/s | 182 kB 00:00 2026-02-25T08:19:38.691 INFO:teuthology.orchestra.run.trial137.stdout:(29/96): zip-3.0-35.el9.x86_64.rpm 11 MB/s | 266 kB 00:00 2026-02-25T08:19:38.722 INFO:teuthology.orchestra.run.trial137.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 961 kB/s | 30 kB 00:00 2026-02-25T08:19:38.782 INFO:teuthology.orchestra.run.trial137.stdout:(31/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 50 MB/s | 3.0 MB 00:00 2026-02-25T08:19:38.791 INFO:teuthology.orchestra.run.trial137.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 1.6 MB/s | 15 kB 00:00 2026-02-25T08:19:38.802 INFO:teuthology.orchestra.run.trial137.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 22 MB/s | 246 kB 00:00 2026-02-25T08:19:38.813 INFO:teuthology.orchestra.run.trial137.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 17 MB/s | 188 kB 00:00 2026-02-25T08:19:38.822 INFO:teuthology.orchestra.run.trial137.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 4.5 MB/s | 42 kB 00:00 2026-02-25T08:19:38.850 INFO:teuthology.orchestra.run.trial137.stdout:(36/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 192 MB/s | 5.3 MB 00:00 2026-02-25T08:19:38.885 INFO:teuthology.orchestra.run.trial137.stdout:(37/96): python3-babel-2.9.1-2.el9.noarch.rpm 175 MB/s | 6.0 MB 00:00 2026-02-25T08:19:38.896 INFO:teuthology.orchestra.run.trial137.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 20 MB/s | 244 kB 00:00 2026-02-25T08:19:38.908 INFO:teuthology.orchestra.run.trial137.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 21 MB/s | 249 kB 00:00 2026-02-25T08:19:38.918 INFO:teuthology.orchestra.run.trial137.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 4.9 MB/s | 48 kB 00:00 2026-02-25T08:19:38.928 INFO:teuthology.orchestra.run.trial137.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 17 MB/s | 177 kB 00:00 2026-02-25T08:19:38.942 INFO:teuthology.orchestra.run.trial137.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 87 MB/s | 1.2 MB 00:00 2026-02-25T08:19:38.952 INFO:teuthology.orchestra.run.trial137.stdout:(43/96): python3-markupsafe-1.1.1-12.el9.x86_64 3.6 MB/s | 35 kB 00:00 2026-02-25T08:19:38.985 INFO:teuthology.orchestra.run.trial137.stdout:(44/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 185 MB/s | 6.1 MB 00:00 2026-02-25T08:19:38.997 INFO:teuthology.orchestra.run.trial137.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 35 MB/s | 442 kB 00:00 2026-02-25T08:19:39.008 INFO:teuthology.orchestra.run.trial137.stdout:(46/96): python3-packaging-20.9-5.el9.noarch.rp 7.4 MB/s | 77 kB 00:00 2026-02-25T08:19:39.018 INFO:teuthology.orchestra.run.trial137.stdout:(47/96): python3-protobuf-3.14.0-17.el9.noarch. 25 MB/s | 267 kB 00:00 2026-02-25T08:19:39.029 INFO:teuthology.orchestra.run.trial137.stdout:(48/96): python3-pyasn1-0.4.8-7.el9.noarch.rpm 14 MB/s | 157 kB 00:00 2026-02-25T08:19:39.041 INFO:teuthology.orchestra.run.trial137.stdout:(49/96): python3-pyasn1-modules-0.4.8-7.el9.noa 23 MB/s | 277 kB 00:00 2026-02-25T08:19:39.051 INFO:teuthology.orchestra.run.trial137.stdout:(50/96): python3-requests-oauthlib-1.3.0-12.el9 5.2 MB/s | 54 kB 00:00 2026-02-25T08:19:39.144 INFO:teuthology.orchestra.run.trial137.stdout:(51/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 208 MB/s | 19 MB 00:00 2026-02-25T08:19:39.153 INFO:teuthology.orchestra.run.trial137.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 4.9 MB/s | 42 kB 00:00 2026-02-25T08:19:39.162 INFO:teuthology.orchestra.run.trial137.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 21 MB/s | 189 kB 00:00 2026-02-25T08:19:39.170 INFO:teuthology.orchestra.run.trial137.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 11 MB/s | 90 kB 00:00 2026-02-25T08:19:39.593 INFO:teuthology.orchestra.run.trial137.stdout:(55/96): lua-devel-5.4.4-4.el9.x86_64.rpm 53 kB/s | 22 kB 00:00 2026-02-25T08:19:40.007 INFO:teuthology.orchestra.run.trial137.stdout:(56/96): protobuf-compiler-3.14.0-17.el9.x86_64 2.0 MB/s | 862 kB 00:00 2026-02-25T08:19:40.515 INFO:teuthology.orchestra.run.trial137.stdout:(57/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.1 MB/s | 551 kB 00:00 2026-02-25T08:19:40.576 INFO:teuthology.orchestra.run.trial137.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 315 kB/s | 19 kB 00:00 2026-02-25T08:19:40.648 INFO:teuthology.orchestra.run.trial137.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 2.1 MB/s | 151 kB 00:00 2026-02-25T08:19:40.750 INFO:teuthology.orchestra.run.trial137.stdout:(60/96): python3-asyncssh-2.13.2-5.el9.noarch.r 5.2 MB/s | 548 kB 00:00 2026-02-25T08:19:40.813 INFO:teuthology.orchestra.run.trial137.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 472 kB/s | 29 kB 00:00 2026-02-25T08:19:40.877 INFO:teuthology.orchestra.run.trial137.stdout:(62/96): python3-backports-tarfile-1.2.0-1.el9. 936 kB/s | 60 kB 00:00 2026-02-25T08:19:40.940 INFO:teuthology.orchestra.run.trial137.stdout:(63/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 686 kB/s | 43 kB 00:00 2026-02-25T08:19:41.003 INFO:teuthology.orchestra.run.trial137.stdout:(64/96): python3-cachetools-4.2.4-1.el9.noarch. 516 kB/s | 32 kB 00:00 2026-02-25T08:19:41.064 INFO:teuthology.orchestra.run.trial137.stdout:(65/96): python3-certifi-2023.05.07-4.el9.noarc 229 kB/s | 14 kB 00:00 2026-02-25T08:19:41.136 INFO:teuthology.orchestra.run.trial137.stdout:(66/96): python3-cheroot-10.0.1-4.el9.noarch.rp 2.4 MB/s | 173 kB 00:00 2026-02-25T08:19:41.219 INFO:teuthology.orchestra.run.trial137.stdout:(67/96): python3-cherrypy-18.6.1-2.el9.noarch.r 4.2 MB/s | 358 kB 00:00 2026-02-25T08:19:41.296 INFO:teuthology.orchestra.run.trial137.stdout:(68/96): python3-google-auth-2.45.0-1.el9.noarc 3.2 MB/s | 254 kB 00:00 2026-02-25T08:19:41.482 INFO:teuthology.orchestra.run.trial137.stdout:(69/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 11 MB/s | 2.0 MB 00:00 2026-02-25T08:19:41.547 INFO:teuthology.orchestra.run.trial137.stdout:(70/96): python3-grpcio-tools-1.46.7-10.el9.x86 2.2 MB/s | 144 kB 00:00 2026-02-25T08:19:41.610 INFO:teuthology.orchestra.run.trial137.stdout:(71/96): python3-influxdb-5.3.1-1.el9.noarch.rp 2.1 MB/s | 139 kB 00:00 2026-02-25T08:19:41.672 INFO:teuthology.orchestra.run.trial137.stdout:(72/96): python3-isodate-0.6.1-3.el9.noarch.rpm 911 kB/s | 56 kB 00:00 2026-02-25T08:19:41.733 INFO:teuthology.orchestra.run.trial137.stdout:(73/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 174 kB/s | 11 kB 00:00 2026-02-25T08:19:41.795 INFO:teuthology.orchestra.run.trial137.stdout:(74/96): python3-jaraco-classes-3.2.1-5.el9.noa 288 kB/s | 18 kB 00:00 2026-02-25T08:19:41.856 INFO:teuthology.orchestra.run.trial137.stdout:(75/96): python3-jaraco-collections-3.0.0-8.el9 377 kB/s | 23 kB 00:00 2026-02-25T08:19:41.918 INFO:teuthology.orchestra.run.trial137.stdout:(76/96): python3-jaraco-context-6.0.1-3.el9.noa 320 kB/s | 20 kB 00:00 2026-02-25T08:19:41.979 INFO:teuthology.orchestra.run.trial137.stdout:(77/96): python3-jaraco-functools-3.5.0-2.el9.n 317 kB/s | 19 kB 00:00 2026-02-25T08:19:42.041 INFO:teuthology.orchestra.run.trial137.stdout:(78/96): python3-jaraco-text-4.0.0-2.el9.noarch 423 kB/s | 26 kB 00:00 2026-02-25T08:19:42.124 INFO:teuthology.orchestra.run.trial137.stdout:(79/96): python3-kubernetes-26.1.0-3.el9.noarch 12 MB/s | 1.0 MB 00:00 2026-02-25T08:19:42.186 INFO:teuthology.orchestra.run.trial137.stdout:(80/96): python3-more-itertools-8.12.0-2.el9.no 1.2 MB/s | 79 kB 00:00 2026-02-25T08:19:42.248 INFO:teuthology.orchestra.run.trial137.stdout:(81/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 1.4 MB/s | 86 kB 00:00 2026-02-25T08:19:42.310 INFO:teuthology.orchestra.run.trial137.stdout:(82/96): python3-natsort-7.1.1-5.el9.noarch.rpm 941 kB/s | 58 kB 00:00 2026-02-25T08:19:42.371 INFO:teuthology.orchestra.run.trial137.stdout:(83/96): python3-portend-3.1.0-2.el9.noarch.rpm 269 kB/s | 16 kB 00:00 2026-02-25T08:19:42.434 INFO:teuthology.orchestra.run.trial137.stdout:(84/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.4 MB/s | 90 kB 00:00 2026-02-25T08:19:42.495 INFO:teuthology.orchestra.run.trial137.stdout:(85/96): python3-repoze-lru-0.7-16.el9.noarch.r 501 kB/s | 31 kB 00:00 2026-02-25T08:19:42.559 INFO:teuthology.orchestra.run.trial137.stdout:(86/96): python3-routes-2.5.1-5.el9.noarch.rpm 2.9 MB/s | 188 kB 00:00 2026-02-25T08:19:42.621 INFO:teuthology.orchestra.run.trial137.stdout:(87/96): python3-rsa-4.9-2.el9.noarch.rpm 962 kB/s | 59 kB 00:00 2026-02-25T08:19:42.684 INFO:teuthology.orchestra.run.trial137.stdout:(88/96): python3-saml-1.16.0-1.el9.noarch.rpm 1.9 MB/s | 125 kB 00:00 2026-02-25T08:19:42.745 INFO:teuthology.orchestra.run.trial137.stdout:(89/96): python3-tempora-5.0.0-2.el9.noarch.rpm 583 kB/s | 36 kB 00:00 2026-02-25T08:19:42.808 INFO:teuthology.orchestra.run.trial137.stdout:(90/96): python3-typing-extensions-4.15.0-1.el9 1.4 MB/s | 86 kB 00:00 2026-02-25T08:19:42.871 INFO:teuthology.orchestra.run.trial137.stdout:(91/96): python3-websocket-client-1.2.3-2.el9.n 1.4 MB/s | 90 kB 00:00 2026-02-25T08:19:42.932 INFO:teuthology.orchestra.run.trial137.stdout:(92/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 788 kB/s | 48 kB 00:00 2026-02-25T08:19:42.994 INFO:teuthology.orchestra.run.trial137.stdout:(93/96): python3-xmltodict-0.12.0-15.el9.noarch 361 kB/s | 22 kB 00:00 2026-02-25T08:19:43.055 INFO:teuthology.orchestra.run.trial137.stdout:(94/96): python3-zc-lockfile-2.0-10.el9.noarch. 326 kB/s | 20 kB 00:00 2026-02-25T08:21:47.879 INFO:teuthology.orchestra.run.trial137.stdout:(95/96): ceph-osd-crimson-20.3.0-5338.g6aeb6975 381 kB/s | 88 MB 03:55 2026-02-25T08:22:03.296 INFO:teuthology.orchestra.run.trial137.stdout:(96/96): ceph-mgr-dashboard-20.3.0-5338.g6aeb69 220 kB/s | 51 MB 03:55 2026-02-25T08:22:03.298 INFO:teuthology.orchestra.run.trial137.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:22:03.298 INFO:teuthology.orchestra.run.trial137.stdout:Total 903 kB/s | 226 MB 04:15 2026-02-25T08:22:03.672 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction check 2026-02-25T08:22:03.711 INFO:teuthology.orchestra.run.trial137.stdout:Transaction check succeeded. 2026-02-25T08:22:03.712 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction test 2026-02-25T08:22:04.516 INFO:teuthology.orchestra.run.trial137.stdout:Transaction test succeeded. 2026-02-25T08:22:04.516 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction 2026-02-25T08:22:05.348 INFO:teuthology.orchestra.run.trial137.stdout: Preparing : 1/1 2026-02-25T08:22:05.350 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-25T08:22:05.355 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-25T08:22:05.376 INFO:teuthology.orchestra.run.trial137.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-25T08:22:05.410 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 4/96 2026-02-25T08:22:05.416 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-25T08:22:05.419 INFO:teuthology.orchestra.run.trial137.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-25T08:22:05.423 INFO:teuthology.orchestra.run.trial137.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-25T08:22:05.446 INFO:teuthology.orchestra.run.trial137.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-25T08:22:05.459 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-25T08:22:05.462 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-25T08:22:05.468 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-25T08:22:05.470 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-25T08:22:05.481 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-25T08:22:05.487 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-25T08:22:05.511 INFO:teuthology.orchestra.run.trial137.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-25T08:22:05.515 INFO:teuthology.orchestra.run.trial137.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-25T08:22:05.565 INFO:teuthology.orchestra.run.trial137.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-25T08:22:05.577 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 18/96 2026-02-25T08:22:05.580 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-25T08:22:05.586 INFO:teuthology.orchestra.run.trial137.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-25T08:22:05.591 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-25T08:22:05.596 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-25T08:22:05.599 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-25T08:22:05.613 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-25T08:22:05.618 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-25T08:22:05.623 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-25T08:22:05.634 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-25T08:22:05.644 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-25T08:22:05.650 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-25T08:22:05.669 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-25T08:22:05.680 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-25T08:22:05.696 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-25T08:22:05.700 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-25T08:22:05.708 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-25T08:22:05.748 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-25T08:22:06.070 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-25T08:22:06.080 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-25T08:22:06.084 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-25T08:22:06.090 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-25T08:22:06.092 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-25T08:22:06.119 INFO:teuthology.orchestra.run.trial137.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-25T08:22:06.162 INFO:teuthology.orchestra.run.trial137.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-25T08:22:06.171 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-25T08:22:06.176 INFO:teuthology.orchestra.run.trial137.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-25T08:22:06.180 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-25T08:22:06.185 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-25T08:22:06.189 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-25T08:22:06.196 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-25T08:22:06.199 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-25T08:22:06.227 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-25T08:22:06.237 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-25T08:22:06.243 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-25T08:22:06.250 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-25T08:22:06.284 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-25T08:22:06.507 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-25T08:22:06.531 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-25T08:22:06.535 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-25T08:22:06.594 INFO:teuthology.orchestra.run.trial137.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-25T08:22:06.596 INFO:teuthology.orchestra.run.trial137.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-25T08:22:06.617 INFO:teuthology.orchestra.run.trial137.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-25T08:22:06.943 INFO:teuthology.orchestra.run.trial137.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-25T08:22:07.015 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-25T08:22:07.686 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-25T08:22:07.702 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-25T08:22:07.712 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-25T08:22:07.718 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-25T08:22:07.732 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-25T08:22:07.748 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-25T08:22:07.823 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-25T08:22:07.832 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-25T08:22:07.855 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-25T08:22:07.885 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-25T08:22:07.937 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-25T08:22:07.943 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-25T08:22:07.946 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-25T08:22:07.947 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-25T08:22:07.966 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-25T08:22:07.966 INFO:teuthology.orchestra.run.trial137.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-25T08:22:07.966 INFO:teuthology.orchestra.run.trial137.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-25T08:22:07.966 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:07.973 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-25T08:22:07.999 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-25T08:22:07.999 INFO:teuthology.orchestra.run.trial137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-25T08:22:07.999 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:08.033 INFO:teuthology.orchestra.run.trial137.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-25T08:22:08.618 INFO:teuthology.orchestra.run.trial137.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-25T08:22:14.025 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_6 80/96 2026-02-25T08:22:14.089 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_6 81/96 2026-02-25T08:22:14.091 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 82/96 2026-02-25T08:22:14.105 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 82/96 2026-02-25T08:22:14.105 INFO:teuthology.orchestra.run.trial137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:22:14.105 INFO:teuthology.orchestra.run.trial137.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-25T08:22:14.105 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:14.116 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 83/96 2026-02-25T08:22:14.139 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 83/96 2026-02-25T08:22:14.139 INFO:teuthology.orchestra.run.trial137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:22:14.139 INFO:teuthology.orchestra.run.trial137.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-25T08:22:14.140 INFO:teuthology.orchestra.run.trial137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T08:22:14.140 INFO:teuthology.orchestra.run.trial137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T08:22:14.140 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:14.196 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 84/96 2026-02-25T08:22:14.198 INFO:teuthology.orchestra.run.trial137.stdout: Installing : cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 84/96 2026-02-25T08:22:14.207 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5338.g6aeb6975.el9 85/96 2026-02-25T08:22:14.230 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5338.g6aeb6975.el 86/96 2026-02-25T08:22:14.232 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarc 87/96 2026-02-25T08:22:17.131 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarc 87/96 2026-02-25T08:22:17.137 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noa 88/96 2026-02-25T08:22:17.591 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noa 88/96 2026-02-25T08:22:17.592 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6 89/96 2026-02-25T08:22:17.604 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6 89/96 2026-02-25T08:22:17.604 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noa 90/96 2026-02-25T08:22:17.666 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noa 90/96 2026-02-25T08:22:17.706 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5338.g6aeb6975.el9. 91/96 2026-02-25T08:22:17.707 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 92/96 2026-02-25T08:22:17.728 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 92/96 2026-02-25T08:22:17.728 INFO:teuthology.orchestra.run.trial137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:22:17.728 INFO:teuthology.orchestra.run.trial137.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-25T08:22:17.728 INFO:teuthology.orchestra.run.trial137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T08:22:17.728 INFO:teuthology.orchestra.run.trial137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T08:22:17.728 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:17.739 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 93/96 2026-02-25T08:22:17.951 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 93/96 2026-02-25T08:22:17.952 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 94/96 2026-02-25T08:22:17.972 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 94/96 2026-02-25T08:22:17.972 INFO:teuthology.orchestra.run.trial137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:22:17.972 INFO:teuthology.orchestra.run.trial137.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-25T08:22:17.972 INFO:teuthology.orchestra.run.trial137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T08:22:17.972 INFO:teuthology.orchestra.run.trial137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T08:22:17.972 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:18.054 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 95/96 2026-02-25T08:22:18.073 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 95/96 2026-02-25T08:22:18.073 INFO:teuthology.orchestra.run.trial137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:22:18.073 INFO:teuthology.orchestra.run.trial137.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-25T08:22:18.073 INFO:teuthology.orchestra.run.trial137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T08:22:18.073 INFO:teuthology.orchestra.run.trial137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T08:22:18.074 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:18.074 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 96/96 2026-02-25T08:22:18.080 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_6 96/96 2026-02-25T08:22:18.102 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_6 96/96 2026-02-25T08:22:19.526 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 96/96 2026-02-25T08:22:19.526 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/96 2026-02-25T08:22:19.526 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/96 2026-02-25T08:22:19.526 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 3/96 2026-02-25T08:22:19.526 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 4/96 2026-02-25T08:22:19.527 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 5/96 2026-02-25T08:22:19.527 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_6 6/96 2026-02-25T08:22:19.527 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_6 7/96 2026-02-25T08:22:19.527 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5338.g6aeb6975.el 8/96 2026-02-25T08:22:19.527 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarc 9/96 2026-02-25T08:22:19.527 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noa 10/96 2026-02-25T08:22:19.527 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6 11/96 2026-02-25T08:22:19.527 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noa 12/96 2026-02-25T08:22:19.527 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5338.g6aeb6975.el9. 13/96 2026-02-25T08:22:19.527 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 14/96 2026-02-25T08:22:19.527 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5338.g6aeb6975.el9 15/96 2026-02-25T08:22:19.527 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 16/96 2026-02-25T08:22:19.528 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 17/96 2026-02-25T08:22:19.528 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-25T08:22:19.528 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-25T08:22:19.528 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-25T08:22:19.528 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-25T08:22:19.528 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-25T08:22:19.528 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-25T08:22:19.528 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-25T08:22:19.528 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-25T08:22:19.528 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-25T08:22:19.528 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-25T08:22:19.528 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-25T08:22:19.529 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-25T08:22:19.529 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-25T08:22:19.529 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-25T08:22:19.529 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-25T08:22:19.529 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-25T08:22:19.529 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-25T08:22:19.529 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-25T08:22:19.529 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-25T08:22:19.529 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-25T08:22:19.529 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-25T08:22:19.529 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-25T08:22:19.529 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-25T08:22:19.530 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-25T08:22:19.530 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-25T08:22:19.530 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-25T08:22:19.530 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-25T08:22:19.530 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-25T08:22:19.530 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-25T08:22:19.530 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-25T08:22:19.530 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-25T08:22:19.530 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-25T08:22:19.530 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 50/96 2026-02-25T08:22:19.530 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 51/96 2026-02-25T08:22:19.530 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-25T08:22:19.530 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-25T08:22:19.531 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-25T08:22:19.531 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-25T08:22:19.531 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-25T08:22:19.531 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-25T08:22:19.531 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-25T08:22:19.531 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-25T08:22:19.531 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-25T08:22:19.531 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-25T08:22:19.531 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-25T08:22:19.531 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-25T08:22:19.531 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-25T08:22:19.531 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-25T08:22:19.531 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-25T08:22:19.532 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-25T08:22:19.532 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-25T08:22:19.532 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-25T08:22:19.532 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-25T08:22:19.532 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-25T08:22:19.532 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-25T08:22:19.532 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-25T08:22:19.532 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-25T08:22:19.532 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-25T08:22:19.532 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-25T08:22:19.532 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-25T08:22:19.532 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-25T08:22:19.533 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-25T08:22:19.533 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-25T08:22:19.533 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-25T08:22:19.533 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-25T08:22:19.533 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-25T08:22:19.533 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-25T08:22:19.533 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-25T08:22:19.533 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-25T08:22:19.533 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-25T08:22:19.533 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-25T08:22:19.533 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-25T08:22:19.533 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-25T08:22:19.534 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-25T08:22:19.534 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-25T08:22:19.534 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-25T08:22:19.534 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-25T08:22:19.534 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-25T08:22:19.594 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-25T08:22:19.594 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:19.594 INFO:teuthology.orchestra.run.trial137.stdout:Installed: 2026-02-25T08:22:19.594 INFO:teuthology.orchestra.run.trial137.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-25T08:22:19.594 INFO:teuthology.orchestra.run.trial137.stdout: ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:22:19.594 INFO:teuthology.orchestra.run.trial137.stdout: ceph-grafana-dashboards-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:22:19.595 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mds-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:22:19.595 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:22:19.595 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:22:19.595 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:22:19.595 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:22:19.595 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mgr-k8sevents-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:22:19.595 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mgr-modules-core-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:22:19.595 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:22:19.595 INFO:teuthology.orchestra.run.trial137.stdout: ceph-mon-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:22:19.595 INFO:teuthology.orchestra.run.trial137.stdout: ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:22:19.595 INFO:teuthology.orchestra.run.trial137.stdout: ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:22:19.595 INFO:teuthology.orchestra.run.trial137.stdout: ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:22:19.596 INFO:teuthology.orchestra.run.trial137.stdout: ceph-prometheus-alerts-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:22:19.596 INFO:teuthology.orchestra.run.trial137.stdout: ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:22:19.596 INFO:teuthology.orchestra.run.trial137.stdout: cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch 2026-02-25T08:22:19.596 INFO:teuthology.orchestra.run.trial137.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-25T08:22:19.596 INFO:teuthology.orchestra.run.trial137.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-25T08:22:19.596 INFO:teuthology.orchestra.run.trial137.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-25T08:22:19.596 INFO:teuthology.orchestra.run.trial137.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-25T08:22:19.596 INFO:teuthology.orchestra.run.trial137.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-25T08:22:19.596 INFO:teuthology.orchestra.run.trial137.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-25T08:22:19.596 INFO:teuthology.orchestra.run.trial137.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-25T08:22:19.596 INFO:teuthology.orchestra.run.trial137.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-25T08:22:19.597 INFO:teuthology.orchestra.run.trial137.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-25T08:22:19.597 INFO:teuthology.orchestra.run.trial137.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T08:22:19.597 INFO:teuthology.orchestra.run.trial137.stdout: lua-5.4.4-4.el9.x86_64 2026-02-25T08:22:19.597 INFO:teuthology.orchestra.run.trial137.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-25T08:22:19.597 INFO:teuthology.orchestra.run.trial137.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-25T08:22:19.597 INFO:teuthology.orchestra.run.trial137.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-25T08:22:19.597 INFO:teuthology.orchestra.run.trial137.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-25T08:22:19.597 INFO:teuthology.orchestra.run.trial137.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-25T08:22:19.597 INFO:teuthology.orchestra.run.trial137.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-25T08:22:19.597 INFO:teuthology.orchestra.run.trial137.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-25T08:22:19.597 INFO:teuthology.orchestra.run.trial137.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-25T08:22:19.597 INFO:teuthology.orchestra.run.trial137.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-25T08:22:19.597 INFO:teuthology.orchestra.run.trial137.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-25T08:22:19.598 INFO:teuthology.orchestra.run.trial137.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-25T08:22:19.598 INFO:teuthology.orchestra.run.trial137.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-25T08:22:19.598 INFO:teuthology.orchestra.run.trial137.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-25T08:22:19.598 INFO:teuthology.orchestra.run.trial137.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-25T08:22:19.598 INFO:teuthology.orchestra.run.trial137.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-25T08:22:19.598 INFO:teuthology.orchestra.run.trial137.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-25T08:22:19.598 INFO:teuthology.orchestra.run.trial137.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-25T08:22:19.598 INFO:teuthology.orchestra.run.trial137.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-25T08:22:19.598 INFO:teuthology.orchestra.run.trial137.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-25T08:22:19.598 INFO:teuthology.orchestra.run.trial137.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-25T08:22:19.598 INFO:teuthology.orchestra.run.trial137.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-25T08:22:19.598 INFO:teuthology.orchestra.run.trial137.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-25T08:22:19.598 INFO:teuthology.orchestra.run.trial137.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-25T08:22:19.599 INFO:teuthology.orchestra.run.trial137.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-25T08:22:19.599 INFO:teuthology.orchestra.run.trial137.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-25T08:22:19.599 INFO:teuthology.orchestra.run.trial137.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-25T08:22:19.599 INFO:teuthology.orchestra.run.trial137.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-25T08:22:19.599 INFO:teuthology.orchestra.run.trial137.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-25T08:22:19.599 INFO:teuthology.orchestra.run.trial137.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-25T08:22:19.599 INFO:teuthology.orchestra.run.trial137.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-25T08:22:19.599 INFO:teuthology.orchestra.run.trial137.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-25T08:22:19.599 INFO:teuthology.orchestra.run.trial137.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T08:22:19.599 INFO:teuthology.orchestra.run.trial137.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-25T08:22:19.599 INFO:teuthology.orchestra.run.trial137.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-25T08:22:19.599 INFO:teuthology.orchestra.run.trial137.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-25T08:22:19.599 INFO:teuthology.orchestra.run.trial137.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-25T08:22:19.600 INFO:teuthology.orchestra.run.trial137.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-25T08:22:19.600 INFO:teuthology.orchestra.run.trial137.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-25T08:22:19.600 INFO:teuthology.orchestra.run.trial137.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-25T08:22:19.600 INFO:teuthology.orchestra.run.trial137.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-25T08:22:19.600 INFO:teuthology.orchestra.run.trial137.stdout: python3-ply-3.11-14.el9.noarch 2026-02-25T08:22:19.600 INFO:teuthology.orchestra.run.trial137.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-25T08:22:19.600 INFO:teuthology.orchestra.run.trial137.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-25T08:22:19.600 INFO:teuthology.orchestra.run.trial137.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-25T08:22:19.600 INFO:teuthology.orchestra.run.trial137.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-25T08:22:19.600 INFO:teuthology.orchestra.run.trial137.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-25T08:22:19.600 INFO:teuthology.orchestra.run.trial137.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-25T08:22:19.600 INFO:teuthology.orchestra.run.trial137.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-25T08:22:19.600 INFO:teuthology.orchestra.run.trial137.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-25T08:22:19.601 INFO:teuthology.orchestra.run.trial137.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-25T08:22:19.601 INFO:teuthology.orchestra.run.trial137.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-25T08:22:19.601 INFO:teuthology.orchestra.run.trial137.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-25T08:22:19.601 INFO:teuthology.orchestra.run.trial137.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-25T08:22:19.601 INFO:teuthology.orchestra.run.trial137.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-25T08:22:19.601 INFO:teuthology.orchestra.run.trial137.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-25T08:22:19.601 INFO:teuthology.orchestra.run.trial137.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-25T08:22:19.601 INFO:teuthology.orchestra.run.trial137.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-25T08:22:19.601 INFO:teuthology.orchestra.run.trial137.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-25T08:22:19.601 INFO:teuthology.orchestra.run.trial137.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-25T08:22:19.601 INFO:teuthology.orchestra.run.trial137.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-25T08:22:19.601 INFO:teuthology.orchestra.run.trial137.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-25T08:22:19.601 INFO:teuthology.orchestra.run.trial137.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-25T08:22:19.602 INFO:teuthology.orchestra.run.trial137.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-25T08:22:19.602 INFO:teuthology.orchestra.run.trial137.stdout: unzip-6.0-59.el9.x86_64 2026-02-25T08:22:19.602 INFO:teuthology.orchestra.run.trial137.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-25T08:22:19.602 INFO:teuthology.orchestra.run.trial137.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-25T08:22:19.602 INFO:teuthology.orchestra.run.trial137.stdout: zip-3.0-35.el9.x86_64 2026-02-25T08:22:19.602 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:19.602 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:19.664 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-base 2026-02-25T08:22:19.988 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:04 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:20.053 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:22:20.072 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:20.073 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:20.073 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:20.094 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install cephadm 2026-02-25T08:22:20.413 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:05 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:20.478 INFO:teuthology.orchestra.run.trial137.stdout:Package cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:22:20.497 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:20.497 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:20.497 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:20.519 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T08:22:20.841 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:05 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:20.925 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:20.926 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:20.926 INFO:teuthology.orchestra.run.trial137.stdout: Package Arch Version Repo Size 2026-02-25T08:22:20.926 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:20.926 INFO:teuthology.orchestra.run.trial137.stdout:Installing: 2026-02-25T08:22:20.926 INFO:teuthology.orchestra.run.trial137.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 149 k 2026-02-25T08:22:20.926 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:20.927 INFO:teuthology.orchestra.run.trial137.stdout:Transaction Summary 2026-02-25T08:22:20.927 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:20.927 INFO:teuthology.orchestra.run.trial137.stdout:Install 1 Package 2026-02-25T08:22:20.927 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:20.927 INFO:teuthology.orchestra.run.trial137.stdout:Total download size: 149 k 2026-02-25T08:22:20.927 INFO:teuthology.orchestra.run.trial137.stdout:Installed size: 455 k 2026-02-25T08:22:20.927 INFO:teuthology.orchestra.run.trial137.stdout:Downloading Packages: 2026-02-25T08:22:21.573 INFO:teuthology.orchestra.run.trial137.stdout:ceph-immutable-object-cache-20.3.0-5338.g6aeb69 231 kB/s | 149 kB 00:00 2026-02-25T08:22:21.574 INFO:teuthology.orchestra.run.trial137.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:22:21.574 INFO:teuthology.orchestra.run.trial137.stdout:Total 230 kB/s | 149 kB 00:00 2026-02-25T08:22:21.574 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction check 2026-02-25T08:22:21.579 INFO:teuthology.orchestra.run.trial137.stdout:Transaction check succeeded. 2026-02-25T08:22:21.579 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction test 2026-02-25T08:22:21.610 INFO:teuthology.orchestra.run.trial137.stdout:Transaction test succeeded. 2026-02-25T08:22:21.610 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction 2026-02-25T08:22:21.666 INFO:teuthology.orchestra.run.trial137.stdout: Preparing : 1/1 2026-02-25T08:22:21.668 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975. 1/1 2026-02-25T08:22:21.690 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975. 1/1 2026-02-25T08:22:21.691 INFO:teuthology.orchestra.run.trial137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:22:21.691 INFO:teuthology.orchestra.run.trial137.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-25T08:22:21.691 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:22.012 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975. 1/1 2026-02-25T08:22:22.012 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:22.012 INFO:teuthology.orchestra.run.trial137.stdout:Installed: 2026-02-25T08:22:22.012 INFO:teuthology.orchestra.run.trial137.stdout: ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:22:22.012 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:22.013 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:22.043 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-mgr 2026-02-25T08:22:22.366 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:07 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:22.431 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:22:22.450 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:22.451 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:22.451 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:22.473 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T08:22:22.790 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:07 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:22.855 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:22:22.874 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:22.875 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:22.875 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:22.897 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T08:22:23.217 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:08 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:23.282 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:22:23.301 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:23.301 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:23.301 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:23.323 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-mgr-rook 2026-02-25T08:22:23.641 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:08 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:23.706 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:22:23.725 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:23.726 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:23.726 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:23.747 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T08:22:24.066 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:09 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:24.131 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:22:24.150 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:24.151 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:24.151 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:24.173 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-osd 2026-02-25T08:22:24.490 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:09 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:24.556 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:22:24.574 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:24.575 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:24.575 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:24.597 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-osd-classic 2026-02-25T08:22:24.912 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:09 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:24.978 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:22:24.997 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:24.997 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:24.997 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:25.019 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-fuse 2026-02-25T08:22:25.335 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:10 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:25.419 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:25.420 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:25.420 INFO:teuthology.orchestra.run.trial137.stdout: Package Arch Version Repo Size 2026-02-25T08:22:25.420 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:25.420 INFO:teuthology.orchestra.run.trial137.stdout:Installing: 2026-02-25T08:22:25.420 INFO:teuthology.orchestra.run.trial137.stdout: ceph-fuse x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 914 k 2026-02-25T08:22:25.420 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:25.420 INFO:teuthology.orchestra.run.trial137.stdout:Transaction Summary 2026-02-25T08:22:25.420 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:25.421 INFO:teuthology.orchestra.run.trial137.stdout:Install 1 Package 2026-02-25T08:22:25.421 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:25.421 INFO:teuthology.orchestra.run.trial137.stdout:Total download size: 914 k 2026-02-25T08:22:25.421 INFO:teuthology.orchestra.run.trial137.stdout:Installed size: 2.9 M 2026-02-25T08:22:25.421 INFO:teuthology.orchestra.run.trial137.stdout:Downloading Packages: 2026-02-25T08:22:26.445 INFO:teuthology.orchestra.run.trial137.stdout:ceph-fuse-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 891 kB/s | 914 kB 00:01 2026-02-25T08:22:26.446 INFO:teuthology.orchestra.run.trial137.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:22:26.446 INFO:teuthology.orchestra.run.trial137.stdout:Total 890 kB/s | 914 kB 00:01 2026-02-25T08:22:26.446 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction check 2026-02-25T08:22:26.451 INFO:teuthology.orchestra.run.trial137.stdout:Transaction check succeeded. 2026-02-25T08:22:26.451 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction test 2026-02-25T08:22:26.485 INFO:teuthology.orchestra.run.trial137.stdout:Transaction test succeeded. 2026-02-25T08:22:26.485 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction 2026-02-25T08:22:26.575 INFO:teuthology.orchestra.run.trial137.stdout: Preparing : 1/1 2026-02-25T08:22:26.593 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:26.824 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:26.885 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:26.886 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:26.886 INFO:teuthology.orchestra.run.trial137.stdout:Installed: 2026-02-25T08:22:26.886 INFO:teuthology.orchestra.run.trial137.stdout: ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:22:26.886 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:26.886 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:26.914 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-volume 2026-02-25T08:22:27.233 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:12 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:27.297 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:22:27.316 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:27.317 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:27.317 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:27.338 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install librados-devel 2026-02-25T08:22:27.652 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:12 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:27.736 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:27.736 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:27.736 INFO:teuthology.orchestra.run.trial137.stdout: Package Arch Version Repo Size 2026-02-25T08:22:27.736 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:27.736 INFO:teuthology.orchestra.run.trial137.stdout:Installing: 2026-02-25T08:22:27.736 INFO:teuthology.orchestra.run.trial137.stdout: librados-devel x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 124 k 2026-02-25T08:22:27.736 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:27.737 INFO:teuthology.orchestra.run.trial137.stdout:Transaction Summary 2026-02-25T08:22:27.737 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:27.737 INFO:teuthology.orchestra.run.trial137.stdout:Install 1 Package 2026-02-25T08:22:27.737 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:27.737 INFO:teuthology.orchestra.run.trial137.stdout:Total download size: 124 k 2026-02-25T08:22:27.737 INFO:teuthology.orchestra.run.trial137.stdout:Installed size: 434 k 2026-02-25T08:22:27.737 INFO:teuthology.orchestra.run.trial137.stdout:Downloading Packages: 2026-02-25T08:22:28.301 INFO:teuthology.orchestra.run.trial137.stdout:librados-devel-20.3.0-5338.g6aeb6975.el9.x86_64 219 kB/s | 124 kB 00:00 2026-02-25T08:22:28.301 INFO:teuthology.orchestra.run.trial137.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:22:28.302 INFO:teuthology.orchestra.run.trial137.stdout:Total 219 kB/s | 124 kB 00:00 2026-02-25T08:22:28.302 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction check 2026-02-25T08:22:28.304 INFO:teuthology.orchestra.run.trial137.stdout:Transaction check succeeded. 2026-02-25T08:22:28.304 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction test 2026-02-25T08:22:28.336 INFO:teuthology.orchestra.run.trial137.stdout:Transaction test succeeded. 2026-02-25T08:22:28.336 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction 2026-02-25T08:22:28.391 INFO:teuthology.orchestra.run.trial137.stdout: Preparing : 1/1 2026-02-25T08:22:28.409 INFO:teuthology.orchestra.run.trial137.stdout: Installing : librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:28.495 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:28.558 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:28.558 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:28.558 INFO:teuthology.orchestra.run.trial137.stdout:Installed: 2026-02-25T08:22:28.558 INFO:teuthology.orchestra.run.trial137.stdout: librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:22:28.558 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:28.558 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:28.585 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install libcephfs2 2026-02-25T08:22:28.902 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:13 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:28.967 INFO:teuthology.orchestra.run.trial137.stdout:Package libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:22:28.985 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:28.986 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:28.986 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:29.007 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install libcephfs-devel 2026-02-25T08:22:29.327 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:14 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:29.411 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:29.412 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:29.412 INFO:teuthology.orchestra.run.trial137.stdout: Package Arch Version Repo Size 2026-02-25T08:22:29.412 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:29.412 INFO:teuthology.orchestra.run.trial137.stdout:Installing: 2026-02-25T08:22:29.412 INFO:teuthology.orchestra.run.trial137.stdout: libcephfs-devel x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 37 k 2026-02-25T08:22:29.413 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:29.413 INFO:teuthology.orchestra.run.trial137.stdout:Transaction Summary 2026-02-25T08:22:29.413 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:29.413 INFO:teuthology.orchestra.run.trial137.stdout:Install 1 Package 2026-02-25T08:22:29.413 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:29.413 INFO:teuthology.orchestra.run.trial137.stdout:Total download size: 37 k 2026-02-25T08:22:29.413 INFO:teuthology.orchestra.run.trial137.stdout:Installed size: 171 k 2026-02-25T08:22:29.413 INFO:teuthology.orchestra.run.trial137.stdout:Downloading Packages: 2026-02-25T08:22:29.770 INFO:teuthology.orchestra.run.trial137.stdout:libcephfs-devel-20.3.0-5338.g6aeb6975.el9.x86_6 104 kB/s | 37 kB 00:00 2026-02-25T08:22:29.771 INFO:teuthology.orchestra.run.trial137.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:22:29.771 INFO:teuthology.orchestra.run.trial137.stdout:Total 104 kB/s | 37 kB 00:00 2026-02-25T08:22:29.771 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction check 2026-02-25T08:22:29.774 INFO:teuthology.orchestra.run.trial137.stdout:Transaction check succeeded. 2026-02-25T08:22:29.774 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction test 2026-02-25T08:22:29.776 INFO:teuthology.orchestra.run.trial137.stdout:Transaction test succeeded. 2026-02-25T08:22:29.776 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction 2026-02-25T08:22:29.799 INFO:teuthology.orchestra.run.trial137.stdout: Preparing : 1/1 2026-02-25T08:22:29.816 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:29.876 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:29.938 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:29.938 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:29.938 INFO:teuthology.orchestra.run.trial137.stdout:Installed: 2026-02-25T08:22:29.938 INFO:teuthology.orchestra.run.trial137.stdout: libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:22:29.938 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:29.938 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:29.964 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install librados2 2026-02-25T08:22:30.278 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:15 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:30.344 INFO:teuthology.orchestra.run.trial137.stdout:Package librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:22:30.361 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:30.362 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:30.362 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:30.384 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install librbd1 2026-02-25T08:22:30.700 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:15 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:30.766 INFO:teuthology.orchestra.run.trial137.stdout:Package librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:22:30.783 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:30.784 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:30.784 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:30.805 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install python3-rados 2026-02-25T08:22:31.123 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:16 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:31.188 INFO:teuthology.orchestra.run.trial137.stdout:Package python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:22:31.206 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:31.206 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:31.206 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:31.228 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install python3-rgw 2026-02-25T08:22:31.545 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:16 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:31.610 INFO:teuthology.orchestra.run.trial137.stdout:Package python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:22:31.627 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:31.628 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:31.628 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:31.649 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install python3-cephfs 2026-02-25T08:22:31.967 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:16 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:32.033 INFO:teuthology.orchestra.run.trial137.stdout:Package python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:22:32.050 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:32.051 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:32.051 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:32.073 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install python3-rbd 2026-02-25T08:22:32.393 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:17 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:32.458 INFO:teuthology.orchestra.run.trial137.stdout:Package python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:22:32.476 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:32.476 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:32.476 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:32.498 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install rbd-fuse 2026-02-25T08:22:32.819 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:17 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:32.903 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:32.903 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:32.903 INFO:teuthology.orchestra.run.trial137.stdout: Package Architecture Version Repository Size 2026-02-25T08:22:32.903 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:32.903 INFO:teuthology.orchestra.run.trial137.stdout:Installing: 2026-02-25T08:22:32.903 INFO:teuthology.orchestra.run.trial137.stdout: rbd-fuse x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 86 k 2026-02-25T08:22:32.903 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:32.903 INFO:teuthology.orchestra.run.trial137.stdout:Transaction Summary 2026-02-25T08:22:32.903 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:32.903 INFO:teuthology.orchestra.run.trial137.stdout:Install 1 Package 2026-02-25T08:22:32.903 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:32.904 INFO:teuthology.orchestra.run.trial137.stdout:Total download size: 86 k 2026-02-25T08:22:32.904 INFO:teuthology.orchestra.run.trial137.stdout:Installed size: 236 k 2026-02-25T08:22:32.904 INFO:teuthology.orchestra.run.trial137.stdout:Downloading Packages: 2026-02-25T08:22:33.699 INFO:teuthology.orchestra.run.trial137.stdout:rbd-fuse-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 108 kB/s | 86 kB 00:00 2026-02-25T08:22:33.699 INFO:teuthology.orchestra.run.trial137.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:22:33.699 INFO:teuthology.orchestra.run.trial137.stdout:Total 108 kB/s | 86 kB 00:00 2026-02-25T08:22:33.699 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction check 2026-02-25T08:22:33.704 INFO:teuthology.orchestra.run.trial137.stdout:Transaction check succeeded. 2026-02-25T08:22:33.704 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction test 2026-02-25T08:22:33.735 INFO:teuthology.orchestra.run.trial137.stdout:Transaction test succeeded. 2026-02-25T08:22:33.736 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction 2026-02-25T08:22:33.787 INFO:teuthology.orchestra.run.trial137.stdout: Preparing : 1/1 2026-02-25T08:22:33.805 INFO:teuthology.orchestra.run.trial137.stdout: Installing : rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:33.890 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:33.953 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:33.953 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:33.953 INFO:teuthology.orchestra.run.trial137.stdout:Installed: 2026-02-25T08:22:33.953 INFO:teuthology.orchestra.run.trial137.stdout: rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:22:33.953 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:33.953 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:33.982 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install rbd-mirror 2026-02-25T08:22:34.302 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:19 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:34.384 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:34.385 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:34.385 INFO:teuthology.orchestra.run.trial137.stdout: Package Arch Version Repo Size 2026-02-25T08:22:34.385 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:34.385 INFO:teuthology.orchestra.run.trial137.stdout:Installing: 2026-02-25T08:22:34.386 INFO:teuthology.orchestra.run.trial137.stdout: rbd-mirror x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 2.6 M 2026-02-25T08:22:34.386 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:34.386 INFO:teuthology.orchestra.run.trial137.stdout:Transaction Summary 2026-02-25T08:22:34.386 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:34.386 INFO:teuthology.orchestra.run.trial137.stdout:Install 1 Package 2026-02-25T08:22:34.386 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:34.386 INFO:teuthology.orchestra.run.trial137.stdout:Total download size: 2.6 M 2026-02-25T08:22:34.386 INFO:teuthology.orchestra.run.trial137.stdout:Installed size: 11 M 2026-02-25T08:22:34.386 INFO:teuthology.orchestra.run.trial137.stdout:Downloading Packages: 2026-02-25T08:22:46.834 INFO:teuthology.orchestra.run.trial137.stdout:rbd-mirror-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 213 kB/s | 2.6 MB 00:12 2026-02-25T08:22:46.835 INFO:teuthology.orchestra.run.trial137.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:22:46.835 INFO:teuthology.orchestra.run.trial137.stdout:Total 213 kB/s | 2.6 MB 00:12 2026-02-25T08:22:46.836 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction check 2026-02-25T08:22:46.841 INFO:teuthology.orchestra.run.trial137.stdout:Transaction check succeeded. 2026-02-25T08:22:46.841 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction test 2026-02-25T08:22:46.879 INFO:teuthology.orchestra.run.trial137.stdout:Transaction test succeeded. 2026-02-25T08:22:46.879 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction 2026-02-25T08:22:47.042 INFO:teuthology.orchestra.run.trial137.stdout: Preparing : 1/1 2026-02-25T08:22:47.044 INFO:teuthology.orchestra.run.trial137.stdout: Installing : rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:47.067 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:47.067 INFO:teuthology.orchestra.run.trial137.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T08:22:47.067 INFO:teuthology.orchestra.run.trial137.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-25T08:22:47.067 INFO:teuthology.orchestra.run.trial137.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T08:22:47.067 INFO:teuthology.orchestra.run.trial137.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T08:22:47.068 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:47.388 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:47.389 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:47.389 INFO:teuthology.orchestra.run.trial137.stdout:Installed: 2026-02-25T08:22:47.389 INFO:teuthology.orchestra.run.trial137.stdout: rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:22:47.389 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:47.389 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:47.416 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install rbd-nbd 2026-02-25T08:22:47.740 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:32 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:47.823 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:47.824 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:47.824 INFO:teuthology.orchestra.run.trial137.stdout: Package Architecture Version Repository Size 2026-02-25T08:22:47.825 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:47.825 INFO:teuthology.orchestra.run.trial137.stdout:Installing: 2026-02-25T08:22:47.825 INFO:teuthology.orchestra.run.trial137.stdout: rbd-nbd x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 166 k 2026-02-25T08:22:47.825 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:47.825 INFO:teuthology.orchestra.run.trial137.stdout:Transaction Summary 2026-02-25T08:22:47.825 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:47.825 INFO:teuthology.orchestra.run.trial137.stdout:Install 1 Package 2026-02-25T08:22:47.825 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:47.825 INFO:teuthology.orchestra.run.trial137.stdout:Total download size: 166 k 2026-02-25T08:22:47.825 INFO:teuthology.orchestra.run.trial137.stdout:Installed size: 496 k 2026-02-25T08:22:47.826 INFO:teuthology.orchestra.run.trial137.stdout:Downloading Packages: 2026-02-25T08:22:48.696 INFO:teuthology.orchestra.run.trial137.stdout:rbd-nbd-20.3.0-5338.g6aeb6975.el9.x86_64.rpm 191 kB/s | 166 kB 00:00 2026-02-25T08:22:48.698 INFO:teuthology.orchestra.run.trial137.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:22:48.698 INFO:teuthology.orchestra.run.trial137.stdout:Total 190 kB/s | 166 kB 00:00 2026-02-25T08:22:48.699 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction check 2026-02-25T08:22:48.704 INFO:teuthology.orchestra.run.trial137.stdout:Transaction check succeeded. 2026-02-25T08:22:48.704 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction test 2026-02-25T08:22:48.735 INFO:teuthology.orchestra.run.trial137.stdout:Transaction test succeeded. 2026-02-25T08:22:48.736 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction 2026-02-25T08:22:48.792 INFO:teuthology.orchestra.run.trial137.stdout: Preparing : 1/1 2026-02-25T08:22:48.809 INFO:teuthology.orchestra.run.trial137.stdout: Installing : rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:48.895 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:48.957 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/1 2026-02-25T08:22:48.958 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:48.958 INFO:teuthology.orchestra.run.trial137.stdout:Installed: 2026-02-25T08:22:48.958 INFO:teuthology.orchestra.run.trial137.stdout: rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:22:48.958 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:48.958 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:48.986 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-osd-crimson 2026-02-25T08:22:49.303 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:34 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:49.369 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-osd-crimson-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:22:49.386 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:49.387 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:22:49.387 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:22:49.409 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-25T08:22:49.732 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:14:34 ago on Wed 25 Feb 2026 08:08:15 AM UTC. 2026-02-25T08:22:49.815 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:22:49.816 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:49.816 INFO:teuthology.orchestra.run.trial137.stdout: Package Arch Version Repo Size 2026-02-25T08:22:49.816 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:49.817 INFO:teuthology.orchestra.run.trial137.stdout:Installing: 2026-02-25T08:22:49.817 INFO:teuthology.orchestra.run.trial137.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 433 M 2026-02-25T08:22:49.817 INFO:teuthology.orchestra.run.trial137.stdout:Installing weak dependencies: 2026-02-25T08:22:49.817 INFO:teuthology.orchestra.run.trial137.stdout: ceph-debugsource x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 12 M 2026-02-25T08:22:49.817 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:49.817 INFO:teuthology.orchestra.run.trial137.stdout:Transaction Summary 2026-02-25T08:22:49.817 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:22:49.817 INFO:teuthology.orchestra.run.trial137.stdout:Install 2 Packages 2026-02-25T08:22:49.817 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:22:49.817 INFO:teuthology.orchestra.run.trial137.stdout:Total download size: 445 M 2026-02-25T08:22:49.818 INFO:teuthology.orchestra.run.trial137.stdout:Installed size: 2.7 G 2026-02-25T08:22:49.818 INFO:teuthology.orchestra.run.trial137.stdout:Downloading Packages: 2026-02-25T08:22:59.021 INFO:teuthology.orchestra.run.trial137.stdout:(1/2): ceph-debugsource-20.3.0-5338.g6aeb6975.e 1.3 MB/s | 12 MB 00:09 2026-02-25T08:39:45.602 INFO:teuthology.orchestra.run.trial137.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5338.g 436 kB/s | 433 MB 16:55 2026-02-25T08:39:45.603 INFO:teuthology.orchestra.run.trial137.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:39:45.603 INFO:teuthology.orchestra.run.trial137.stdout:Total 448 kB/s | 445 MB 16:55 2026-02-25T08:39:45.603 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction check 2026-02-25T08:39:45.608 INFO:teuthology.orchestra.run.trial137.stdout:Transaction check succeeded. 2026-02-25T08:39:45.609 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction test 2026-02-25T08:39:46.982 INFO:teuthology.orchestra.run.trial137.stdout:Transaction test succeeded. 2026-02-25T08:39:46.983 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction 2026-02-25T08:39:50.580 INFO:teuthology.orchestra.run.trial137.stdout: Preparing : 1/1 2026-02-25T08:40:12.852 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-debugsource-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/2 2026-02-25T08:40:12.870 INFO:teuthology.orchestra.run.trial137.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.e 2/2 2026-02-25T08:40:14.756 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.e 2/2 2026-02-25T08:40:14.756 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-debugsource-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/2 2026-02-25T08:40:14.819 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.e 2/2 2026-02-25T08:40:14.819 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:14.819 INFO:teuthology.orchestra.run.trial137.stdout:Installed: 2026-02-25T08:40:14.819 INFO:teuthology.orchestra.run.trial137.stdout: ceph-debugsource-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:40:14.819 INFO:teuthology.orchestra.run.trial137.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:40:14.819 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:14.819 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:14.894 DEBUG:teuthology.parallel:result is None 2026-02-25T08:40:14.894 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:40:15.016 DEBUG:teuthology.orchestra.run.trial137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T08:40:15.037 INFO:teuthology.orchestra.run.trial137.stdout:20.3.0-5338.g6aeb6975.el9 2026-02-25T08:40:15.038 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5338.g6aeb6975.el9 2026-02-25T08:40:15.038 INFO:teuthology.task.install:The correct ceph version 20.3.0-5338.g6aeb6975 is installed. 2026-02-25T08:40:15.040 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-25T08:40:15.040 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:40:15.040 DEBUG:teuthology.orchestra.run.trial137:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-25T08:40:15.105 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-25T08:40:15.105 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:40:15.105 DEBUG:teuthology.orchestra.run.trial137:> sudo dd of=/usr/bin/daemon-helper 2026-02-25T08:40:15.169 DEBUG:teuthology.orchestra.run.trial137:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-25T08:40:15.234 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-25T08:40:15.234 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:40:15.234 DEBUG:teuthology.orchestra.run.trial137:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-25T08:40:15.298 DEBUG:teuthology.orchestra.run.trial137:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-25T08:40:15.362 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-25T08:40:15.362 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:40:15.362 DEBUG:teuthology.orchestra.run.trial137:> sudo dd of=/usr/bin/stdin-killer 2026-02-25T08:40:15.425 DEBUG:teuthology.orchestra.run.trial137:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-25T08:40:15.490 INFO:teuthology.run_tasks:Running task pexec... 2026-02-25T08:40:15.508 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-25T08:40:15.509 DEBUG:teuthology.orchestra.run.trial137:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-25T08:40:15.532 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial137.front.sepia.ceph.com 2026-02-25T08:40:15.532 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-25T08:40:15.582 DEBUG:teuthology.parallel:result is None 2026-02-25T08:40:15.582 INFO:teuthology.run_tasks:Running task install... 2026-02-25T08:40:15.596 DEBUG:teuthology.task.install:project ceph 2026-02-25T08:40:15.596 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}} 2026-02-25T08:40:15.597 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'} 2026-02-25T08:40:15.597 INFO:teuthology.task.install:Using flavor: debug 2026-02-25T08:40:15.606 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-25T08:40:15.606 INFO:teuthology.task.install:extra packages: [] 2026-02-25T08:40:15.606 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819', 'tag': None, 'wait_for_package': False} 2026-02-25T08:40:15.607 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:40:15.766 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/ 2026-02-25T08:40:15.766 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5338.g6aeb6975 2026-02-25T08:40:15.898 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-logstoremanager/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T08:40:15.898 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:40:15.898 DEBUG:teuthology.orchestra.run.trial137:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T08:40:15.923 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-25T08:40:15.923 DEBUG:teuthology.orchestra.run.trial137:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/6aeb6975bc9a4790a399c9cc70dba9635d2a7819/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T08:40:15.989 DEBUG:teuthology.orchestra.run.trial137:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T08:40:16.053 DEBUG:teuthology.orchestra.run.trial137:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T08:40:16.109 INFO:teuthology.orchestra.run.trial137.stdout:check_obsoletes = 1 2026-02-25T08:40:16.119 DEBUG:teuthology.orchestra.run.trial137:> sudo yum clean all 2026-02-25T08:40:16.278 INFO:teuthology.orchestra.run.trial137.stdout:56 files removed 2026-02-25T08:40:16.292 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-radosgw 2026-02-25T08:40:16.598 INFO:teuthology.orchestra.run.trial137.stdout:ceph packages for x86_64 625 kB/s | 94 kB 00:00 2026-02-25T08:40:16.750 INFO:teuthology.orchestra.run.trial137.stdout:ceph noarch packages 200 kB/s | 24 kB 00:00 2026-02-25T08:40:16.884 INFO:teuthology.orchestra.run.trial137.stdout:ceph source packages 19 kB/s | 2.1 kB 00:00 2026-02-25T08:40:17.619 INFO:teuthology.orchestra.run.trial137.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.9 MB 00:00 2026-02-25T08:40:20.848 INFO:teuthology.orchestra.run.trial137.stdout:CentOS Stream 9 - AppStream 10 MB/s | 27 MB 00:02 2026-02-25T08:40:24.957 INFO:teuthology.orchestra.run.trial137.stdout:CentOS Stream 9 - CRB 4.7 MB/s | 8.0 MB 00:01 2026-02-25T08:40:29.524 INFO:teuthology.orchestra.run.trial137.stdout:CentOS Stream 9 - Extras packages 5.3 kB/s | 20 kB 00:03 2026-02-25T08:40:30.568 INFO:teuthology.orchestra.run.trial137.stdout:Extra Packages for Enterprise Linux 21 MB/s | 20 MB 00:00 2026-02-25T08:40:33.778 INFO:teuthology.orchestra.run.trial137.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-25T08:40:34.971 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-radosgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:34.989 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:34.989 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:34.989 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:35.012 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-test 2026-02-25T08:40:35.331 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:35.396 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-test-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:35.414 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:35.414 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:35.414 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:35.437 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph 2026-02-25T08:40:35.762 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:35.827 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:35.845 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:35.846 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:35.846 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:35.867 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-base 2026-02-25T08:40:36.186 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:36.251 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-base-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:36.269 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:36.269 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:36.269 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:36.292 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install cephadm 2026-02-25T08:40:36.613 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:36.678 INFO:teuthology.orchestra.run.trial137.stdout:Package cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:36.696 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:36.696 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:36.696 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:36.718 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T08:40:37.037 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:04 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:37.102 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-immutable-object-cache-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:37.119 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:37.120 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:37.120 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:37.142 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-mgr 2026-02-25T08:40:37.460 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:04 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:37.525 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-mgr-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:37.543 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:37.544 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:37.544 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:37.566 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T08:40:37.888 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:04 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:37.954 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-mgr-dashboard-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:37.971 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:37.972 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:37.972 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:37.994 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T08:40:38.315 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:38.381 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:38.399 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:38.399 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:38.399 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:38.421 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-mgr-rook 2026-02-25T08:40:38.741 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:38.807 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-mgr-rook-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:38.824 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:38.825 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:38.825 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:38.847 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T08:40:39.164 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:06 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:39.229 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-mgr-cephadm-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:39.247 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:39.248 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:39.248 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:39.270 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-osd 2026-02-25T08:40:39.588 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:06 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:39.653 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-osd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:39.671 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:39.672 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:39.672 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:39.693 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-osd-classic 2026-02-25T08:40:40.008 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:07 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:40.074 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-osd-classic-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:40.092 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:40.092 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:40.092 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:40.114 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-fuse 2026-02-25T08:40:40.436 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:07 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:40.503 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:40.520 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:40.521 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:40.521 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:40.543 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install ceph-volume 2026-02-25T08:40:40.859 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:07 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:40.925 INFO:teuthology.orchestra.run.trial137.stdout:Package ceph-volume-2:20.3.0-5338.g6aeb6975.el9.noarch is already installed. 2026-02-25T08:40:40.943 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:40.943 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:40.943 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:40.965 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install librados-devel 2026-02-25T08:40:41.285 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:41.350 INFO:teuthology.orchestra.run.trial137.stdout:Package librados-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:41.368 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:41.369 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:41.369 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:41.391 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install libcephfs2 2026-02-25T08:40:41.711 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:41.776 INFO:teuthology.orchestra.run.trial137.stdout:Package libcephfs2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:41.793 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:41.794 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:41.794 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:41.816 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install libcephfs-devel 2026-02-25T08:40:42.135 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:42.200 INFO:teuthology.orchestra.run.trial137.stdout:Package libcephfs-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:42.218 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:42.218 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:42.218 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:42.240 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install librados2 2026-02-25T08:40:42.561 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:42.627 INFO:teuthology.orchestra.run.trial137.stdout:Package librados2-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:42.645 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:42.646 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:42.646 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:42.668 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install librbd1 2026-02-25T08:40:42.986 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:43.052 INFO:teuthology.orchestra.run.trial137.stdout:Package librbd1-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:43.071 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:43.071 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:43.072 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:43.093 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install python3-rados 2026-02-25T08:40:43.416 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:10 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:43.481 INFO:teuthology.orchestra.run.trial137.stdout:Package python3-rados-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:43.499 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:43.500 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:43.500 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:43.522 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install python3-rgw 2026-02-25T08:40:43.845 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:10 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:43.910 INFO:teuthology.orchestra.run.trial137.stdout:Package python3-rgw-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:43.928 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:43.929 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:43.950 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install python3-cephfs 2026-02-25T08:40:44.275 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:44.341 INFO:teuthology.orchestra.run.trial137.stdout:Package python3-cephfs-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:44.358 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:44.359 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:44.359 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:44.382 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install python3-rbd 2026-02-25T08:40:44.699 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:44.765 INFO:teuthology.orchestra.run.trial137.stdout:Package python3-rbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:44.783 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:44.784 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:44.784 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:44.805 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install rbd-fuse 2026-02-25T08:40:45.125 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:12 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:45.190 INFO:teuthology.orchestra.run.trial137.stdout:Package rbd-fuse-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:45.207 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:45.208 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:45.208 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:45.230 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install rbd-mirror 2026-02-25T08:40:45.544 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:12 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:45.609 INFO:teuthology.orchestra.run.trial137.stdout:Package rbd-mirror-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:45.627 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:45.627 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:45.627 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:45.648 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install rbd-nbd 2026-02-25T08:40:45.968 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:12 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:40:46.033 INFO:teuthology.orchestra.run.trial137.stdout:Package rbd-nbd-2:20.3.0-5338.g6aeb6975.el9.x86_64 is already installed. 2026-02-25T08:40:46.050 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:40:46.051 INFO:teuthology.orchestra.run.trial137.stdout:Nothing to do. 2026-02-25T08:40:46.051 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:40:46.076 DEBUG:teuthology.parallel:result is None 2026-02-25T08:40:46.077 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=6aeb6975bc9a4790a399c9cc70dba9635d2a7819 2026-02-25T08:40:46.235 DEBUG:teuthology.orchestra.run.trial137:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T08:40:46.256 INFO:teuthology.orchestra.run.trial137.stdout:20.3.0-5338.g6aeb6975.el9 2026-02-25T08:40:46.257 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5338.g6aeb6975.el9 2026-02-25T08:40:46.257 INFO:teuthology.task.install:The correct ceph version 20.3.0-5338.g6aeb6975 is installed. 2026-02-25T08:40:46.259 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-25T08:40:46.259 INFO:teuthology.run_tasks:Running task ceph... 2026-02-25T08:40:46.347 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-25T08:40:46.347 DEBUG:teuthology.orchestra.run.trial137:> sudo chmod 777 /var/log/ceph 2026-02-25T08:40:46.372 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-25T08:40:46.372 DEBUG:teuthology.orchestra.run.trial137:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-25T08:40:46.435 INFO:tasks.ceph:Creating extra log directories... 2026-02-25T08:40:46.436 DEBUG:teuthology.orchestra.run.trial137:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-25T08:40:46.509 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-25T08:40:46.509 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-25T08:40:46.509 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69108', 'branch': 'wip-logstoremanager', 'description': 'crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} perf_ignorelist settings/optimized workloads/radosbench_4M_write}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '69108', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-logstoremanager'}, 'ceph': {'conf': {'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'c32469671c6b0222f786dd36c34eb702f780a11d'}}, 'owner': 'scheduled_matan@soko04.front.sepia.ceph.com', 'priority': 70, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 869, 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': 'c32469671c6b0222f786dd36c34eb702f780a11d', 'targets': {'trial137.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBH3WPWOa+mBftIaV7aiPNtRWdWFz/J2quv1MdtWj5IQ7tlOtV5xba1lj00Zq81vlQIk8DXonJt2cXQn0rKdnV88='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'wait-for-scrub': False, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': '6aeb6975bc9a4790a399c9cc70dba9635d2a7819', 'cluster': 'ceph'}}, {'ssh_keys': None}, {'cbt': {'benchmarks': {'radosbench': {'concurrent_ops': 4, 'concurrent_procs': 2, 'op_size': [4194304], 'pool_monitoring_list': ['collectl'], 'pool_profile': 'replicated', 'run_monitoring_list': ['collectl'], 'time': 60, 'write_only': True}}, 'cluster': {'iterations': 1, 'osds_per_node': 3, 'pool_profiles': {'replicated': {'pg_size': 256, 'pgp_size': 256, 'replication': 'replicated'}}, 'user': 'ubuntu'}, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-25_07:55:35', 'tube': 'trial', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699'} 2026-02-25T08:40:46.510 DEBUG:teuthology.orchestra.run.trial137:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-25T08:40:46.556 DEBUG:teuthology.orchestra.run.trial137:> sudo install -d -m0777 -- /var/run/ceph 2026-02-25T08:40:46.620 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:40:46.620 DEBUG:teuthology.orchestra.run.trial137:> dd if=/scratch_devs of=/dev/stdout 2026-02-25T08:40:46.676 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-25T08:40:46.676 DEBUG:teuthology.orchestra.run.trial137:> stat /dev/vg_nvme/lv_1 2026-02-25T08:40:46.732 INFO:teuthology.orchestra.run.trial137.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-25T08:40:46.733 INFO:teuthology.orchestra.run.trial137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:46.733 INFO:teuthology.orchestra.run.trial137.stdout:Device: 6h/6d Inode: 1319 Links: 1 2026-02-25T08:40:46.733 INFO:teuthology.orchestra.run.trial137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:46.733 INFO:teuthology.orchestra.run.trial137.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:46.733 INFO:teuthology.orchestra.run.trial137.stdout:Access: 2026-02-25 08:22:47.210418222 +0000 2026-02-25T08:40:46.733 INFO:teuthology.orchestra.run.trial137.stdout:Modify: 2026-02-25 08:09:06.635825833 +0000 2026-02-25T08:40:46.733 INFO:teuthology.orchestra.run.trial137.stdout:Change: 2026-02-25 08:09:06.635825833 +0000 2026-02-25T08:40:46.733 INFO:teuthology.orchestra.run.trial137.stdout: Birth: 2026-02-25 08:09:06.635825833 +0000 2026-02-25T08:40:46.734 DEBUG:teuthology.orchestra.run.trial137:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-25T08:40:46.796 INFO:teuthology.orchestra.run.trial137.stderr:1+0 records in 2026-02-25T08:40:46.796 INFO:teuthology.orchestra.run.trial137.stderr:1+0 records out 2026-02-25T08:40:46.796 INFO:teuthology.orchestra.run.trial137.stderr:512 bytes copied, 0.000233483 s, 2.2 MB/s 2026-02-25T08:40:46.797 DEBUG:teuthology.orchestra.run.trial137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-25T08:40:46.854 DEBUG:teuthology.orchestra.run.trial137:> stat /dev/vg_nvme/lv_2 2026-02-25T08:40:46.910 INFO:teuthology.orchestra.run.trial137.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-25T08:40:46.910 INFO:teuthology.orchestra.run.trial137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:46.910 INFO:teuthology.orchestra.run.trial137.stdout:Device: 6h/6d Inode: 1308 Links: 1 2026-02-25T08:40:46.911 INFO:teuthology.orchestra.run.trial137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:46.911 INFO:teuthology.orchestra.run.trial137.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:46.911 INFO:teuthology.orchestra.run.trial137.stdout:Access: 2026-02-25 08:22:47.211418207 +0000 2026-02-25T08:40:46.911 INFO:teuthology.orchestra.run.trial137.stdout:Modify: 2026-02-25 08:09:06.635825833 +0000 2026-02-25T08:40:46.911 INFO:teuthology.orchestra.run.trial137.stdout:Change: 2026-02-25 08:09:06.635825833 +0000 2026-02-25T08:40:46.911 INFO:teuthology.orchestra.run.trial137.stdout: Birth: 2026-02-25 08:09:06.635825833 +0000 2026-02-25T08:40:46.911 DEBUG:teuthology.orchestra.run.trial137:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-25T08:40:46.975 INFO:teuthology.orchestra.run.trial137.stderr:1+0 records in 2026-02-25T08:40:46.975 INFO:teuthology.orchestra.run.trial137.stderr:1+0 records out 2026-02-25T08:40:46.975 INFO:teuthology.orchestra.run.trial137.stderr:512 bytes copied, 0.000221411 s, 2.3 MB/s 2026-02-25T08:40:46.976 DEBUG:teuthology.orchestra.run.trial137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-25T08:40:47.034 DEBUG:teuthology.orchestra.run.trial137:> stat /dev/vg_nvme/lv_3 2026-02-25T08:40:47.090 INFO:teuthology.orchestra.run.trial137.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-25T08:40:47.090 INFO:teuthology.orchestra.run.trial137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:47.090 INFO:teuthology.orchestra.run.trial137.stdout:Device: 6h/6d Inode: 1304 Links: 1 2026-02-25T08:40:47.090 INFO:teuthology.orchestra.run.trial137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:47.090 INFO:teuthology.orchestra.run.trial137.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:47.090 INFO:teuthology.orchestra.run.trial137.stdout:Access: 2026-02-25 08:22:47.211418207 +0000 2026-02-25T08:40:47.090 INFO:teuthology.orchestra.run.trial137.stdout:Modify: 2026-02-25 08:09:06.635825833 +0000 2026-02-25T08:40:47.091 INFO:teuthology.orchestra.run.trial137.stdout:Change: 2026-02-25 08:09:06.635825833 +0000 2026-02-25T08:40:47.091 INFO:teuthology.orchestra.run.trial137.stdout: Birth: 2026-02-25 08:09:06.635825833 +0000 2026-02-25T08:40:47.091 DEBUG:teuthology.orchestra.run.trial137:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-25T08:40:47.152 INFO:teuthology.orchestra.run.trial137.stderr:1+0 records in 2026-02-25T08:40:47.152 INFO:teuthology.orchestra.run.trial137.stderr:1+0 records out 2026-02-25T08:40:47.153 INFO:teuthology.orchestra.run.trial137.stderr:512 bytes copied, 0.000157151 s, 3.3 MB/s 2026-02-25T08:40:47.154 DEBUG:teuthology.orchestra.run.trial137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-25T08:40:47.211 DEBUG:teuthology.orchestra.run.trial137:> stat /dev/vg_nvme/lv_4 2026-02-25T08:40:47.268 INFO:teuthology.orchestra.run.trial137.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-25T08:40:47.268 INFO:teuthology.orchestra.run.trial137.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T08:40:47.268 INFO:teuthology.orchestra.run.trial137.stdout:Device: 6h/6d Inode: 1300 Links: 1 2026-02-25T08:40:47.268 INFO:teuthology.orchestra.run.trial137.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T08:40:47.268 INFO:teuthology.orchestra.run.trial137.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T08:40:47.269 INFO:teuthology.orchestra.run.trial137.stdout:Access: 2026-02-25 08:22:47.211418207 +0000 2026-02-25T08:40:47.269 INFO:teuthology.orchestra.run.trial137.stdout:Modify: 2026-02-25 08:09:06.634825872 +0000 2026-02-25T08:40:47.269 INFO:teuthology.orchestra.run.trial137.stdout:Change: 2026-02-25 08:09:06.634825872 +0000 2026-02-25T08:40:47.269 INFO:teuthology.orchestra.run.trial137.stdout: Birth: 2026-02-25 08:09:06.634825872 +0000 2026-02-25T08:40:47.269 DEBUG:teuthology.orchestra.run.trial137:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-25T08:40:47.330 INFO:teuthology.orchestra.run.trial137.stderr:1+0 records in 2026-02-25T08:40:47.330 INFO:teuthology.orchestra.run.trial137.stderr:1+0 records out 2026-02-25T08:40:47.330 INFO:teuthology.orchestra.run.trial137.stderr:512 bytes copied, 0.000143596 s, 3.6 MB/s 2026-02-25T08:40:47.331 DEBUG:teuthology.orchestra.run.trial137:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-25T08:40:47.389 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-25T08:40:47.389 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial137.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-25T08:40:47.389 INFO:tasks.ceph:Generating config... 2026-02-25T08:40:47.391 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-25T08:40:47.392 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-25T08:40:47.392 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-25T08:40:47.392 INFO:tasks.ceph:[osd] debug filestore = 0/0 2026-02-25T08:40:47.392 INFO:tasks.ceph:[osd] debug journal = 0/0 2026-02-25T08:40:47.392 INFO:tasks.ceph:[osd] debug ms = 0/0 2026-02-25T08:40:47.392 INFO:tasks.ceph:[osd] debug osd = 0/0 2026-02-25T08:40:47.392 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-25T08:40:47.392 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-25T08:40:47.393 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-25T08:40:47.393 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-25T08:40:47.393 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-25T08:40:47.393 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-25T08:40:47.393 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-25T08:40:47.393 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-25T08:40:47.393 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-25T08:40:47.393 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-25T08:40:47.393 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-25T08:40:47.393 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-25T08:40:47.393 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-25T08:40:47.393 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-25T08:40:47.393 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-25T08:40:47.394 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-25T08:40:47.394 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-25T08:40:47.394 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-25T08:40:47.394 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-25T08:40:47.394 INFO:tasks.ceph:[global] auth client required = none 2026-02-25T08:40:47.394 INFO:tasks.ceph:[global] auth cluster required = none 2026-02-25T08:40:47.394 INFO:tasks.ceph:[global] auth service required = none 2026-02-25T08:40:47.394 INFO:tasks.ceph:[global] auth supported = none 2026-02-25T08:40:47.394 INFO:tasks.ceph:[global] debug asok = 0/0 2026-02-25T08:40:47.394 INFO:tasks.ceph:[global] debug auth = 0/0 2026-02-25T08:40:47.394 INFO:tasks.ceph:[global] debug bdev = 0/0 2026-02-25T08:40:47.394 INFO:tasks.ceph:[global] debug bluefs = 0/0 2026-02-25T08:40:47.395 INFO:tasks.ceph:[global] debug bluestore = 0/0 2026-02-25T08:40:47.395 INFO:tasks.ceph:[global] debug buffer = 0/0 2026-02-25T08:40:47.395 INFO:tasks.ceph:[global] debug civetweb = 0/0 2026-02-25T08:40:47.395 INFO:tasks.ceph:[global] debug client = 0/0 2026-02-25T08:40:47.395 INFO:tasks.ceph:[global] debug compressor = 0/0 2026-02-25T08:40:47.395 INFO:tasks.ceph:[global] debug context = 0/0 2026-02-25T08:40:47.395 INFO:tasks.ceph:[global] debug crush = 0/0 2026-02-25T08:40:47.395 INFO:tasks.ceph:[global] debug crypto = 0/0 2026-02-25T08:40:47.395 INFO:tasks.ceph:[global] debug dpdk = 0/0 2026-02-25T08:40:47.395 INFO:tasks.ceph:[global] debug eventtrace = 0/0 2026-02-25T08:40:47.395 INFO:tasks.ceph:[global] debug filer = 0/0 2026-02-25T08:40:47.395 INFO:tasks.ceph:[global] debug filestore = 0/0 2026-02-25T08:40:47.395 INFO:tasks.ceph:[global] debug finisher = 0/0 2026-02-25T08:40:47.396 INFO:tasks.ceph:[global] debug fuse = 0/0 2026-02-25T08:40:47.396 INFO:tasks.ceph:[global] debug heartbeatmap = 0/0 2026-02-25T08:40:47.396 INFO:tasks.ceph:[global] debug javaclient = 0/0 2026-02-25T08:40:47.396 INFO:tasks.ceph:[global] debug journal = 0/0 2026-02-25T08:40:47.396 INFO:tasks.ceph:[global] debug journaler = 0/0 2026-02-25T08:40:47.396 INFO:tasks.ceph:[global] debug kstore = 0/0 2026-02-25T08:40:47.396 INFO:tasks.ceph:[global] debug leveldb = 0/0 2026-02-25T08:40:47.396 INFO:tasks.ceph:[global] debug lockdep = 0/0 2026-02-25T08:40:47.396 INFO:tasks.ceph:[global] debug mds = 0/0 2026-02-25T08:40:47.396 INFO:tasks.ceph:[global] debug mds balancer = 0/0 2026-02-25T08:40:47.396 INFO:tasks.ceph:[global] debug mds locker = 0/0 2026-02-25T08:40:47.396 INFO:tasks.ceph:[global] debug mds log = 0/0 2026-02-25T08:40:47.396 INFO:tasks.ceph:[global] debug mds log expire = 0/0 2026-02-25T08:40:47.397 INFO:tasks.ceph:[global] debug mds migrator = 0/0 2026-02-25T08:40:47.397 INFO:tasks.ceph:[global] debug memdb = 0/0 2026-02-25T08:40:47.397 INFO:tasks.ceph:[global] debug mgr = 0/0 2026-02-25T08:40:47.397 INFO:tasks.ceph:[global] debug mgrc = 0/0 2026-02-25T08:40:47.397 INFO:tasks.ceph:[global] debug mon = 0/0 2026-02-25T08:40:47.397 INFO:tasks.ceph:[global] debug monc = 0/0 2026-02-25T08:40:47.397 INFO:tasks.ceph:[global] debug ms = 0/0 2026-02-25T08:40:47.397 INFO:tasks.ceph:[global] debug objclass = 0/0 2026-02-25T08:40:47.397 INFO:tasks.ceph:[global] debug objectcacher = 0/0 2026-02-25T08:40:47.397 INFO:tasks.ceph:[global] debug objecter = 0/0 2026-02-25T08:40:47.397 INFO:tasks.ceph:[global] debug optracker = 0/0 2026-02-25T08:40:47.397 INFO:tasks.ceph:[global] debug osd = 0/0 2026-02-25T08:40:47.397 INFO:tasks.ceph:[global] debug paxos = 0/0 2026-02-25T08:40:47.398 INFO:tasks.ceph:[global] debug perfcounter = 0/0 2026-02-25T08:40:47.398 INFO:tasks.ceph:[global] debug rados = 0/0 2026-02-25T08:40:47.398 INFO:tasks.ceph:[global] debug rbd = 0/0 2026-02-25T08:40:47.398 INFO:tasks.ceph:[global] debug rbd mirror = 0/0 2026-02-25T08:40:47.398 INFO:tasks.ceph:[global] debug rbd replay = 0/0 2026-02-25T08:40:47.398 INFO:tasks.ceph:[global] debug refs = 0/0 2026-02-25T08:40:47.398 INFO:tasks.ceph:[global] debug rgw = 0/0 2026-02-25T08:40:47.398 INFO:tasks.ceph:[global] debug rgw lifecycle = 0/0 2026-02-25T08:40:47.398 INFO:tasks.ceph:[global] debug rgw notification = 0/0 2026-02-25T08:40:47.398 INFO:tasks.ceph:[global] debug rgw sync = 0/0 2026-02-25T08:40:47.398 INFO:tasks.ceph:[global] debug rocksdb = 0/0 2026-02-25T08:40:47.398 INFO:tasks.ceph:[global] debug striper = 0/0 2026-02-25T08:40:47.398 INFO:tasks.ceph:[global] debug throttle = 0/0 2026-02-25T08:40:47.399 INFO:tasks.ceph:[global] debug timer = 0/0 2026-02-25T08:40:47.399 INFO:tasks.ceph:[global] debug tp = 0/0 2026-02-25T08:40:47.399 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-25T08:40:47.399 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-25T08:40:47.399 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-25T08:40:47.399 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-25T08:40:47.399 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-25T08:40:47.399 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-25T08:40:47.399 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-25T08:40:47.399 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-25T08:40:47.399 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-25T08:40:47.399 INFO:tasks.ceph:[mon] debug mon = 0/0 2026-02-25T08:40:47.399 INFO:tasks.ceph:[mon] debug ms = 0/0 2026-02-25T08:40:47.400 INFO:tasks.ceph:[mon] debug paxos = 0/0 2026-02-25T08:40:47.400 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-25T08:40:47.400 INFO:tasks.ceph:Setting up mon.a... 2026-02-25T08:40:47.400 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-25T08:40:47.469 INFO:teuthology.orchestra.run.trial137.stdout:creating /etc/ceph/ceph.keyring 2026-02-25T08:40:47.472 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-25T08:40:47.556 DEBUG:teuthology.orchestra.run.trial137:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-25T08:40:47.619 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.137')] 2026-02-25T08:40:47.620 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'none', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.137', 'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': '0/0', 'debug mon': '0/0', 'debug paxos': '0/0', 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-25T08:40:47.622 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:40:47.622 DEBUG:teuthology.orchestra.run.trial137:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-25T08:40:47.676 DEBUG:teuthology.orchestra.run.trial137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.137 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:47.756 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:40:47.755+0000 7fd0a4295f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:47.756 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:40:47.755+0000 7fd0a4295f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:47.756 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:40:47.755+0000 7fd0a4295f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:40:47.758 INFO:teuthology.orchestra.run.trial137.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:47.758 INFO:teuthology.orchestra.run.trial137.stdout:monmaptool: generated fsid 72d844e8-55f4-45a1-8a75-b04b9565f21f 2026-02-25T08:40:47.758 INFO:teuthology.orchestra.run.trial137.stdout:setting min_mon_release = tentacle 2026-02-25T08:40:47.758 INFO:teuthology.orchestra.run.trial137.stdout:epoch 0 2026-02-25T08:40:47.758 INFO:teuthology.orchestra.run.trial137.stdout:fsid 72d844e8-55f4-45a1-8a75-b04b9565f21f 2026-02-25T08:40:47.758 INFO:teuthology.orchestra.run.trial137.stdout:last_changed 2026-02-25T08:40:47.755962+0000 2026-02-25T08:40:47.758 INFO:teuthology.orchestra.run.trial137.stdout:created 2026-02-25T08:40:47.755962+0000 2026-02-25T08:40:47.758 INFO:teuthology.orchestra.run.trial137.stdout:min_mon_release 20 (tentacle) 2026-02-25T08:40:47.758 INFO:teuthology.orchestra.run.trial137.stdout:election_strategy: 1 2026-02-25T08:40:47.759 INFO:teuthology.orchestra.run.trial137.stdout:0: [v2:10.20.193.137:3300/0,v1:10.20.193.137:6789/0] mon.a 2026-02-25T08:40:47.759 INFO:teuthology.orchestra.run.trial137.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-25T08:40:47.759 DEBUG:teuthology.orchestra.run.trial137:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-25T08:40:47.814 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 72d844e8-55f4-45a1-8a75-b04b9565f21f... 2026-02-25T08:40:47.816 DEBUG:teuthology.orchestra.run.trial137:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-25T08:40:47.895 INFO:teuthology.orchestra.run.trial137.stdout:[global] 2026-02-25T08:40:47.896 INFO:teuthology.orchestra.run.trial137.stdout: chdir = "" 2026-02-25T08:40:47.896 INFO:teuthology.orchestra.run.trial137.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-25T08:40:47.896 INFO:teuthology.orchestra.run.trial137.stdout: auth supported = none 2026-02-25T08:40:47.896 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.896 INFO:teuthology.orchestra.run.trial137.stdout: filestore xattr use omap = true 2026-02-25T08:40:47.896 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.896 INFO:teuthology.orchestra.run.trial137.stdout: mon clock drift allowed = 1.000 2026-02-25T08:40:47.896 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.897 INFO:teuthology.orchestra.run.trial137.stdout: osd crush chooseleaf type = 0 2026-02-25T08:40:47.897 INFO:teuthology.orchestra.run.trial137.stdout: auth debug = true 2026-02-25T08:40:47.897 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.897 INFO:teuthology.orchestra.run.trial137.stdout: ms die on old message = true 2026-02-25T08:40:47.897 INFO:teuthology.orchestra.run.trial137.stdout: ms die on bug = true 2026-02-25T08:40:47.897 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.897 INFO:teuthology.orchestra.run.trial137.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-25T08:40:47.897 INFO:teuthology.orchestra.run.trial137.stdout: mon pg warn max object skew = 0 2026-02-25T08:40:47.897 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.897 INFO:teuthology.orchestra.run.trial137.stdout: # disable pg_autoscaler by default for new pools 2026-02-25T08:40:47.897 INFO:teuthology.orchestra.run.trial137.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-25T08:40:47.897 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.898 INFO:teuthology.orchestra.run.trial137.stdout: osd pool default size = 2 2026-02-25T08:40:47.898 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.898 INFO:teuthology.orchestra.run.trial137.stdout: mon osd allow primary affinity = true 2026-02-25T08:40:47.898 INFO:teuthology.orchestra.run.trial137.stdout: mon osd allow pg remap = true 2026-02-25T08:40:47.898 INFO:teuthology.orchestra.run.trial137.stdout: mon warn on legacy crush tunables = false 2026-02-25T08:40:47.898 INFO:teuthology.orchestra.run.trial137.stdout: mon warn on crush straw calc version zero = false 2026-02-25T08:40:47.898 INFO:teuthology.orchestra.run.trial137.stdout: mon warn on no sortbitwise = false 2026-02-25T08:40:47.898 INFO:teuthology.orchestra.run.trial137.stdout: mon warn on osd down out interval zero = false 2026-02-25T08:40:47.898 INFO:teuthology.orchestra.run.trial137.stdout: mon warn on too few osds = false 2026-02-25T08:40:47.898 INFO:teuthology.orchestra.run.trial137.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-25T08:40:47.898 INFO:teuthology.orchestra.run.trial137.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-25T08:40:47.898 INFO:teuthology.orchestra.run.trial137.stdout: mon_allow_pool_size_one = true 2026-02-25T08:40:47.898 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.899 INFO:teuthology.orchestra.run.trial137.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-25T08:40:47.899 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.899 INFO:teuthology.orchestra.run.trial137.stdout: osd default data pool replay window = 5 2026-02-25T08:40:47.899 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.899 INFO:teuthology.orchestra.run.trial137.stdout: mon allow pool delete = true 2026-02-25T08:40:47.899 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.899 INFO:teuthology.orchestra.run.trial137.stdout: mon cluster log file level = debug 2026-02-25T08:40:47.899 INFO:teuthology.orchestra.run.trial137.stdout: debug asserts on shutdown = true 2026-02-25T08:40:47.899 INFO:teuthology.orchestra.run.trial137.stdout: mon health detail to clog = false 2026-02-25T08:40:47.899 INFO:teuthology.orchestra.run.trial137.stdout: mon host = 10.20.193.137 2026-02-25T08:40:47.899 INFO:teuthology.orchestra.run.trial137.stdout: auth client required = none 2026-02-25T08:40:47.899 INFO:teuthology.orchestra.run.trial137.stdout: auth cluster required = none 2026-02-25T08:40:47.900 INFO:teuthology.orchestra.run.trial137.stdout: auth service required = none 2026-02-25T08:40:47.900 INFO:teuthology.orchestra.run.trial137.stdout: debug asok = 0/0 2026-02-25T08:40:47.900 INFO:teuthology.orchestra.run.trial137.stdout: debug auth = 0/0 2026-02-25T08:40:47.900 INFO:teuthology.orchestra.run.trial137.stdout: debug bdev = 0/0 2026-02-25T08:40:47.900 INFO:teuthology.orchestra.run.trial137.stdout: debug bluefs = 0/0 2026-02-25T08:40:47.900 INFO:teuthology.orchestra.run.trial137.stdout: debug bluestore = 0/0 2026-02-25T08:40:47.900 INFO:teuthology.orchestra.run.trial137.stdout: debug buffer = 0/0 2026-02-25T08:40:47.900 INFO:teuthology.orchestra.run.trial137.stdout: debug civetweb = 0/0 2026-02-25T08:40:47.900 INFO:teuthology.orchestra.run.trial137.stdout: debug client = 0/0 2026-02-25T08:40:47.900 INFO:teuthology.orchestra.run.trial137.stdout: debug compressor = 0/0 2026-02-25T08:40:47.900 INFO:teuthology.orchestra.run.trial137.stdout: debug context = 0/0 2026-02-25T08:40:47.900 INFO:teuthology.orchestra.run.trial137.stdout: debug crush = 0/0 2026-02-25T08:40:47.900 INFO:teuthology.orchestra.run.trial137.stdout: debug crypto = 0/0 2026-02-25T08:40:47.901 INFO:teuthology.orchestra.run.trial137.stdout: debug dpdk = 0/0 2026-02-25T08:40:47.901 INFO:teuthology.orchestra.run.trial137.stdout: debug eventtrace = 0/0 2026-02-25T08:40:47.901 INFO:teuthology.orchestra.run.trial137.stdout: debug filer = 0/0 2026-02-25T08:40:47.901 INFO:teuthology.orchestra.run.trial137.stdout: debug filestore = 0/0 2026-02-25T08:40:47.901 INFO:teuthology.orchestra.run.trial137.stdout: debug finisher = 0/0 2026-02-25T08:40:47.901 INFO:teuthology.orchestra.run.trial137.stdout: debug fuse = 0/0 2026-02-25T08:40:47.901 INFO:teuthology.orchestra.run.trial137.stdout: debug heartbeatmap = 0/0 2026-02-25T08:40:47.901 INFO:teuthology.orchestra.run.trial137.stdout: debug javaclient = 0/0 2026-02-25T08:40:47.901 INFO:teuthology.orchestra.run.trial137.stdout: debug journal = 0/0 2026-02-25T08:40:47.901 INFO:teuthology.orchestra.run.trial137.stdout: debug journaler = 0/0 2026-02-25T08:40:47.901 INFO:teuthology.orchestra.run.trial137.stdout: debug kstore = 0/0 2026-02-25T08:40:47.901 INFO:teuthology.orchestra.run.trial137.stdout: debug leveldb = 0/0 2026-02-25T08:40:47.901 INFO:teuthology.orchestra.run.trial137.stdout: debug lockdep = 0/0 2026-02-25T08:40:47.901 INFO:teuthology.orchestra.run.trial137.stdout: debug mds = 0/0 2026-02-25T08:40:47.902 INFO:teuthology.orchestra.run.trial137.stdout: debug mds balancer = 0/0 2026-02-25T08:40:47.902 INFO:teuthology.orchestra.run.trial137.stdout: debug mds locker = 0/0 2026-02-25T08:40:47.902 INFO:teuthology.orchestra.run.trial137.stdout: debug mds log = 0/0 2026-02-25T08:40:47.902 INFO:teuthology.orchestra.run.trial137.stdout: debug mds log expire = 0/0 2026-02-25T08:40:47.902 INFO:teuthology.orchestra.run.trial137.stdout: debug mds migrator = 0/0 2026-02-25T08:40:47.902 INFO:teuthology.orchestra.run.trial137.stdout: debug memdb = 0/0 2026-02-25T08:40:47.902 INFO:teuthology.orchestra.run.trial137.stdout: debug mgr = 0/0 2026-02-25T08:40:47.902 INFO:teuthology.orchestra.run.trial137.stdout: debug mgrc = 0/0 2026-02-25T08:40:47.902 INFO:teuthology.orchestra.run.trial137.stdout: debug mon = 0/0 2026-02-25T08:40:47.902 INFO:teuthology.orchestra.run.trial137.stdout: debug monc = 0/0 2026-02-25T08:40:47.902 INFO:teuthology.orchestra.run.trial137.stdout: debug ms = 0/0 2026-02-25T08:40:47.902 INFO:teuthology.orchestra.run.trial137.stdout: debug objclass = 0/0 2026-02-25T08:40:47.902 INFO:teuthology.orchestra.run.trial137.stdout: debug objectcacher = 0/0 2026-02-25T08:40:47.903 INFO:teuthology.orchestra.run.trial137.stdout: debug objecter = 0/0 2026-02-25T08:40:47.903 INFO:teuthology.orchestra.run.trial137.stdout: debug optracker = 0/0 2026-02-25T08:40:47.903 INFO:teuthology.orchestra.run.trial137.stdout: debug osd = 0/0 2026-02-25T08:40:47.903 INFO:teuthology.orchestra.run.trial137.stdout: debug paxos = 0/0 2026-02-25T08:40:47.903 INFO:teuthology.orchestra.run.trial137.stdout: debug perfcounter = 0/0 2026-02-25T08:40:47.903 INFO:teuthology.orchestra.run.trial137.stdout: debug rados = 0/0 2026-02-25T08:40:47.903 INFO:teuthology.orchestra.run.trial137.stdout: debug rbd = 0/0 2026-02-25T08:40:47.903 INFO:teuthology.orchestra.run.trial137.stdout: debug rbd mirror = 0/0 2026-02-25T08:40:47.903 INFO:teuthology.orchestra.run.trial137.stdout: debug rbd replay = 0/0 2026-02-25T08:40:47.903 INFO:teuthology.orchestra.run.trial137.stdout: debug refs = 0/0 2026-02-25T08:40:47.903 INFO:teuthology.orchestra.run.trial137.stdout: debug rgw = 0/0 2026-02-25T08:40:47.903 INFO:teuthology.orchestra.run.trial137.stdout: debug rgw lifecycle = 0/0 2026-02-25T08:40:47.903 INFO:teuthology.orchestra.run.trial137.stdout: debug rgw notification = 0/0 2026-02-25T08:40:47.903 INFO:teuthology.orchestra.run.trial137.stdout: debug rgw sync = 0/0 2026-02-25T08:40:47.904 INFO:teuthology.orchestra.run.trial137.stdout: debug rocksdb = 0/0 2026-02-25T08:40:47.904 INFO:teuthology.orchestra.run.trial137.stdout: debug striper = 0/0 2026-02-25T08:40:47.904 INFO:teuthology.orchestra.run.trial137.stdout: debug throttle = 0/0 2026-02-25T08:40:47.904 INFO:teuthology.orchestra.run.trial137.stdout: debug timer = 0/0 2026-02-25T08:40:47.904 INFO:teuthology.orchestra.run.trial137.stdout: debug tp = 0/0 2026-02-25T08:40:47.904 INFO:teuthology.orchestra.run.trial137.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-25T08:40:47.904 INFO:teuthology.orchestra.run.trial137.stdout: ms client mode = crc 2026-02-25T08:40:47.904 INFO:teuthology.orchestra.run.trial137.stdout: ms cluster mode = crc 2026-02-25T08:40:47.904 INFO:teuthology.orchestra.run.trial137.stdout: ms mon client mode = crc 2026-02-25T08:40:47.904 INFO:teuthology.orchestra.run.trial137.stdout: ms mon cluster mode = crc 2026-02-25T08:40:47.904 INFO:teuthology.orchestra.run.trial137.stdout: ms mon service mode = crc 2026-02-25T08:40:47.904 INFO:teuthology.orchestra.run.trial137.stdout: ms service mode = crc 2026-02-25T08:40:47.904 INFO:teuthology.orchestra.run.trial137.stdout: fsid = 72d844e8-55f4-45a1-8a75-b04b9565f21f 2026-02-25T08:40:47.905 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.905 INFO:teuthology.orchestra.run.trial137.stdout:[osd] 2026-02-25T08:40:47.905 INFO:teuthology.orchestra.run.trial137.stdout: osd journal size = 100 2026-02-25T08:40:47.905 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.905 INFO:teuthology.orchestra.run.trial137.stdout: osd scrub load threshold = 5.0 2026-02-25T08:40:47.905 INFO:teuthology.orchestra.run.trial137.stdout: osd scrub max interval = 600 2026-02-25T08:40:47.905 INFO:teuthology.orchestra.run.trial137.stdout: osd mclock profile = high_recovery_ops 2026-02-25T08:40:47.905 INFO:teuthology.orchestra.run.trial137.stdout: osd mclock skip benchmark = true 2026-02-25T08:40:47.905 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.905 INFO:teuthology.orchestra.run.trial137.stdout: osd recover clone overlap = true 2026-02-25T08:40:47.905 INFO:teuthology.orchestra.run.trial137.stdout: osd recovery max chunk = 1048576 2026-02-25T08:40:47.905 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.905 INFO:teuthology.orchestra.run.trial137.stdout: osd debug shutdown = true 2026-02-25T08:40:47.906 INFO:teuthology.orchestra.run.trial137.stdout: osd debug op order = true 2026-02-25T08:40:47.906 INFO:teuthology.orchestra.run.trial137.stdout: osd debug verify stray on activate = true 2026-02-25T08:40:47.906 INFO:teuthology.orchestra.run.trial137.stdout: osd debug trim objects = true 2026-02-25T08:40:47.906 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.906 INFO:teuthology.orchestra.run.trial137.stdout: osd open classes on start = true 2026-02-25T08:40:47.906 INFO:teuthology.orchestra.run.trial137.stdout: osd debug pg log writeout = true 2026-02-25T08:40:47.906 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.906 INFO:teuthology.orchestra.run.trial137.stdout: osd deep scrub update digest min age = 30 2026-02-25T08:40:47.906 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.906 INFO:teuthology.orchestra.run.trial137.stdout: osd map max advance = 10 2026-02-25T08:40:47.906 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.906 INFO:teuthology.orchestra.run.trial137.stdout: journal zero on create = true 2026-02-25T08:40:47.906 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.906 INFO:teuthology.orchestra.run.trial137.stdout: filestore ondisk finisher threads = 3 2026-02-25T08:40:47.907 INFO:teuthology.orchestra.run.trial137.stdout: filestore apply finisher threads = 3 2026-02-25T08:40:47.907 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.907 INFO:teuthology.orchestra.run.trial137.stdout: bdev debug aio = true 2026-02-25T08:40:47.907 INFO:teuthology.orchestra.run.trial137.stdout: osd debug misdirected ops = true 2026-02-25T08:40:47.907 INFO:teuthology.orchestra.run.trial137.stdout: debug monc = 20 2026-02-25T08:40:47.907 INFO:teuthology.orchestra.run.trial137.stdout: crimson cpu num = 4 2026-02-25T08:40:47.907 INFO:teuthology.orchestra.run.trial137.stdout: crimson osd obc lru size = 10 2026-02-25T08:40:47.907 INFO:teuthology.orchestra.run.trial137.stdout: debug filestore = 0/0 2026-02-25T08:40:47.907 INFO:teuthology.orchestra.run.trial137.stdout: debug journal = 0/0 2026-02-25T08:40:47.907 INFO:teuthology.orchestra.run.trial137.stdout: debug ms = 0/0 2026-02-25T08:40:47.907 INFO:teuthology.orchestra.run.trial137.stdout: debug osd = 0/0 2026-02-25T08:40:47.907 INFO:teuthology.orchestra.run.trial137.stdout: debug seastore = 20 2026-02-25T08:40:47.907 INFO:teuthology.orchestra.run.trial137.stdout: debug seastore backref = 20 2026-02-25T08:40:47.908 INFO:teuthology.orchestra.run.trial137.stdout: debug seastore cache = 20 2026-02-25T08:40:47.908 INFO:teuthology.orchestra.run.trial137.stdout: debug seastore cleaner = 20 2026-02-25T08:40:47.908 INFO:teuthology.orchestra.run.trial137.stdout: debug seastore device = 20 2026-02-25T08:40:47.908 INFO:teuthology.orchestra.run.trial137.stdout: debug seastore epm = 20 2026-02-25T08:40:47.908 INFO:teuthology.orchestra.run.trial137.stdout: debug seastore fixedkv tree = 20 2026-02-25T08:40:47.908 INFO:teuthology.orchestra.run.trial137.stdout: debug seastore journal = 20 2026-02-25T08:40:47.908 INFO:teuthology.orchestra.run.trial137.stdout: debug seastore lba = 20 2026-02-25T08:40:47.908 INFO:teuthology.orchestra.run.trial137.stdout: debug seastore odata = 20 2026-02-25T08:40:47.908 INFO:teuthology.orchestra.run.trial137.stdout: debug seastore omap = 20 2026-02-25T08:40:47.908 INFO:teuthology.orchestra.run.trial137.stdout: debug seastore onode = 20 2026-02-25T08:40:47.908 INFO:teuthology.orchestra.run.trial137.stdout: debug seastore t = 20 2026-02-25T08:40:47.908 INFO:teuthology.orchestra.run.trial137.stdout: debug seastore tm = 20 2026-02-25T08:40:47.908 INFO:teuthology.orchestra.run.trial137.stdout: osd objectstore = seastore 2026-02-25T08:40:47.908 INFO:teuthology.orchestra.run.trial137.stdout: osd shutdown pgref assert = True 2026-02-25T08:40:47.909 INFO:teuthology.orchestra.run.trial137.stdout: seastore cachepin size pershard = 64M 2026-02-25T08:40:47.909 INFO:teuthology.orchestra.run.trial137.stdout: seastore cachepin type = 2Q 2026-02-25T08:40:47.909 INFO:teuthology.orchestra.run.trial137.stdout: seastore max concurrent transactions = 8 2026-02-25T08:40:47.909 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.909 INFO:teuthology.orchestra.run.trial137.stdout:[mgr] 2026-02-25T08:40:47.909 INFO:teuthology.orchestra.run.trial137.stdout: debug ms = 1 2026-02-25T08:40:47.909 INFO:teuthology.orchestra.run.trial137.stdout: debug mgr = 20 2026-02-25T08:40:47.909 INFO:teuthology.orchestra.run.trial137.stdout: debug mon = 20 2026-02-25T08:40:47.909 INFO:teuthology.orchestra.run.trial137.stdout: debug auth = 20 2026-02-25T08:40:47.909 INFO:teuthology.orchestra.run.trial137.stdout: mon reweight min pgs per osd = 4 2026-02-25T08:40:47.909 INFO:teuthology.orchestra.run.trial137.stdout: mon reweight min bytes per osd = 10 2026-02-25T08:40:47.909 INFO:teuthology.orchestra.run.trial137.stdout: mgr/telemetry/nag = false 2026-02-25T08:40:47.909 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.910 INFO:teuthology.orchestra.run.trial137.stdout:[mon] 2026-02-25T08:40:47.910 INFO:teuthology.orchestra.run.trial137.stdout: debug ms = 0/0 2026-02-25T08:40:47.910 INFO:teuthology.orchestra.run.trial137.stdout: debug mon = 0/0 2026-02-25T08:40:47.910 INFO:teuthology.orchestra.run.trial137.stdout: debug paxos = 0/0 2026-02-25T08:40:47.910 INFO:teuthology.orchestra.run.trial137.stdout: debug auth = 20 2026-02-25T08:40:47.910 INFO:teuthology.orchestra.run.trial137.stdout: mon data avail warn = 5 2026-02-25T08:40:47.910 INFO:teuthology.orchestra.run.trial137.stdout: mon mgr mkfs grace = 240 2026-02-25T08:40:47.910 INFO:teuthology.orchestra.run.trial137.stdout: mon reweight min pgs per osd = 4 2026-02-25T08:40:47.910 INFO:teuthology.orchestra.run.trial137.stdout: mon osd reporter subtree level = osd 2026-02-25T08:40:47.910 INFO:teuthology.orchestra.run.trial137.stdout: mon osd prime pg temp = true 2026-02-25T08:40:47.910 INFO:teuthology.orchestra.run.trial137.stdout: mon reweight min bytes per osd = 10 2026-02-25T08:40:47.910 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.910 INFO:teuthology.orchestra.run.trial137.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-25T08:40:47.911 INFO:teuthology.orchestra.run.trial137.stdout: auth mon ticket ttl = 660 # 11m 2026-02-25T08:40:47.911 INFO:teuthology.orchestra.run.trial137.stdout: auth service ticket ttl = 240 # 4m 2026-02-25T08:40:47.911 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.911 INFO:teuthology.orchestra.run.trial137.stdout: # don't complain about insecure global_id in the test suite 2026-02-25T08:40:47.911 INFO:teuthology.orchestra.run.trial137.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-25T08:40:47.911 INFO:teuthology.orchestra.run.trial137.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-25T08:40:47.911 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.911 INFO:teuthology.orchestra.run.trial137.stdout: # 1m isn't quite enough 2026-02-25T08:40:47.911 INFO:teuthology.orchestra.run.trial137.stdout: mon_down_mkfs_grace = 2m 2026-02-25T08:40:47.911 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.911 INFO:teuthology.orchestra.run.trial137.stdout: mon_warn_on_filestore_osds = false 2026-02-25T08:40:47.911 INFO:teuthology.orchestra.run.trial137.stdout: osd pool default crimson = True 2026-02-25T08:40:47.912 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:40:47.912 INFO:teuthology.orchestra.run.trial137.stdout:[client] 2026-02-25T08:40:47.912 INFO:teuthology.orchestra.run.trial137.stdout: rgw cache enabled = true 2026-02-25T08:40:47.912 INFO:teuthology.orchestra.run.trial137.stdout: rgw enable ops log = true 2026-02-25T08:40:47.912 INFO:teuthology.orchestra.run.trial137.stdout: rgw enable usage log = true 2026-02-25T08:40:47.912 INFO:teuthology.orchestra.run.trial137.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-25T08:40:47.912 INFO:teuthology.orchestra.run.trial137.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-25T08:40:47.912 INFO:teuthology.orchestra.run.trial137.stdout:[mon.a] 2026-02-25T08:40:47.913 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-25T08:40:47.913 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-25T08:40:48.004 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-25T08:40:48.004 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:40:48.004 DEBUG:teuthology.orchestra.run.trial137:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-25T08:40:48.021 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:40:48.021 DEBUG:teuthology.orchestra.run.trial137:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-25T08:40:48.076 INFO:tasks.ceph:Sending monmap to node ubuntu@trial137.front.sepia.ceph.com 2026-02-25T08:40:48.076 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:40:48.076 DEBUG:teuthology.orchestra.run.trial137:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-25T08:40:48.077 DEBUG:teuthology.orchestra.run.trial137:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-25T08:40:48.149 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:40:48.149 DEBUG:teuthology.orchestra.run.trial137:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:48.204 INFO:tasks.ceph:Setting up mon nodes... 2026-02-25T08:40:48.205 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-25T08:40:48.205 DEBUG:teuthology.orchestra.run.trial137:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-25T08:40:48.294 INFO:teuthology.orchestra.run.trial137.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-25T08:40:48.297 INFO:tasks.ceph:Setting up mds nodes... 2026-02-25T08:40:48.297 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-25T08:40:48.298 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-25T08:40:48.339 INFO:teuthology.orchestra.run.trial137.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-25T08:40:48.350 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-25T08:40:48.350 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial137.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-25T08:40:48.350 DEBUG:teuthology.orchestra.run.trial137:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-25T08:40:48.415 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-25T08:40:48.415 INFO:tasks.ceph:role: osd.0 2026-02-25T08:40:48.415 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial137.front.sepia.ceph.com 2026-02-25T08:40:48.416 DEBUG:teuthology.orchestra.run.trial137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-25T08:40:48.480 INFO:teuthology.orchestra.run.trial137.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T08:40:48.480 INFO:teuthology.orchestra.run.trial137.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T08:40:48.480 INFO:teuthology.orchestra.run.trial137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T08:40:48.480 INFO:teuthology.orchestra.run.trial137.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T08:40:48.480 INFO:teuthology.orchestra.run.trial137.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T08:40:48.480 INFO:teuthology.orchestra.run.trial137.stdout: = sunit=0 swidth=0 blks 2026-02-25T08:40:48.480 INFO:teuthology.orchestra.run.trial137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T08:40:48.481 INFO:teuthology.orchestra.run.trial137.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T08:40:48.481 INFO:teuthology.orchestra.run.trial137.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T08:40:48.481 INFO:teuthology.orchestra.run.trial137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T08:40:49.050 INFO:teuthology.orchestra.run.trial137.stdout:Discarding blocks...Done. 2026-02-25T08:40:49.052 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial137.front.sepia.ceph.com -o noatime 2026-02-25T08:40:49.052 DEBUG:teuthology.orchestra.run.trial137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-25T08:40:49.121 DEBUG:teuthology.orchestra.run.trial137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-25T08:40:49.188 DEBUG:teuthology.orchestra.run.trial137:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-25T08:40:49.252 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-25T08:40:49.252 INFO:tasks.ceph:role: osd.1 2026-02-25T08:40:49.252 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial137.front.sepia.ceph.com 2026-02-25T08:40:49.252 DEBUG:teuthology.orchestra.run.trial137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-25T08:40:49.316 INFO:teuthology.orchestra.run.trial137.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T08:40:49.317 INFO:teuthology.orchestra.run.trial137.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T08:40:49.317 INFO:teuthology.orchestra.run.trial137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T08:40:49.317 INFO:teuthology.orchestra.run.trial137.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T08:40:49.317 INFO:teuthology.orchestra.run.trial137.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T08:40:49.317 INFO:teuthology.orchestra.run.trial137.stdout: = sunit=0 swidth=0 blks 2026-02-25T08:40:49.317 INFO:teuthology.orchestra.run.trial137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T08:40:49.317 INFO:teuthology.orchestra.run.trial137.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T08:40:49.317 INFO:teuthology.orchestra.run.trial137.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T08:40:49.317 INFO:teuthology.orchestra.run.trial137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T08:40:49.876 INFO:teuthology.orchestra.run.trial137.stdout:Discarding blocks...Done. 2026-02-25T08:40:49.878 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial137.front.sepia.ceph.com -o noatime 2026-02-25T08:40:49.878 DEBUG:teuthology.orchestra.run.trial137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-25T08:40:49.946 DEBUG:teuthology.orchestra.run.trial137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-25T08:40:50.014 DEBUG:teuthology.orchestra.run.trial137:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-25T08:40:50.078 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-25T08:40:50.078 INFO:tasks.ceph:role: osd.2 2026-02-25T08:40:50.078 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial137.front.sepia.ceph.com 2026-02-25T08:40:50.078 DEBUG:teuthology.orchestra.run.trial137:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-25T08:40:50.143 INFO:teuthology.orchestra.run.trial137.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T08:40:50.143 INFO:teuthology.orchestra.run.trial137.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T08:40:50.143 INFO:teuthology.orchestra.run.trial137.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T08:40:50.143 INFO:teuthology.orchestra.run.trial137.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T08:40:50.143 INFO:teuthology.orchestra.run.trial137.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T08:40:50.143 INFO:teuthology.orchestra.run.trial137.stdout: = sunit=0 swidth=0 blks 2026-02-25T08:40:50.143 INFO:teuthology.orchestra.run.trial137.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T08:40:50.143 INFO:teuthology.orchestra.run.trial137.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T08:40:50.143 INFO:teuthology.orchestra.run.trial137.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T08:40:50.144 INFO:teuthology.orchestra.run.trial137.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T08:40:50.700 INFO:teuthology.orchestra.run.trial137.stdout:Discarding blocks...Done. 2026-02-25T08:40:50.702 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial137.front.sepia.ceph.com -o noatime 2026-02-25T08:40:50.702 DEBUG:teuthology.orchestra.run.trial137:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-25T08:40:50.770 DEBUG:teuthology.orchestra.run.trial137:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-25T08:40:50.837 DEBUG:teuthology.orchestra.run.trial137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:50.989 INFO:teuthology.orchestra.run.trial137.stderr:INFO  osd - OSD::main: parsing early config 2026-02-25T08:40:51.042 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:40:51,041 seastar - Reactor backend: linux-aio 2026-02-25T08:40:51.059 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:40:51,059 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:51.060 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:40:51,059 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:51.061 INFO:teuthology.orchestra.run.trial137.stderr:==39491==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:51.120 INFO:teuthology.orchestra.run.trial137.stderr:DEBUG 2026-02-25 08:40:51,120 [shard 0:main] seastar - reactor::drain 2026-02-25T08:40:53.161 INFO:teuthology.orchestra.run.trial137.stderr:----------------------------------------------------- 2026-02-25T08:40:53.161 INFO:teuthology.orchestra.run.trial137.stderr:Suppressions used: 2026-02-25T08:40:53.162 INFO:teuthology.orchestra.run.trial137.stderr: count bytes template 2026-02-25T08:40:53.162 INFO:teuthology.orchestra.run.trial137.stderr: 1 8 MallocExtension::Register 2026-02-25T08:40:53.162 INFO:teuthology.orchestra.run.trial137.stderr:----------------------------------------------------- 2026-02-25T08:40:53.162 INFO:teuthology.orchestra.run.trial137.stderr: 2026-02-25T08:40:53.180 INFO:teuthology.orchestra.run.trial137.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-25T08:40:53.180 INFO:teuthology.orchestra.run.trial137.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-25T08:40:53.223 INFO:teuthology.orchestra.run.trial137.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-25T08:40:53.223 INFO:teuthology.orchestra.run.trial137.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-25T08:40:53.230 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:40:53,230 seastar - Reactor backend: linux-aio 2026-02-25T08:40:53.242 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:40:53,242 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:53.243 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:40:53,242 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:53.246 INFO:teuthology.orchestra.run.trial137.stderr:==39489==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:53.246 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:40:53,246 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-25T08:40:53.389 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0xccdc96 0x9dc0e62 0x9ef6d8f 0x9f1372b 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:53.390 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x157c7 0x10a2823 0xf47938 0x9e218ac 0x9e21ba9 0x9e21e1c 0x9e23b43 0x9f09a9d 0x9f09d0c 0x9f13f9a 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:53.390 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x9e2189b 0x9e21ba9 0x9e21e1c 0x9e220d2 0x9e22b81 0x9e2307f 0x9e236af 0x9e23819 0x9f03b89 0x9f1698c 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:53.422 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xea7eb 0xe40c90 0xe4104b 0x9ef6526 0x9ef66e2 0x9ef684b 0x9ef69ac 0x9ef6ea0 0x9f1568c 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba75ab 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7045030 0x74fc8bc 0x74fca10 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:53.489 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:53.529 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:53.529 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65d9 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:53.530 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a68ff7 0x7d56d00 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:53.657 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x9e2404d 0x9e2428f 0x9f095b1 0x9f09d0c 0x9f13f9a 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:53.658 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x9f0737a 0x9f0916c 0x9f097c6 0x9f09d0c 0x9f13f9a 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:53.658 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x9dddd3d 0x9f0a13c 0x9f0f82c 0x9f0fe6a 0x9f1a337 0x78b69db 0x7ba11e3 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:53.740 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xfcaef /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:53.756 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a68594 0x7d56cde 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:53.756 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a6865f 0x7d56cde 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:53.756 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1294d6 /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:55.916 INFO:teuthology.orchestra.run.trial137.stderr:----------------------------------------------------- 2026-02-25T08:40:55.916 INFO:teuthology.orchestra.run.trial137.stderr:Suppressions used: 2026-02-25T08:40:55.916 INFO:teuthology.orchestra.run.trial137.stderr: count bytes template 2026-02-25T08:40:55.916 INFO:teuthology.orchestra.run.trial137.stderr: 1 8 MallocExtension::Register 2026-02-25T08:40:55.916 INFO:teuthology.orchestra.run.trial137.stderr:----------------------------------------------------- 2026-02-25T08:40:55.916 INFO:teuthology.orchestra.run.trial137.stderr: 2026-02-25T08:40:55.953 DEBUG:teuthology.orchestra.run.trial137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-25T08:40:55.979 DEBUG:teuthology.orchestra.run.trial137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:40:56.130 INFO:teuthology.orchestra.run.trial137.stderr:INFO  osd - OSD::main: parsing early config 2026-02-25T08:40:56.185 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:40:56,185 seastar - Reactor backend: linux-aio 2026-02-25T08:40:56.203 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:40:56,203 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:56.204 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:40:56,203 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:56.206 INFO:teuthology.orchestra.run.trial137.stderr:==39558==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:56.265 INFO:teuthology.orchestra.run.trial137.stderr:DEBUG 2026-02-25 08:40:56,264 [shard 0:main] seastar - reactor::drain 2026-02-25T08:40:58.319 INFO:teuthology.orchestra.run.trial137.stderr:----------------------------------------------------- 2026-02-25T08:40:58.319 INFO:teuthology.orchestra.run.trial137.stderr:Suppressions used: 2026-02-25T08:40:58.319 INFO:teuthology.orchestra.run.trial137.stderr: count bytes template 2026-02-25T08:40:58.319 INFO:teuthology.orchestra.run.trial137.stderr: 1 8 MallocExtension::Register 2026-02-25T08:40:58.319 INFO:teuthology.orchestra.run.trial137.stderr:----------------------------------------------------- 2026-02-25T08:40:58.319 INFO:teuthology.orchestra.run.trial137.stderr: 2026-02-25T08:40:58.339 INFO:teuthology.orchestra.run.trial137.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-25T08:40:58.339 INFO:teuthology.orchestra.run.trial137.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-25T08:40:58.381 INFO:teuthology.orchestra.run.trial137.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-25T08:40:58.381 INFO:teuthology.orchestra.run.trial137.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-25T08:40:58.388 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:40:58,388 seastar - Reactor backend: linux-aio 2026-02-25T08:40:58.397 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:40:58,397 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:58.397 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:40:58,397 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:40:58.401 INFO:teuthology.orchestra.run.trial137.stderr:==39556==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:40:58.401 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:40:58,401 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-25T08:40:58.546 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0xe40c9c 0xe4104b 0x9d7520f 0x9eec474 0x9eec56d 0x7ba1825 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:58.547 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x10a275f 0xf47938 0x9f044e9 0x9f04ad6 0x9f04e59 0x9f14fb0 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:58.547 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x157c7 0x10a2823 0xf47938 0x9e216df 0x9e21ba9 0x9e21e1c 0x9e220d2 0x9e22b81 0x9e2307f 0x9e236af 0x9e23819 0x9f03b89 0x9f1698c 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:58.579 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x11380c8 0xf4792d 0x9e26106 0x9e26a45 0x9f12a15 0x9f15a0e 0x9f18413 0x9ed89e1 0x9ed79e1 0x911fd1b 0x9120f30 0x91240a1 0x7857565 0x7a67c92 0x704540f 0x74fc8bc 0x74fca10 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:58.622 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0f /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x157c7 0xca7484 0xcb94be 0x9e695c1 0x9f186e4 0x9ed89e1 0x9ed79e1 0x7f64059 0x7fb93c0 0x9121733 0x91240a1 0x7857565 0x7a67c92 0x704540f 0x74fc8bc 0x74fca10 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:58.622 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x129749 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x11380c8 0xf4792d 0x9e26106 0x9e26a45 0x9f12a15 0x9f151f1 0x9f18413 0x9ed89e1 0x9ed79e1 0x7f64059 0x7fb93c0 0x9121733 0x91240a1 0x7857565 0x7a67c92 0x704540f 0x74fc8bc 0x74fca10 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:58.622 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xfd193 /lib64/libasan.so.8+0xee859 0x9eec45d 0x9eec56d 0x7ba6efd 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7045030 0x74fc8bc 0x74fca10 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:58.645 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a1526e 0x2a69674 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:58.687 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:58.687 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a15220 0x2a69674 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:58.689 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a694fe 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:58.813 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x157c7 0x35fc095 0x35fc215 0x3618092 0x9e33da7 0x9e33ed4 0x9ef3c57 0x9ef3dec 0x9ef417e 0x9ef5765 0x9ef6a80 0x9ef6ea0 0x9f1372b 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:58.813 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65d5 /lib64/libasan.so.8+0x15197 0x11380c8 0xf4792d 0x9e216df 0x9e21ba9 0x9e21e1c 0x9e220d2 0x9e22b81 0x9e2307f 0x9e236af 0x9e23819 0x9f03b89 0x9f1698c 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:58.813 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb0fb0 /lib64/libasan.so.8+0xb1437 0xb7ac87 0x10a27ac 0xf47938 0x9e218ac 0x9e21ba9 0x9e21e1c 0x9e220d2 0x9e22694 0x9e2ab3b 0x9e2b21b 0x9f13d68 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:58.878 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:58.895 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x129747 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:40:58.911 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:40:58.911 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:01.056 INFO:teuthology.orchestra.run.trial137.stderr:----------------------------------------------------- 2026-02-25T08:41:01.056 INFO:teuthology.orchestra.run.trial137.stderr:Suppressions used: 2026-02-25T08:41:01.056 INFO:teuthology.orchestra.run.trial137.stderr: count bytes template 2026-02-25T08:41:01.056 INFO:teuthology.orchestra.run.trial137.stderr: 1 8 MallocExtension::Register 2026-02-25T08:41:01.056 INFO:teuthology.orchestra.run.trial137.stderr:----------------------------------------------------- 2026-02-25T08:41:01.056 INFO:teuthology.orchestra.run.trial137.stderr: 2026-02-25T08:41:01.090 DEBUG:teuthology.orchestra.run.trial137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-25T08:41:01.116 DEBUG:teuthology.orchestra.run.trial137:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:41:01.271 INFO:teuthology.orchestra.run.trial137.stderr:INFO  osd - OSD::main: parsing early config 2026-02-25T08:41:01.322 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:41:01,322 seastar - Reactor backend: linux-aio 2026-02-25T08:41:01.336 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:41:01,336 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:41:01.336 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:41:01,336 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:41:01.338 INFO:teuthology.orchestra.run.trial137.stderr:==39623==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:41:01.398 INFO:teuthology.orchestra.run.trial137.stderr:DEBUG 2026-02-25 08:41:01,398 [shard 0:main] seastar - reactor::drain 2026-02-25T08:41:03.452 INFO:teuthology.orchestra.run.trial137.stderr:----------------------------------------------------- 2026-02-25T08:41:03.452 INFO:teuthology.orchestra.run.trial137.stderr:Suppressions used: 2026-02-25T08:41:03.452 INFO:teuthology.orchestra.run.trial137.stderr: count bytes template 2026-02-25T08:41:03.452 INFO:teuthology.orchestra.run.trial137.stderr: 1 8 MallocExtension::Register 2026-02-25T08:41:03.453 INFO:teuthology.orchestra.run.trial137.stderr:----------------------------------------------------- 2026-02-25T08:41:03.453 INFO:teuthology.orchestra.run.trial137.stderr: 2026-02-25T08:41:03.472 INFO:teuthology.orchestra.run.trial137.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-25T08:41:03.472 INFO:teuthology.orchestra.run.trial137.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-25T08:41:03.513 INFO:teuthology.orchestra.run.trial137.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-25T08:41:03.513 INFO:teuthology.orchestra.run.trial137.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-25T08:41:03.520 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:41:03,520 seastar - Reactor backend: linux-aio 2026-02-25T08:41:03.538 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:41:03,538 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:41:03.538 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:41:03,538 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T08:41:03.542 INFO:teuthology.orchestra.run.trial137.stderr:==39621==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T08:41:03.542 INFO:teuthology.orchestra.run.trial137.stderr:INFO 2026-02-25 08:41:03,542 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-25T08:41:03.688 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x157e6 0x10a2823 0xf47938 0x9f10115 0x9f1a760 0x78b69db 0x7ba11e3 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:03.689 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0xe2a7cd 0x9ed14bf 0x9f16e40 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:03.689 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc25 /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x11380c8 0xf4792d 0x9e216df 0x9e21ba9 0x9e21e1c 0x9e220d2 0x9e22b81 0x9e2307f 0x9e236af 0x9e23819 0x9f03b89 0x9f1698c 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7044244 0x7048045 0x704845b 0x73b7c8f 0x73b7d14 0x73b7e16 0x73b7f0e 0x74780b4 0x72214da 0x7266e18 0x7398e9a 0x73efcd2 0x7221561 0x726701f 0x755dcc5 0x73b7f93 0x73b8071 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:03.721 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x157c7 0x10a2823 0xf47938 0x9e216df 0x9e21ba9 0x9e21e1c 0x9e23b43 0x9f0751a 0x9f0916c 0x9f097c6 0x9f09d0c 0x9f13f9a 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba5d84 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x7045030 0x74fc8bc 0x74fca10 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:41:03.764 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc589 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe40f5d 0x9edd3f3 0x9edd89d 0x9edda5f 0x9eddccf 0x9edddd4 0x9f1248b 0x9f12c2e 0x9f150f7 0x9f18413 0x9ed89e1 0x9ed79e1 0x7f64059 0x7fb93c0 0x9121733 0x91240a1 0x7857565 0x7a67c92 0x704540f 0x74fc8bc 0x74fca10 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:03.788 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:41:03.828 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x7d56cbd 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:03.828 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6583 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:03.830 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1294d2 /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:03.957 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119114 /lib64/libasan.so.8+0xebbf /lib64/libasan.so.8+0xef905 0x9ef902b 0x9ef9236 0x9ef9508 0x7ba18ef 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:03.957 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb052c /lib64/libasan.so.8+0xb1437 0xb7ac87 0x10a27ac 0xf47938 0x9e218ac 0x9e21ba9 0x9e21e1c 0x9e23b43 0x9e24090 0x9e2428f 0x9f095b1 0x9f09d0c 0x9f13f9a 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:03.957 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297b3 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x11380c8 0xf4792d 0x9e218ac 0x9e21ba9 0x9e21e1c 0x9e22e1c 0x9e2ae9f 0x9e2b21b 0x9f13d68 0x9f18413 0x9ed89e1 0x9ed79e1 0x7ba18b1 0x7bab3d4 0x7a28841 0x7a561e3 0x716287f 0x6fec95e 0x6ed2592 0x73b5492 0x73b5517 0x73b5619 0x73b5711 0x7477f38 0x72201a5 0x7260160 0x73973b5 0x73ea713 0x722022c 0x7260367 0x755b9c5 0x73b5796 0x73b5874 0x7397530 0x7495d5a 0x7529f3a 0x7529fb5 0x755ba46 0x747862e 0x75ba6f2 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:04.040 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa14b388 0x9dc9011 0x9dc984c 0xefd639 /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58474 2026-02-25T08:41:04.054 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a694cc 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:04.055 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1521e 0x2a696d3 0x7d56cc5 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:04.056 INFO:teuthology.orchestra.run.trial137.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ff8dc8 0x9ff90b1 0xa059a4f 0xa05a242 0xa05a4b0 0xa05a5f8 0xa05aab7 /lib64/libc.so.6+0x3fc2f 0x2a68696 0x7d56cde 0x7e509f0 0x7e51257 0x7e4eef9 0x785215d 0x791938e 0x792debd 0x792e01c 0x79715a5 0x7971723 0x797199c 0x7aad4d2 0x791a321 0x791a39c 0x7934952 0x7a8e989 0x7a8ea18 0x7a8eabc 0x754f8ce 0x744618d 0x7510b35 0x760171a 0xa0b3917 0xa007bec 0xa0e1387 0xa149b6a 0xa1645fa 0xa164c76 0x272f695 0xa772b80 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T08:41:06.224 INFO:teuthology.orchestra.run.trial137.stderr:----------------------------------------------------- 2026-02-25T08:41:06.225 INFO:teuthology.orchestra.run.trial137.stderr:Suppressions used: 2026-02-25T08:41:06.225 INFO:teuthology.orchestra.run.trial137.stderr: count bytes template 2026-02-25T08:41:06.225 INFO:teuthology.orchestra.run.trial137.stderr: 1 8 MallocExtension::Register 2026-02-25T08:41:06.225 INFO:teuthology.orchestra.run.trial137.stderr:----------------------------------------------------- 2026-02-25T08:41:06.225 INFO:teuthology.orchestra.run.trial137.stderr: 2026-02-25T08:41:06.259 DEBUG:teuthology.orchestra.run.trial137:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-25T08:41:06.285 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-25T08:41:06.285 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:41:06.286 DEBUG:teuthology.orchestra.run.trial137:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-25T08:41:06.349 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:41:06.349 DEBUG:teuthology.orchestra.run.trial137:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-25T08:41:06.413 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:41:06.413 DEBUG:teuthology.orchestra.run.trial137:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-25T08:41:06.477 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:41:06.477 DEBUG:teuthology.orchestra.run.trial137:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-25T08:41:06.541 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:41:06.541 DEBUG:teuthology.orchestra.run.trial137:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-25T08:41:06.597 INFO:tasks.ceph:Adding keys to all mons... 2026-02-25T08:41:06.597 DEBUG:teuthology.orchestra.run.trial137:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-25T08:41:06.659 INFO:teuthology.orchestra.run.trial137.stdout:[mgr.x] 2026-02-25T08:41:06.659 INFO:teuthology.orchestra.run.trial137.stdout: key = AQCQtZ5pVYONERAAmfWNmCYtaDBgn5kvq/eARw== 2026-02-25T08:41:06.659 INFO:teuthology.orchestra.run.trial137.stdout:[osd.0] 2026-02-25T08:41:06.659 INFO:teuthology.orchestra.run.trial137.stdout: key = AQCVtZ5phw6LEhAATT6DXDqX7ZCLGkuuFT3JgQ== 2026-02-25T08:41:06.659 INFO:teuthology.orchestra.run.trial137.stdout:[osd.1] 2026-02-25T08:41:06.659 INFO:teuthology.orchestra.run.trial137.stdout: key = AQCatZ5phQ78GxAAiBAV3iw3V66yvwuYvTU4Zg== 2026-02-25T08:41:06.659 INFO:teuthology.orchestra.run.trial137.stdout:[osd.2] 2026-02-25T08:41:06.660 INFO:teuthology.orchestra.run.trial137.stdout: key = AQCftZ5pXzY6JBAACiVBt7/Bq9NEmI9NPyR9yw== 2026-02-25T08:41:06.660 INFO:teuthology.orchestra.run.trial137.stdout:[client.0] 2026-02-25T08:41:06.660 INFO:teuthology.orchestra.run.trial137.stdout: key = AQCQtZ5pNdUxFBAA2hW+kvB5USbjCnQBN6K/NQ== 2026-02-25T08:41:06.661 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-25T08:41:06.750 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T08:41:06.799 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T08:41:06.847 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T08:41:06.895 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-25T08:41:06.945 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-25T08:41:06.945 DEBUG:teuthology.orchestra.run.trial137:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-25T08:41:06.969 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-25T08:41:07.056 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:07.054+0000 7f9fb245dc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:07.056 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:07.054+0000 7f9fb245dc00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:07.064 DEBUG:teuthology.orchestra.run.trial137:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-25T08:41:07.089 DEBUG:teuthology.orchestra.run.trial137:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-25T08:41:07.145 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-25T08:41:07.145 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-25T08:41:07.146 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-25T08:41:07.187 INFO:tasks.ceph.mon.a:Started 2026-02-25T08:41:07.187 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-25T08:41:07.187 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-25T08:41:07.188 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-25T08:41:07.190 INFO:tasks.ceph.mgr.x:Started 2026-02-25T08:41:07.190 DEBUG:tasks.ceph:set 0 configs 2026-02-25T08:41:07.190 DEBUG:teuthology.orchestra.run.trial137:> sudo ceph --cluster ceph config dump 2026-02-25T08:41:07.250 INFO:tasks.ceph.mon.a.trial137.stderr:2026-02-25T08:41:07.248+0000 7f5e35ec6c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:07.251 INFO:tasks.ceph.mon.a.trial137.stderr:2026-02-25T08:41:07.248+0000 7f5e35ec6c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:07.252 INFO:tasks.ceph.mon.a.trial137.stderr:2026-02-25T08:41:07.250+0000 7f5e35ec6c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:07.253 INFO:tasks.ceph.mgr.x.trial137.stderr:2026-02-25T08:41:07.251+0000 7fe6f3bc6fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:07.266 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:07.262+0000 7f39b97dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:07.267 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:07.266+0000 7f39b97dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:07.298 INFO:teuthology.orchestra.run.trial137.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-25T08:41:07.306 INFO:tasks.ceph:Setting crush tunables to default 2026-02-25T08:41:07.306 DEBUG:teuthology.orchestra.run.trial137:> sudo ceph --cluster ceph osd crush tunables default 2026-02-25T08:41:07.379 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:07.374+0000 7f21e24c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:07.380 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:07.379+0000 7f21e24c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:07.413 INFO:teuthology.orchestra.run.trial137.stderr:adjusted tunables profile to default 2026-02-25T08:41:07.421 INFO:tasks.ceph:check_enable_crimson: True 2026-02-25T08:41:07.422 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-25T08:41:07.422 DEBUG:teuthology.orchestra.run.trial137:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-25T08:41:07.456 INFO:tasks.ceph.mgr.x.trial137.stderr:2026-02-25T08:41:07.454+0000 7fe6f3bc6fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:07.457 INFO:tasks.ceph.mgr.x.trial137.stderr:2026-02-25T08:41:07.455+0000 7fe6f3bc6fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:07.495 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:07.490+0000 7f4c3675c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:07.496 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:07.494+0000 7f4c3675c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:07.533 INFO:tasks.ceph.mon.a.trial137.stderr:2026-02-25T08:41:07.531+0000 7f5e2d5f1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-25T08:41:07.533 INFO:tasks.ceph.mon.a.trial137.stderr:Please be aware that this feature is experimental, untested, 2026-02-25T08:41:07.533 INFO:tasks.ceph.mon.a.trial137.stderr:unsupported, and may result in data corruption, data loss, 2026-02-25T08:41:07.533 INFO:tasks.ceph.mon.a.trial137.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-25T08:41:07.533 INFO:tasks.ceph.mon.a.trial137.stderr:feature with important data. 2026-02-25T08:41:07.533 INFO:tasks.ceph.mon.a.trial137.stderr: 2026-02-25T08:41:08.294 INFO:tasks.ceph.mgr.x.trial137.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-25T08:41:08.295 INFO:tasks.ceph.mgr.x.trial137.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-25T08:41:08.295 INFO:tasks.ceph.mgr.x.trial137.stderr: from numpy import show_config as show_numpy_config 2026-02-25T08:41:08.424 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-25T08:41:08.424 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:41:08.424 DEBUG:teuthology.orchestra.run.trial137:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-25T08:41:08.490 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:41:08.490 DEBUG:teuthology.orchestra.run.trial137:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-25T08:41:08.554 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:41:08.554 DEBUG:teuthology.orchestra.run.trial137:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-25T08:41:08.619 DEBUG:teuthology.orchestra.run.trial137:> sudo ceph --cluster ceph osd new d7097b9b-2f34-4807-9c32-784203605e75 0 2026-02-25T08:41:08.735 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:08.730+0000 7f765fe5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.736 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:08.735+0000 7f765fe5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.772 INFO:teuthology.orchestra.run.trial137.stdout:0 2026-02-25T08:41:08.781 DEBUG:teuthology.orchestra.run.trial137:> sudo ceph --cluster ceph osd new 1503b297-ff9c-463e-820a-6e7e49d684a6 1 2026-02-25T08:41:08.857 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:08.852+0000 7f4009a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.858 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:08.856+0000 7f4009a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.894 INFO:teuthology.orchestra.run.trial137.stdout:1 2026-02-25T08:41:08.903 DEBUG:teuthology.orchestra.run.trial137:> sudo ceph --cluster ceph osd new 44844da5-0458-4857-b894-947ab31731f7 2 2026-02-25T08:41:08.977 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:08.972+0000 7f64e9f83640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:08.978 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:08.977+0000 7f64e9f83640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:09.015 INFO:teuthology.orchestra.run.trial137.stdout:2 2026-02-25T08:41:09.023 DEBUG:teuthology.orchestra.run.trial137:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-25T08:41:09.052 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-25T08:41:09.052 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-25T08:41:09.093 INFO:tasks.ceph.osd.0:Started 2026-02-25T08:41:09.093 DEBUG:teuthology.orchestra.run.trial137:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-25T08:41:09.121 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-25T08:41:09.121 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-25T08:41:09.162 INFO:tasks.ceph.osd.1:Started 2026-02-25T08:41:09.162 DEBUG:teuthology.orchestra.run.trial137:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-25T08:41:09.190 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-25T08:41:09.190 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-25T08:41:09.231 INFO:tasks.ceph.osd.2:Started 2026-02-25T08:41:09.232 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T08:41:09.310 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:09.307+0000 7fcc953dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:09.312 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:09.310+0000 7fcc953dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:09.344 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:41:09.344 INFO:teuthology.orchestra.run.trial137.stdout:{"epoch":6,"fsid":"72d844e8-55f4-45a1-8a75-b04b9565f21f","created":"2026-02-25T08:41:07.261353+0000","modified":"2026-02-25T08:41:09.014116+0000","last_up_change":"0.000000","last_in_change":"2026-02-25T08:41:09.014116+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"d7097b9b-2f34-4807-9c32-784203605e75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1503b297-ff9c-463e-820a-6e7e49d684a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"44844da5-0458-4857-b894-947ab31731f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:41:09.352 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-25T08:41:09.352 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-25T08:41:09.654 DEBUG:teuthology.orchestra.run.trial137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-25T08:41:09.722 INFO:teuthology.misc.health.trial137.stderr:2026-02-25T08:41:09.718+0000 7f3c0145a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:09.723 INFO:teuthology.misc.health.trial137.stderr:2026-02-25T08:41:09.721+0000 7f3c0145a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:09.755 INFO:teuthology.misc.health.trial137.stdout: 2026-02-25T08:41:09.755 INFO:teuthology.misc.health.trial137.stdout:{"epoch":6,"fsid":"72d844e8-55f4-45a1-8a75-b04b9565f21f","created":"2026-02-25T08:41:07.261353+0000","modified":"2026-02-25T08:41:09.014116+0000","last_up_change":"0.000000","last_in_change":"2026-02-25T08:41:09.014116+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"d7097b9b-2f34-4807-9c32-784203605e75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1503b297-ff9c-463e-820a-6e7e49d684a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"44844da5-0458-4857-b894-947ab31731f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:41:09.762 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-25T08:41:10.714 INFO:tasks.ceph.mgr.x.trial137.stderr:2026-02-25T08:41:10.712+0000 7fe60ead9640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-25T08:41:16.065 DEBUG:teuthology.orchestra.run.trial137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-25T08:41:16.155 INFO:teuthology.misc.health.trial137.stderr:2026-02-25T08:41:16.151+0000 7fb87310b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:16.158 INFO:teuthology.misc.health.trial137.stderr:2026-02-25T08:41:16.156+0000 7fb87310b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:16.288 INFO:teuthology.misc.health.trial137.stdout: 2026-02-25T08:41:16.288 INFO:teuthology.misc.health.trial137.stdout:{"epoch":8,"fsid":"72d844e8-55f4-45a1-8a75-b04b9565f21f","created":"2026-02-25T08:41:07.261353+0000","modified":"2026-02-25T08:41:16.034148+0000","last_up_change":"0.000000","last_in_change":"2026-02-25T08:41:09.014116+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"d7097b9b-2f34-4807-9c32-784203605e75","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"1503b297-ff9c-463e-820a-6e7e49d684a6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"44844da5-0458-4857-b894-947ab31731f7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:41:16.301 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-25T08:41:22.602 DEBUG:teuthology.orchestra.run.trial137:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-25T08:41:22.672 INFO:teuthology.misc.health.trial137.stderr:2026-02-25T08:41:22.668+0000 7f162e21a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:22.673 INFO:teuthology.misc.health.trial137.stderr:2026-02-25T08:41:22.671+0000 7f162e21a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:22.782 INFO:teuthology.misc.health.trial137.stdout: 2026-02-25T08:41:22.782 INFO:teuthology.misc.health.trial137.stdout:{"epoch":13,"fsid":"72d844e8-55f4-45a1-8a75-b04b9565f21f","created":"2026-02-25T08:41:07.261353+0000","modified":"2026-02-25T08:41:22.050091+0000","last_up_change":"2026-02-25T08:41:18.039754+0000","last_in_change":"2026-02-25T08:41:09.014116+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T08:41:19.726592+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7097b9b-2f34-4807-9c32-784203605e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6804","nonce":2317516262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6805","nonce":2317516262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6807","nonce":2317516262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6806","nonce":2317516262}]},"public_addr":"10.20.193.137:6804/2317516262","cluster_addr":"10.20.193.137:6805/2317516262","heartbeat_back_addr":"10.20.193.137:6807/2317516262","heartbeat_front_addr":"10.20.193.137:6806/2317516262","state":["exists","up"]},{"osd":1,"uuid":"1503b297-ff9c-463e-820a-6e7e49d684a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6802","nonce":1263939596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6803","nonce":1263939596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6816","nonce":1263939596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6815","nonce":1263939596}]},"public_addr":"10.20.193.137:6802/1263939596","cluster_addr":"10.20.193.137:6803/1263939596","heartbeat_back_addr":"10.20.193.137:6816/1263939596","heartbeat_front_addr":"10.20.193.137:6815/1263939596","state":["exists","up"]},{"osd":2,"uuid":"44844da5-0458-4857-b894-947ab31731f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6809","nonce":2379373857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6810","nonce":2379373857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6808","nonce":2379373857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6811","nonce":2379373857}]},"public_addr":"10.20.193.137:6809/2379373857","cluster_addr":"10.20.193.137:6810/2379373857","heartbeat_back_addr":"10.20.193.137:6808/2379373857","heartbeat_front_addr":"10.20.193.137:6811/2379373857","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:41:22.789 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-25T08:41:22.789 INFO:tasks.ceph:Creating RBD pool 2026-02-25T08:41:22.789 DEBUG:teuthology.orchestra.run.trial137:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-25T08:41:22.863 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:22.858+0000 7f1278a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:22.864 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:22.863+0000 7f1278a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:23.361 INFO:teuthology.orchestra.run.trial137.stderr:pool 'rbd' created 2026-02-25T08:41:23.374 DEBUG:teuthology.orchestra.run.trial137:> rbd --cluster ceph pool init rbd 2026-02-25T08:41:23.419 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:23.417+0000 7f4a9de9b240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:23.419 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:23.417+0000 7f4a9de9b240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:23.420 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:23.419+0000 7f4a9de9b240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:26.393 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-25T08:41:26.394 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-25T08:41:26.394 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-25T08:41:26.471 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:26.466+0000 7f58c4974640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:26.472 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:26.470+0000 7f58c4974640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:26.581 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:41:26.590 INFO:teuthology.orchestra.run.trial137.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-25T08:41:26.590 INFO:tasks.ceph_manager:config epoch is 1 2026-02-25T08:41:26.590 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-25T08:41:26.590 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-25T08:41:26.590 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-25T08:41:26.666 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:26.662+0000 7f701d1f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:26.667 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:26.665+0000 7f701d1f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:26.808 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:41:26.817 INFO:teuthology.orchestra.run.trial137.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6800","nonce":2346996079},{"type":"v1","addr":"10.20.193.137:6801","nonce":2346996079}]},"active_addr":"10.20.193.137:6801/2346996079","active_change":"2026-02-25T08:41:09.708044+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.137:0","nonce":3889584223}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.137:0","nonce":1874200405}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.137:0","nonce":3231931823}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.137:0","nonce":2591050478}]}]} 2026-02-25T08:41:26.821 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-25T08:41:26.821 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-25T08:41:26.821 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T08:41:26.898 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:26.894+0000 7f2089a11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:26.899 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:26.897+0000 7f2089a11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.008 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:41:27.008 INFO:teuthology.orchestra.run.trial137.stdout:{"epoch":17,"fsid":"72d844e8-55f4-45a1-8a75-b04b9565f21f","created":"2026-02-25T08:41:07.261353+0000","modified":"2026-02-25T08:41:26.367906+0000","last_up_change":"2026-02-25T08:41:18.039754+0000","last_in_change":"2026-02-25T08:41:09.014116+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T08:41:19.726592+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-25T08:41:22.976297+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7097b9b-2f34-4807-9c32-784203605e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6804","nonce":2317516262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6805","nonce":2317516262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6807","nonce":2317516262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6806","nonce":2317516262}]},"public_addr":"10.20.193.137:6804/2317516262","cluster_addr":"10.20.193.137:6805/2317516262","heartbeat_back_addr":"10.20.193.137:6807/2317516262","heartbeat_front_addr":"10.20.193.137:6806/2317516262","state":["exists","up"]},{"osd":1,"uuid":"1503b297-ff9c-463e-820a-6e7e49d684a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6802","nonce":1263939596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6803","nonce":1263939596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6816","nonce":1263939596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6815","nonce":1263939596}]},"public_addr":"10.20.193.137:6802/1263939596","cluster_addr":"10.20.193.137:6803/1263939596","heartbeat_back_addr":"10.20.193.137:6816/1263939596","heartbeat_front_addr":"10.20.193.137:6815/1263939596","state":["exists","up"]},{"osd":2,"uuid":"44844da5-0458-4857-b894-947ab31731f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6809","nonce":2379373857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6810","nonce":2379373857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6808","nonce":2379373857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6811","nonce":2379373857}]},"public_addr":"10.20.193.137:6809/2379373857","cluster_addr":"10.20.193.137:6810/2379373857","heartbeat_back_addr":"10.20.193.137:6808/2379373857","heartbeat_front_addr":"10.20.193.137:6811/2379373857","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:41:27.018 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-25T08:41:27.018 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T08:41:27.093 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.089+0000 7f56f4b0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.094 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.093+0000 7f56f4b0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.204 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:41:27.204 INFO:teuthology.orchestra.run.trial137.stdout:{"epoch":17,"fsid":"72d844e8-55f4-45a1-8a75-b04b9565f21f","created":"2026-02-25T08:41:07.261353+0000","modified":"2026-02-25T08:41:26.367906+0000","last_up_change":"2026-02-25T08:41:18.039754+0000","last_in_change":"2026-02-25T08:41:09.014116+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-25T08:41:19.726592+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-25T08:41:22.976297+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"d7097b9b-2f34-4807-9c32-784203605e75","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6804","nonce":2317516262}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6805","nonce":2317516262}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6807","nonce":2317516262}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6806","nonce":2317516262}]},"public_addr":"10.20.193.137:6804/2317516262","cluster_addr":"10.20.193.137:6805/2317516262","heartbeat_back_addr":"10.20.193.137:6807/2317516262","heartbeat_front_addr":"10.20.193.137:6806/2317516262","state":["exists","up"]},{"osd":1,"uuid":"1503b297-ff9c-463e-820a-6e7e49d684a6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6802","nonce":1263939596}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6803","nonce":1263939596}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6816","nonce":1263939596}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6815","nonce":1263939596}]},"public_addr":"10.20.193.137:6802/1263939596","cluster_addr":"10.20.193.137:6803/1263939596","heartbeat_back_addr":"10.20.193.137:6816/1263939596","heartbeat_front_addr":"10.20.193.137:6815/1263939596","state":["exists","up"]},{"osd":2,"uuid":"44844da5-0458-4857-b894-947ab31731f7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6809","nonce":2379373857}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6810","nonce":2379373857}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6808","nonce":2379373857}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.137:6811","nonce":2379373857}]},"public_addr":"10.20.193.137:6809/2379373857","cluster_addr":"10.20.193.137:6810/2379373857","heartbeat_back_addr":"10.20.193.137:6808/2379373857","heartbeat_front_addr":"10.20.193.137:6811/2379373857","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T08:41:27.214 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-25T08:41:27.214 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-25T08:41:27.215 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-25T08:41:27.296 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.291+0000 7f5cf0411640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.296 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.292+0000 7f91e059c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.297 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.291+0000 7fbb9f25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.297 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.296+0000 7f5cf0411640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.298 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.296+0000 7f91e059c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.298 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.296+0000 7fbb9f25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.312 INFO:teuthology.orchestra.run.trial137.stdout:42949672969 2026-02-25T08:41:27.313 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-25T08:41:27.314 INFO:teuthology.orchestra.run.trial137.stdout:38654705674 2026-02-25T08:41:27.314 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-25T08:41:27.315 INFO:teuthology.orchestra.run.trial137.stdout:42949672969 2026-02-25T08:41:27.316 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-25T08:41:27.434 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.430+0000 7f18665a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.435 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.430+0000 7f0890b9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.435 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.431+0000 7f3ca32cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.435 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.434+0000 7f0890b9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.436 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.434+0000 7f18665a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.436 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.435+0000 7f3ca32cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.545 INFO:teuthology.orchestra.run.trial137.stdout:42949672969 2026-02-25T08:41:27.550 INFO:teuthology.orchestra.run.trial137.stdout:38654705674 2026-02-25T08:41:27.552 INFO:teuthology.orchestra.run.trial137.stdout:42949672969 2026-02-25T08:41:27.556 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672969 for osd.2 2026-02-25T08:41:27.557 DEBUG:teuthology.parallel:result is None 2026-02-25T08:41:27.561 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705674 for osd.0 2026-02-25T08:41:27.561 DEBUG:teuthology.parallel:result is None 2026-02-25T08:41:27.562 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672969 for osd.1 2026-02-25T08:41:27.563 DEBUG:teuthology.parallel:result is None 2026-02-25T08:41:27.563 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-25T08:41:27.563 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T08:41:27.681 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.677+0000 7f449410b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.682 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.680+0000 7f449410b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.789 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:41:27.789 INFO:teuthology.orchestra.run.trial137.stderr:dumped all 2026-02-25T08:41:27.798 INFO:teuthology.orchestra.run.trial137.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-25T08:41:27.712108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":2836,"kb_used_data":2836,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156497132,"statfs":{"total":160255967232,"available":160253063168,"internally_reserved":0,"allocated":2904064,"data_stored":2904064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.405874"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.379755+0000","last_change":"2026-02-25T08:41:26.382188+0000","last_active":"2026-02-25T08:41:26.379755+0000","last_peered":"2026-02-25T08:41:26.379755+0000","last_clean":"2026-02-25T08:41:26.379755+0000","last_became_active":"2026-02-25T08:41:24.395137+0000","last_became_peered":"2026-02-25T08:41:24.395137+0000","last_unstale":"2026-02-25T08:41:26.379755+0000","last_undegraded":"2026-02-25T08:41:26.379755+0000","last_fullsized":"2026-02-25T08:41:26.379755+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_clean_scrub_stamp":"2026-02-25T08:41:23.360021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.381240+0000","last_change":"2026-02-25T08:41:26.383318+0000","last_active":"2026-02-25T08:41:26.381240+0000","last_peered":"2026-02-25T08:41:26.381240+0000","last_clean":"2026-02-25T08:41:26.381240+0000","last_became_active":"2026-02-25T08:41:24.393227+0000","last_became_peered":"2026-02-25T08:41:24.393227+0000","last_unstale":"2026-02-25T08:41:26.381240+0000","last_undegraded":"2026-02-25T08:41:26.381240+0000","last_fullsized":"2026-02-25T08:41:26.381240+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_clean_scrub_stamp":"2026-02-25T08:41:23.360021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.381283+0000","last_change":"2026-02-25T08:41:26.384977+0000","last_active":"2026-02-25T08:41:26.381283+0000","last_peered":"2026-02-25T08:41:26.381283+0000","last_clean":"2026-02-25T08:41:26.381283+0000","last_became_active":"2026-02-25T08:41:24.400095+0000","last_became_peered":"2026-02-25T08:41:24.400095+0000","last_unstale":"2026-02-25T08:41:26.381283+0000","last_undegraded":"2026-02-25T08:41:26.381283+0000","last_fullsized":"2026-02-25T08:41:26.381283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_clean_scrub_stamp":"2026-02-25T08:41:23.360021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.381249+0000","last_change":"2026-02-25T08:41:26.383435+0000","last_active":"2026-02-25T08:41:26.381249+0000","last_peered":"2026-02-25T08:41:26.381249+0000","last_clean":"2026-02-25T08:41:26.381249+0000","last_became_active":"2026-02-25T08:41:24.396672+0000","last_became_peered":"2026-02-25T08:41:24.396672+0000","last_unstale":"2026-02-25T08:41:26.381249+0000","last_undegraded":"2026-02-25T08:41:26.381249+0000","last_fullsized":"2026-02-25T08:41:26.381249+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_clean_scrub_stamp":"2026-02-25T08:41:23.360021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.382060+0000","last_change":"2026-02-25T08:41:26.382060+0000","last_active":"2026-02-25T08:41:26.382060+0000","last_peered":"2026-02-25T08:41:26.382060+0000","last_clean":"2026-02-25T08:41:26.382060+0000","last_became_active":"2026-02-25T08:41:24.400419+0000","last_became_peered":"2026-02-25T08:41:24.400419+0000","last_unstale":"2026-02-25T08:41:26.382060+0000","last_undegraded":"2026-02-25T08:41:26.382060+0000","last_fullsized":"2026-02-25T08:41:26.382060+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_clean_scrub_stamp":"2026-02-25T08:41:23.360021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.381004+0000","last_change":"2026-02-25T08:41:26.382802+0000","last_active":"2026-02-25T08:41:26.381004+0000","last_peered":"2026-02-25T08:41:26.381004+0000","last_clean":"2026-02-25T08:41:26.381004+0000","last_became_active":"2026-02-25T08:41:24.398428+0000","last_became_peered":"2026-02-25T08:41:24.398428+0000","last_unstale":"2026-02-25T08:41:26.381004+0000","last_undegraded":"2026-02-25T08:41:26.381004+0000","last_fullsized":"2026-02-25T08:41:26.381004+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_clean_scrub_stamp":"2026-02-25T08:41:23.360021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.379291+0000","last_change":"2026-02-25T08:41:26.382014+0000","last_active":"2026-02-25T08:41:26.379291+0000","last_peered":"2026-02-25T08:41:26.379291+0000","last_clean":"2026-02-25T08:41:26.379291+0000","last_became_active":"2026-02-25T08:41:24.395112+0000","last_became_peered":"2026-02-25T08:41:24.395112+0000","last_unstale":"2026-02-25T08:41:26.379291+0000","last_undegraded":"2026-02-25T08:41:26.379291+0000","last_fullsized":"2026-02-25T08:41:26.379291+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_clean_scrub_stamp":"2026-02-25T08:41:23.360021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.381361+0000","last_change":"2026-02-25T08:41:26.384903+0000","last_active":"2026-02-25T08:41:26.381361+0000","last_peered":"2026-02-25T08:41:26.381361+0000","last_clean":"2026-02-25T08:41:26.381361+0000","last_became_active":"2026-02-25T08:41:24.396287+0000","last_became_peered":"2026-02-25T08:41:24.396287+0000","last_unstale":"2026-02-25T08:41:26.381361+0000","last_undegraded":"2026-02-25T08:41:26.381361+0000","last_fullsized":"2026-02-25T08:41:26.381361+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_clean_scrub_stamp":"2026-02-25T08:41:23.360021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.379657+0000","last_change":"2026-02-25T08:41:21.077842+0000","last_active":"2026-02-25T08:41:26.379657+0000","last_peered":"2026-02-25T08:41:26.379657+0000","last_clean":"2026-02-25T08:41:26.379657+0000","last_became_active":"2026-02-25T08:41:21.071106+0000","last_became_peered":"2026-02-25T08:41:21.071106+0000","last_unstale":"2026-02-25T08:41:26.379657+0000","last_undegraded":"2026-02-25T08:41:26.379657+0000","last_fullsized":"2026-02-25T08:41:26.379657+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:20.041498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:20.041498+0000","last_clean_scrub_stamp":"2026-02-25T08:41:20.041498+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253063168,"internally_reserved":0,"allocated":2904064,"data_stored":2904064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":364,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166292,"statfs":{"total":53418655744,"available":53418283008,"internally_reserved":0,"allocated":372736,"data_stored":372736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1284,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165372,"statfs":{"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1188,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165468,"statfs":{"total":53418655744,"available":53417439232,"internally_reserved":0,"allocated":1216512,"data_stored":1216512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417439232,"internally_reserved":0,"allocated":1216512,"data_stored":1216512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418283008,"internally_reserved":0,"allocated":372736,"data_stored":372736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T08:41:27.799 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T08:41:27.875 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.871+0000 7ff5ff6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.876 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:27.874+0000 7ff5ff6cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:27.983 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:41:27.983 INFO:teuthology.orchestra.run.trial137.stderr:dumped all 2026-02-25T08:41:27.992 INFO:teuthology.orchestra.run.trial137.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-25T08:41:27.712108+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":2836,"kb_used_data":2836,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156497132,"statfs":{"total":160255967232,"available":160253063168,"internally_reserved":0,"allocated":2904064,"data_stored":2904064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.405874"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.379755+0000","last_change":"2026-02-25T08:41:26.382188+0000","last_active":"2026-02-25T08:41:26.379755+0000","last_peered":"2026-02-25T08:41:26.379755+0000","last_clean":"2026-02-25T08:41:26.379755+0000","last_became_active":"2026-02-25T08:41:24.395137+0000","last_became_peered":"2026-02-25T08:41:24.395137+0000","last_unstale":"2026-02-25T08:41:26.379755+0000","last_undegraded":"2026-02-25T08:41:26.379755+0000","last_fullsized":"2026-02-25T08:41:26.379755+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_clean_scrub_stamp":"2026-02-25T08:41:23.360021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.381240+0000","last_change":"2026-02-25T08:41:26.383318+0000","last_active":"2026-02-25T08:41:26.381240+0000","last_peered":"2026-02-25T08:41:26.381240+0000","last_clean":"2026-02-25T08:41:26.381240+0000","last_became_active":"2026-02-25T08:41:24.393227+0000","last_became_peered":"2026-02-25T08:41:24.393227+0000","last_unstale":"2026-02-25T08:41:26.381240+0000","last_undegraded":"2026-02-25T08:41:26.381240+0000","last_fullsized":"2026-02-25T08:41:26.381240+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_clean_scrub_stamp":"2026-02-25T08:41:23.360021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.381283+0000","last_change":"2026-02-25T08:41:26.384977+0000","last_active":"2026-02-25T08:41:26.381283+0000","last_peered":"2026-02-25T08:41:26.381283+0000","last_clean":"2026-02-25T08:41:26.381283+0000","last_became_active":"2026-02-25T08:41:24.400095+0000","last_became_peered":"2026-02-25T08:41:24.400095+0000","last_unstale":"2026-02-25T08:41:26.381283+0000","last_undegraded":"2026-02-25T08:41:26.381283+0000","last_fullsized":"2026-02-25T08:41:26.381283+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_clean_scrub_stamp":"2026-02-25T08:41:23.360021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.381249+0000","last_change":"2026-02-25T08:41:26.383435+0000","last_active":"2026-02-25T08:41:26.381249+0000","last_peered":"2026-02-25T08:41:26.381249+0000","last_clean":"2026-02-25T08:41:26.381249+0000","last_became_active":"2026-02-25T08:41:24.396672+0000","last_became_peered":"2026-02-25T08:41:24.396672+0000","last_unstale":"2026-02-25T08:41:26.381249+0000","last_undegraded":"2026-02-25T08:41:26.381249+0000","last_fullsized":"2026-02-25T08:41:26.381249+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_clean_scrub_stamp":"2026-02-25T08:41:23.360021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.382060+0000","last_change":"2026-02-25T08:41:26.382060+0000","last_active":"2026-02-25T08:41:26.382060+0000","last_peered":"2026-02-25T08:41:26.382060+0000","last_clean":"2026-02-25T08:41:26.382060+0000","last_became_active":"2026-02-25T08:41:24.400419+0000","last_became_peered":"2026-02-25T08:41:24.400419+0000","last_unstale":"2026-02-25T08:41:26.382060+0000","last_undegraded":"2026-02-25T08:41:26.382060+0000","last_fullsized":"2026-02-25T08:41:26.382060+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_clean_scrub_stamp":"2026-02-25T08:41:23.360021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.381004+0000","last_change":"2026-02-25T08:41:26.382802+0000","last_active":"2026-02-25T08:41:26.381004+0000","last_peered":"2026-02-25T08:41:26.381004+0000","last_clean":"2026-02-25T08:41:26.381004+0000","last_became_active":"2026-02-25T08:41:24.398428+0000","last_became_peered":"2026-02-25T08:41:24.398428+0000","last_unstale":"2026-02-25T08:41:26.381004+0000","last_undegraded":"2026-02-25T08:41:26.381004+0000","last_fullsized":"2026-02-25T08:41:26.381004+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_clean_scrub_stamp":"2026-02-25T08:41:23.360021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.379291+0000","last_change":"2026-02-25T08:41:26.382014+0000","last_active":"2026-02-25T08:41:26.379291+0000","last_peered":"2026-02-25T08:41:26.379291+0000","last_clean":"2026-02-25T08:41:26.379291+0000","last_became_active":"2026-02-25T08:41:24.395112+0000","last_became_peered":"2026-02-25T08:41:24.395112+0000","last_unstale":"2026-02-25T08:41:26.379291+0000","last_undegraded":"2026-02-25T08:41:26.379291+0000","last_fullsized":"2026-02-25T08:41:26.379291+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_clean_scrub_stamp":"2026-02-25T08:41:23.360021+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.381361+0000","last_change":"2026-02-25T08:41:26.384903+0000","last_active":"2026-02-25T08:41:26.381361+0000","last_peered":"2026-02-25T08:41:26.381361+0000","last_clean":"2026-02-25T08:41:26.381361+0000","last_became_active":"2026-02-25T08:41:24.396287+0000","last_became_peered":"2026-02-25T08:41:24.396287+0000","last_unstale":"2026-02-25T08:41:26.381361+0000","last_undegraded":"2026-02-25T08:41:26.381361+0000","last_fullsized":"2026-02-25T08:41:26.381361+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:23.360021+0000","last_clean_scrub_stamp":"2026-02-25T08:41:23.360021+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-25T08:41:26.379657+0000","last_change":"2026-02-25T08:41:21.077842+0000","last_active":"2026-02-25T08:41:26.379657+0000","last_peered":"2026-02-25T08:41:26.379657+0000","last_clean":"2026-02-25T08:41:26.379657+0000","last_became_active":"2026-02-25T08:41:21.071106+0000","last_became_peered":"2026-02-25T08:41:21.071106+0000","last_unstale":"2026-02-25T08:41:26.379657+0000","last_undegraded":"2026-02-25T08:41:26.379657+0000","last_fullsized":"2026-02-25T08:41:26.379657+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T08:41:20.041498+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T08:41:20.041498+0000","last_clean_scrub_stamp":"2026-02-25T08:41:20.041498+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253063168,"internally_reserved":0,"allocated":2904064,"data_stored":2904064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":364,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166292,"statfs":{"total":53418655744,"available":53418283008,"internally_reserved":0,"allocated":372736,"data_stored":372736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1284,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165372,"statfs":{"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1188,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165468,"statfs":{"total":53418655744,"available":53417439232,"internally_reserved":0,"allocated":1216512,"data_stored":1216512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417439232,"internally_reserved":0,"allocated":1216512,"data_stored":1216512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418283008,"internally_reserved":0,"allocated":372736,"data_stored":372736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T08:41:27.993 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-25T08:41:27.993 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-25T08:41:27.994 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-25T08:41:27.994 DEBUG:teuthology.orchestra.run.trial137:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-25T08:41:28.071 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:28.068+0000 7f4a273d3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:28.073 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:41:28.071+0000 7f4a273d3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:41:28.209 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:41:28.209 INFO:teuthology.orchestra.run.trial137.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-25T08:41:28.219 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-25T08:41:28.219 INFO:teuthology.run_tasks:Running task ssh_keys... 2026-02-25T08:41:28.321 DEBUG:teuthology.orchestra.run.trial137:> echo 'StrictHostKeyChecking no 2026-02-25T08:41:28.321 DEBUG:teuthology.orchestra.run.trial137:> ' > /home/ubuntu/.ssh/config && echo 'UserKnownHostsFile ' /dev/null >> /home/ubuntu/.ssh/config && chmod 600 /home/ubuntu/.ssh/config 2026-02-25T08:41:28.337 INFO:teuthology.task.ssh_keys:generated public key AAAAB3NzaC1yc2EAAAADAQABAAABAQCwysQzFxXM47D+Pkj6IUHCAKE4v/x/IrUxvjkz5FjIcNaQAK/l26BKvzNW+ve30ZoqFRQCJPKbG4IX/gh/C5vj5ikb0ydcIlAN4u0gx0JeCU7Hste4iSN/RrfOwa4Gqml64qNbyXiyV0/+21KxijnC6gPz/XpeHER+O5RHvHwXBYpfjsp58TQlsBcLCUyh7Lv6+TrZ6Aabkg+Y8dRZb2A/D+wk9+nVExG214bFJ7kw7VipX1iInUoM7/lfMDlvFjIP8nlH0n6LCfaA2swlTV5P2FwvNykkIWRhcANGXTr8XssvmpYTeT4woH9UClSbPdYpgf/BkjV/8me6zJvwekkD 2026-02-25T08:41:28.337 INFO:teuthology.task.ssh_keys:pushing keys to trial137.front.sepia.ceph.com for ubuntu 2026-02-25T08:41:28.337 DEBUG:teuthology.orchestra.run.trial137:> rm -f -- /home/ubuntu/.ssh/id_rsa 2026-02-25T08:41:28.393 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:41:28.393 DEBUG:teuthology.orchestra.run.trial137:> dd of=/home/ubuntu/.ssh/id_rsa 2026-02-25T08:41:28.393 DEBUG:teuthology.orchestra.run.trial137:> chmod 0500 /home/ubuntu/.ssh/id_rsa 2026-02-25T08:41:28.449 DEBUG:teuthology.orchestra.run.trial137:> rm -f -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-25T08:41:28.506 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:41:28.506 DEBUG:teuthology.orchestra.run.trial137:> dd of=/home/ubuntu/.ssh/id_rsa.pub 2026-02-25T08:41:28.561 DEBUG:teuthology.orchestra.run.trial137:> cp -v -a /home/ubuntu/.ssh/authorized_keys /home/ubuntu/.ssh/authorized_keys_2026-02-25_08:41:28:561671 2026-02-25T08:41:28.618 INFO:teuthology.orchestra.run.trial137.stdout:'/home/ubuntu/.ssh/authorized_keys' -> '/home/ubuntu/.ssh/authorized_keys_2026-02-25_08:41:28:561671' 2026-02-25T08:41:28.619 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:41:28.619 DEBUG:teuthology.orchestra.run.trial137:> dd of=/home/ubuntu/.ssh/authorized_keys conv=notrunc oflag=append 2026-02-25T08:41:28.676 INFO:teuthology.run_tasks:Running task cbt... 2026-02-25T08:41:28.701 INFO:tasks.cbt:cbt configuration is {'cluster': {'user': 'ubuntu', 'head': 'ubuntu@trial137.front.sepia.ceph.com', 'osds': ['ubuntu@trial137.front.sepia.ceph.com'], 'mons': ['ubuntu@trial137.front.sepia.ceph.com'], 'clients': ['ubuntu@trial137.front.sepia.ceph.com'], 'rgws': {'ubuntu@trial137.front.sepia.ceph.com': None}, 'osds_per_node': 3, 'rebuild_every_test': False, 'use_existing': True, 'is_teuthology': True, 'iterations': 1, 'tmp_dir': '/tmp/cbt', 'pool_profiles': {'replicated': {'pg_size': 256, 'pgp_size': 256, 'replication': 'replicated'}}, 'pid_dir': '/var/run/ceph'}, 'benchmarks': {'radosbench': {'concurrent_ops': 4, 'concurrent_procs': 2, 'op_size': [4194304], 'pool_monitoring_list': ['collectl'], 'pool_profile': 'replicated', 'run_monitoring_list': ['collectl'], 'time': 60, 'write_only': True}}, 'client_endpoints': None, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}} 2026-02-25T08:41:28.701 DEBUG:teuthology.orchestra.run.trial137:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/cbt 2026-02-25T08:41:28.736 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:41:28.736 DEBUG:teuthology.orchestra.run.trial137:> dd of=/home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2026-02-25T08:41:28.788 DEBUG:teuthology.orchestra.run.trial137:> git clone --depth 1 -b master https://github.com/ceph/cbt.git /home/ubuntu/cephtest/cbt 2026-02-25T08:41:28.852 INFO:teuthology.orchestra.run.trial137.stderr:Cloning into '/home/ubuntu/cephtest/cbt'... 2026-02-25T08:41:29.081 INFO:tasks.cbt:Installing collectl 2026-02-25T08:41:29.081 DEBUG:teuthology.orchestra.run.trial137:> sudo mkdir collectl && cd collectl && sudo wget https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download -O collectl.tar.gz && sudo tar -xvf collectl.tar.gz && cd collectl-4.3.1 && sudo ./INSTALL 2026-02-25T08:41:29.156 INFO:teuthology.orchestra.run.trial137.stderr:--2026-02-25 08:41:29-- https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download 2026-02-25T08:41:29.205 INFO:teuthology.orchestra.run.trial137.stderr:Resolving sourceforge.net (sourceforge.net)... 104.18.12.149, 104.18.13.149, 2606:4700::6812:d95, ... 2026-02-25T08:41:29.209 INFO:teuthology.orchestra.run.trial137.stderr:Connecting to sourceforge.net (sourceforge.net)|104.18.12.149|:443... connected. 2026-02-25T08:41:29.508 INFO:teuthology.orchestra.run.trial137.stderr:HTTP request sent, awaiting response... 302 Found 2026-02-25T08:41:29.509 INFO:teuthology.orchestra.run.trial137.stderr:Location: https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABpnrW5ju7EYGcgLmve07Q3tYGpMi7EhkXqykDQ5EMJpUDbmbqJ_oaEho4N9tDCmrMyF5ew7qd7OE4N_HoDhUqCwsVOPw%3D%3D&use_mirror=pilotfiber&r= [following] 2026-02-25T08:41:29.509 INFO:teuthology.orchestra.run.trial137.stderr:--2026-02-25 08:41:29-- https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABpnrW5ju7EYGcgLmve07Q3tYGpMi7EhkXqykDQ5EMJpUDbmbqJ_oaEho4N9tDCmrMyF5ew7qd7OE4N_HoDhUqCwsVOPw%3D%3D&use_mirror=pilotfiber&r= 2026-02-25T08:41:29.509 INFO:teuthology.orchestra.run.trial137.stderr:Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 104.18.13.149, 104.18.12.149, 2606:4700::6812:d95, ... 2026-02-25T08:41:29.512 INFO:teuthology.orchestra.run.trial137.stderr:Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|104.18.13.149|:443... connected. 2026-02-25T08:41:29.688 INFO:teuthology.orchestra.run.trial137.stderr:HTTP request sent, awaiting response... 302 Found 2026-02-25T08:41:29.688 INFO:teuthology.orchestra.run.trial137.stderr:Location: https://pilotfiber.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 [following] 2026-02-25T08:41:29.688 INFO:teuthology.orchestra.run.trial137.stderr:--2026-02-25 08:41:29-- https://pilotfiber.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 2026-02-25T08:41:29.694 INFO:teuthology.orchestra.run.trial137.stderr:Resolving pilotfiber.dl.sourceforge.net (pilotfiber.dl.sourceforge.net)... 216.158.133.250 2026-02-25T08:41:29.697 INFO:teuthology.orchestra.run.trial137.stderr:Connecting to pilotfiber.dl.sourceforge.net (pilotfiber.dl.sourceforge.net)|216.158.133.250|:443... connected. 2026-02-25T08:41:29.794 INFO:teuthology.orchestra.run.trial137.stderr:HTTP request sent, awaiting response... 200 OK 2026-02-25T08:41:29.794 INFO:teuthology.orchestra.run.trial137.stderr:Length: 620693 (606K) [application/x-gzip] 2026-02-25T08:41:29.794 INFO:teuthology.orchestra.run.trial137.stderr:Saving to: ‘collectl.tar.gz’ 2026-02-25T08:41:29.794 INFO:teuthology.orchestra.run.trial137.stderr: 2026-02-25T08:41:29.797 INFO:teuthology.orchestra.run.trial137.stderr: 0K .......... .......... .......... .......... .......... 8% 15.0M 0s 2026-02-25T08:41:29.799 INFO:teuthology.orchestra.run.trial137.stderr: 50K .......... .......... .......... .......... .......... 16% 16.1M 0s 2026-02-25T08:41:29.803 INFO:teuthology.orchestra.run.trial137.stderr: 100K .......... .......... .......... .......... .......... 24% 14.9M 0s 2026-02-25T08:41:29.813 INFO:teuthology.orchestra.run.trial137.stderr: 150K .......... .......... .......... .......... .......... 32% 15.6M 0s 2026-02-25T08:41:29.814 INFO:teuthology.orchestra.run.trial137.stderr: 200K .......... .......... .......... .......... .......... 41% 379M 0s 2026-02-25T08:41:29.814 INFO:teuthology.orchestra.run.trial137.stderr: 250K .......... .......... .......... .......... .......... 49% 403M 0s 2026-02-25T08:41:29.814 INFO:teuthology.orchestra.run.trial137.stderr: 300K .......... .......... .......... .......... .......... 57% 396M 0s 2026-02-25T08:41:29.814 INFO:teuthology.orchestra.run.trial137.stderr: 350K .......... .......... .......... .......... .......... 65% 17.0M 0s 2026-02-25T08:41:29.814 INFO:teuthology.orchestra.run.trial137.stderr: 400K .......... .......... .......... .......... .......... 74% 395M 0s 2026-02-25T08:41:29.814 INFO:teuthology.orchestra.run.trial137.stderr: 450K .......... .......... .......... .......... .......... 82% 397M 0s 2026-02-25T08:41:29.814 INFO:teuthology.orchestra.run.trial137.stderr: 500K .......... .......... .......... .......... .......... 90% 404M 0s 2026-02-25T08:41:29.814 INFO:teuthology.orchestra.run.trial137.stderr: 550K .......... .......... .......... .......... .......... 98% 364M 0s 2026-02-25T08:41:29.814 INFO:teuthology.orchestra.run.trial137.stderr: 600K ...... 100% 11.4T=0.02s 2026-02-25T08:41:29.814 INFO:teuthology.orchestra.run.trial137.stderr: 2026-02-25T08:41:29.815 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25 08:41:29 (36.0 MB/s) - ‘collectl.tar.gz’ saved [620693/620693] 2026-02-25T08:41:29.815 INFO:teuthology.orchestra.run.trial137.stderr: 2026-02-25T08:41:29.831 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/ 2026-02-25T08:41:29.831 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/collectl.conf 2026-02-25T08:41:29.831 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/ 2026-02-25T08:41:29.831 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/SlowProc.html 2026-02-25T08:41:29.831 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/BuddyInfo.html 2026-02-25T08:41:29.831 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/WhySummary.html 2026-02-25T08:41:29.831 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Socket.html 2026-02-25T08:41:29.831 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/colmux.html 2026-02-25T08:41:29.831 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Tutorial.html 2026-02-25T08:41:29.831 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Gexpr.html 2026-02-25T08:41:29.832 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Import.html 2026-02-25T08:41:29.832 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/style.css 2026-02-25T08:41:29.832 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Ganglia3.jpg 2026-02-25T08:41:29.832 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Ganglia2.jpg 2026-02-25T08:41:29.832 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/FileNaming.html 2026-02-25T08:41:29.832 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/TheMath.html 2026-02-25T08:41:29.832 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Environmental.html 2026-02-25T08:41:29.832 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Process.html 2026-02-25T08:41:29.832 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/OutputFiles.html 2026-02-25T08:41:29.832 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/HiResTime.html 2026-02-25T08:41:29.832 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/OutputFormats.html 2026-02-25T08:41:29.832 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/slub.jpg 2026-02-25T08:41:29.832 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Lustre.html 2026-02-25T08:41:29.832 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Documentation.html 2026-02-25T08:41:29.833 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Inodes.html 2026-02-25T08:41:29.833 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Graphite.html 2026-02-25T08:41:29.833 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/InputFiles.html 2026-02-25T08:41:29.833 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Export.html 2026-02-25T08:41:29.833 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Interrupts.html 2026-02-25T08:41:29.833 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Startup.html 2026-02-25T08:41:29.833 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Performance.html 2026-02-25T08:41:29.833 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Messages.html 2026-02-25T08:41:29.833 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Logging.html 2026-02-25T08:41:29.833 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Data-detail.html 2026-02-25T08:41:29.833 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/CPUs.html 2026-02-25T08:41:29.833 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Matrix.html 2026-02-25T08:41:29.833 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Data-brief.html 2026-02-25T08:41:29.833 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/SlabInfo.html 2026-02-25T08:41:29.834 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Misc.html 2026-02-25T08:41:29.834 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Examples.html 2026-02-25T08:41:29.834 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/OpenStack.html 2026-02-25T08:41:29.834 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Plotfiles.html 2026-02-25T08:41:29.834 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/ColmuxCPU.jpg 2026-02-25T08:41:29.834 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Tutorial-Lustre.html 2026-02-25T08:41:29.834 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Data-verbose.html 2026-02-25T08:41:29.834 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Data.html 2026-02-25T08:41:29.834 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Exceptions.html 2026-02-25T08:41:29.834 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/RunningAsAService.html 2026-02-25T08:41:29.834 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/FAQ-collectl.html 2026-02-25T08:41:29.834 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Ganglia.jpg 2026-02-25T08:41:29.834 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/OperationalModes.html 2026-02-25T08:41:29.834 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Playback.html 2026-02-25T08:41:29.834 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Hello.html 2026-02-25T08:41:29.835 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Sockets.html 2026-02-25T08:41:29.835 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Network.html 2026-02-25T08:41:29.835 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Infiniband.html 2026-02-25T08:41:29.835 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Memory.html 2026-02-25T08:41:29.835 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/ColmuxLustre.jpg 2026-02-25T08:41:29.835 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/Disks.html 2026-02-25T08:41:29.835 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/docs/NfsInfo.html 2026-02-25T08:41:29.835 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/service/ 2026-02-25T08:41:29.835 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/service/collectl.service 2026-02-25T08:41:29.835 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/initd/ 2026-02-25T08:41:29.835 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/initd/collectl-suse 2026-02-25T08:41:29.835 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/initd/collectl-debian 2026-02-25T08:41:29.835 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/initd/collectl-generic 2026-02-25T08:41:29.835 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/initd/collectl 2026-02-25T08:41:29.835 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/misc.ph 2026-02-25T08:41:29.836 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/gexpr.ph 2026-02-25T08:41:29.836 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/statsd.ph 2026-02-25T08:41:29.836 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/envrules.std 2026-02-25T08:41:29.836 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/hello.ph 2026-02-25T08:41:29.836 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/vmsum.ph 2026-02-25T08:41:29.836 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/proctree.ph 2026-02-25T08:41:29.836 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/README-WINDOWS 2026-02-25T08:41:29.836 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/formatit.ph 2026-02-25T08:41:29.836 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/colmux 2026-02-25T08:41:29.836 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/INSTALL 2026-02-25T08:41:29.836 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/graphite.ph 2026-02-25T08:41:29.836 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/RELEASE-collectl 2026-02-25T08:41:29.836 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/vmstat.ph 2026-02-25T08:41:29.836 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/README 2026-02-25T08:41:29.836 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/man1/ 2026-02-25T08:41:29.837 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/man1/collectl.1 2026-02-25T08:41:29.837 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/man1/colmux.1 2026-02-25T08:41:29.837 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/lexpr.ph 2026-02-25T08:41:29.837 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/client.pl 2026-02-25T08:41:29.837 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/collectl 2026-02-25T08:41:29.838 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/COPYING 2026-02-25T08:41:29.838 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/vnet.ph 2026-02-25T08:41:29.838 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/GPL 2026-02-25T08:41:29.838 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/UNINSTALL 2026-02-25T08:41:29.838 INFO:teuthology.orchestra.run.trial137.stdout:collectl-4.3.1/ARTISTIC 2026-02-25T08:41:29.873 INFO:teuthology.orchestra.run.trial137.stderr:./INSTALL: line 90: chkconfig: command not found 2026-02-25T08:41:29.875 DEBUG:teuthology.orchestra.run.trial137:> sudo yum -y install librbd-devel pdsh pdsh-rcmd-ssh perf 2026-02-25T08:41:30.237 INFO:teuthology.orchestra.run.trial137.stdout:Last metadata expiration check: 0:00:57 ago on Wed 25 Feb 2026 08:40:33 AM UTC. 2026-02-25T08:41:30.322 INFO:teuthology.orchestra.run.trial137.stdout:Dependencies resolved. 2026-02-25T08:41:30.323 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:41:30.323 INFO:teuthology.orchestra.run.trial137.stdout: Package Arch Version Repository Size 2026-02-25T08:41:30.323 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:41:30.324 INFO:teuthology.orchestra.run.trial137.stdout:Installing: 2026-02-25T08:41:30.324 INFO:teuthology.orchestra.run.trial137.stdout: librbd-devel x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 24 k 2026-02-25T08:41:30.324 INFO:teuthology.orchestra.run.trial137.stdout: pdsh x86_64 2.34-7.el9 epel 118 k 2026-02-25T08:41:30.324 INFO:teuthology.orchestra.run.trial137.stdout: pdsh-rcmd-ssh x86_64 2.34-7.el9 epel 13 k 2026-02-25T08:41:30.324 INFO:teuthology.orchestra.run.trial137.stdout: perf x86_64 5.14.0-686.el9 appstream 3.3 M 2026-02-25T08:41:30.324 INFO:teuthology.orchestra.run.trial137.stdout:Installing dependencies: 2026-02-25T08:41:30.324 INFO:teuthology.orchestra.run.trial137.stdout: libradospp-devel x86_64 2:20.3.0-5338.g6aeb6975.el9 ceph 31 k 2026-02-25T08:41:30.324 INFO:teuthology.orchestra.run.trial137.stdout: libtraceevent x86_64 1.8.4-2.el9 baseos 290 k 2026-02-25T08:41:30.324 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:41:30.324 INFO:teuthology.orchestra.run.trial137.stdout:Transaction Summary 2026-02-25T08:41:30.324 INFO:teuthology.orchestra.run.trial137.stdout:================================================================================ 2026-02-25T08:41:30.325 INFO:teuthology.orchestra.run.trial137.stdout:Install 6 Packages 2026-02-25T08:41:30.325 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:41:30.325 INFO:teuthology.orchestra.run.trial137.stdout:Total download size: 3.8 M 2026-02-25T08:41:30.325 INFO:teuthology.orchestra.run.trial137.stdout:Installed size: 14 M 2026-02-25T08:41:30.325 INFO:teuthology.orchestra.run.trial137.stdout:Downloading Packages: 2026-02-25T08:41:30.792 INFO:teuthology.orchestra.run.trial137.stdout:(1/6): librbd-devel-20.3.0-5338.g6aeb6975.el9.x 378 kB/s | 24 kB 00:00 2026-02-25T08:41:30.794 INFO:teuthology.orchestra.run.trial137.stdout:(2/6): libradospp-devel-20.3.0-5338.g6aeb6975.e 477 kB/s | 31 kB 00:00 2026-02-25T08:41:30.844 INFO:teuthology.orchestra.run.trial137.stdout:(3/6): libtraceevent-1.8.4-2.el9.x86_64.rpm 2.5 MB/s | 290 kB 00:00 2026-02-25T08:41:30.908 INFO:teuthology.orchestra.run.trial137.stdout:(4/6): pdsh-rcmd-ssh-2.34-7.el9.x86_64.rpm 206 kB/s | 13 kB 00:00 2026-02-25T08:41:30.930 INFO:teuthology.orchestra.run.trial137.stdout:(5/6): pdsh-2.34-7.el9.x86_64.rpm 870 kB/s | 118 kB 00:00 2026-02-25T08:41:31.047 INFO:teuthology.orchestra.run.trial137.stdout:(6/6): perf-5.14.0-686.el9.x86_64.rpm 13 MB/s | 3.3 MB 00:00 2026-02-25T08:41:31.049 INFO:teuthology.orchestra.run.trial137.stdout:-------------------------------------------------------------------------------- 2026-02-25T08:41:31.049 INFO:teuthology.orchestra.run.trial137.stdout:Total 5.2 MB/s | 3.8 MB 00:00 2026-02-25T08:41:31.073 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction check 2026-02-25T08:41:31.083 INFO:teuthology.orchestra.run.trial137.stdout:Transaction check succeeded. 2026-02-25T08:41:31.083 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction test 2026-02-25T08:41:31.141 INFO:teuthology.orchestra.run.trial137.stdout:Transaction test succeeded. 2026-02-25T08:41:31.142 INFO:teuthology.orchestra.run.trial137.stdout:Running transaction 2026-02-25T08:41:31.217 INFO:teuthology.orchestra.run.trial137.stdout: Preparing : 1/1 2026-02-25T08:41:31.222 INFO:teuthology.orchestra.run.trial137.stdout: Installing : pdsh-rcmd-ssh-2.34-7.el9.x86_64 1/6 2026-02-25T08:41:31.262 INFO:teuthology.orchestra.run.trial137.stdout: Installing : pdsh-2.34-7.el9.x86_64 2/6 2026-02-25T08:41:31.267 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libtraceevent-1.8.4-2.el9.x86_64 3/6 2026-02-25T08:41:31.269 INFO:teuthology.orchestra.run.trial137.stdout: Installing : libradospp-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 4/6 2026-02-25T08:41:31.437 INFO:teuthology.orchestra.run.trial137.stdout: Installing : librbd-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 5/6 2026-02-25T08:41:31.458 INFO:teuthology.orchestra.run.trial137.stdout: Installing : perf-5.14.0-686.el9.x86_64 6/6 2026-02-25T08:41:31.573 INFO:teuthology.orchestra.run.trial137.stdout: Running scriptlet: perf-5.14.0-686.el9.x86_64 6/6 2026-02-25T08:41:31.573 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libradospp-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 1/6 2026-02-25T08:41:31.573 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : librbd-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 2/6 2026-02-25T08:41:31.573 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : libtraceevent-1.8.4-2.el9.x86_64 3/6 2026-02-25T08:41:31.573 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : perf-5.14.0-686.el9.x86_64 4/6 2026-02-25T08:41:31.573 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : pdsh-2.34-7.el9.x86_64 5/6 2026-02-25T08:41:31.639 INFO:teuthology.orchestra.run.trial137.stdout: Verifying : pdsh-rcmd-ssh-2.34-7.el9.x86_64 6/6 2026-02-25T08:41:31.639 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:41:31.639 INFO:teuthology.orchestra.run.trial137.stdout:Installed: 2026-02-25T08:41:31.639 INFO:teuthology.orchestra.run.trial137.stdout: libradospp-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:41:31.639 INFO:teuthology.orchestra.run.trial137.stdout: librbd-devel-2:20.3.0-5338.g6aeb6975.el9.x86_64 2026-02-25T08:41:31.639 INFO:teuthology.orchestra.run.trial137.stdout: libtraceevent-1.8.4-2.el9.x86_64 2026-02-25T08:41:31.639 INFO:teuthology.orchestra.run.trial137.stdout: pdsh-2.34-7.el9.x86_64 2026-02-25T08:41:31.639 INFO:teuthology.orchestra.run.trial137.stdout: pdsh-rcmd-ssh-2.34-7.el9.x86_64 2026-02-25T08:41:31.639 INFO:teuthology.orchestra.run.trial137.stdout: perf-5.14.0-686.el9.x86_64 2026-02-25T08:41:31.639 INFO:teuthology.orchestra.run.trial137.stdout: 2026-02-25T08:41:31.639 INFO:teuthology.orchestra.run.trial137.stdout:Complete! 2026-02-25T08:41:31.669 DEBUG:teuthology.orchestra.run.trial137:> python3 -m pip install -r /home/ubuntu/cephtest/cbt/requirements.txt 2026-02-25T08:41:31.809 INFO:teuthology.orchestra.run.trial137.stdout:Defaulting to user installation because normal site-packages is not writeable 2026-02-25T08:41:31.819 INFO:teuthology.orchestra.run.trial137.stdout:Requirement already satisfied: pyyaml in /usr/lib64/python3.9/site-packages (from -r /home/ubuntu/cephtest/cbt/requirements.txt (line 1)) (5.4.1) 2026-02-25T08:41:31.819 INFO:teuthology.orchestra.run.trial137.stdout:Requirement already satisfied: lxml in /usr/lib64/python3.9/site-packages (from -r /home/ubuntu/cephtest/cbt/requirements.txt (line 2)) (4.6.5) 2026-02-25T08:41:32.146 INFO:teuthology.orchestra.run.trial137.stdout:Collecting matplotlib 2026-02-25T08:41:32.164 INFO:teuthology.orchestra.run.trial137.stdout: Downloading matplotlib-3.9.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.3 MB) 2026-02-25T08:41:32.635 INFO:teuthology.orchestra.run.trial137.stdout:Collecting pillow>=8 2026-02-25T08:41:32.642 INFO:teuthology.orchestra.run.trial137.stdout: Downloading pillow-11.3.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (6.6 MB) 2026-02-25T08:41:32.824 INFO:teuthology.orchestra.run.trial137.stdout:Collecting kiwisolver>=1.3.1 2026-02-25T08:41:32.830 INFO:teuthology.orchestra.run.trial137.stdout: Downloading kiwisolver-1.4.7-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB) 2026-02-25T08:41:32.838 INFO:teuthology.orchestra.run.trial137.stdout:Requirement already satisfied: pyparsing>=2.3.1 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (2.4.7) 2026-02-25T08:41:32.873 INFO:teuthology.orchestra.run.trial137.stdout:Collecting importlib-resources>=3.2.0 2026-02-25T08:41:32.876 INFO:teuthology.orchestra.run.trial137.stdout: Downloading importlib_resources-6.5.2-py3-none-any.whl (37 kB) 2026-02-25T08:41:32.877 INFO:teuthology.orchestra.run.trial137.stdout:Requirement already satisfied: packaging>=20.0 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (20.9) 2026-02-25T08:41:33.017 INFO:teuthology.orchestra.run.trial137.stdout:Collecting contourpy>=1.0.1 2026-02-25T08:41:33.020 INFO:teuthology.orchestra.run.trial137.stdout: Downloading contourpy-1.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (321 kB) 2026-02-25T08:41:33.023 INFO:teuthology.orchestra.run.trial137.stdout:Requirement already satisfied: python-dateutil>=2.7 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (2.9.0.post0) 2026-02-25T08:41:33.023 INFO:teuthology.orchestra.run.trial137.stdout:Requirement already satisfied: numpy>=1.23 in /usr/lib64/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (1.23.5) 2026-02-25T08:41:33.368 INFO:teuthology.orchestra.run.trial137.stdout:Collecting fonttools>=4.22.0 2026-02-25T08:41:33.374 INFO:teuthology.orchestra.run.trial137.stdout: Downloading fonttools-4.60.2-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.8 MB) 2026-02-25T08:41:33.408 INFO:teuthology.orchestra.run.trial137.stdout:Collecting cycler>=0.10 2026-02-25T08:41:33.412 INFO:teuthology.orchestra.run.trial137.stdout: Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB) 2026-02-25T08:41:33.463 INFO:teuthology.orchestra.run.trial137.stdout:Collecting zipp>=3.1.0 2026-02-25T08:41:33.466 INFO:teuthology.orchestra.run.trial137.stdout: Downloading zipp-3.23.0-py3-none-any.whl (10 kB) 2026-02-25T08:41:33.477 INFO:teuthology.orchestra.run.trial137.stdout:Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (1.15.0) 2026-02-25T08:41:33.593 INFO:teuthology.orchestra.run.trial137.stdout:Installing collected packages: zipp, pillow, kiwisolver, importlib-resources, fonttools, cycler, contourpy, matplotlib 2026-02-25T08:41:34.652 INFO:teuthology.orchestra.run.trial137.stdout:Successfully installed contourpy-1.3.0 cycler-0.12.1 fonttools-4.60.2 importlib-resources-6.5.2 kiwisolver-1.4.7 matplotlib-3.9.4 pillow-11.3.0 zipp-3.23.0 2026-02-25T08:41:34.760 INFO:tasks.cbt:benchmark: radosbench 2026-02-25T08:41:34.760 DEBUG:teuthology.orchestra.run.trial137:> sudo /sbin/sysctl -q -w kernel.perf_event_paranoid=0 2026-02-25T08:41:34.787 DEBUG:teuthology.orchestra.run.trial137:> /home/ubuntu/cephtest/cbt/cbt.py -a /home/ubuntu/cephtest/archive/cbt /home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2026-02-25T08:41:34.898 INFO:teuthology.orchestra.run.trial137.stderr:08:41:34 - DEBUG - cbt - Settings.cluster: 2026-02-25T08:41:34.898 INFO:teuthology.orchestra.run.trial137.stderr: {'archive_dir': '/home/ubuntu/cephtest/archive/cbt', 2026-02-25T08:41:34.898 INFO:teuthology.orchestra.run.trial137.stderr: 'clients': ['ubuntu@trial137.front.sepia.ceph.com'], 2026-02-25T08:41:34.898 INFO:teuthology.orchestra.run.trial137.stderr: 'conf_file': '/etc/ceph/ceph.conf', 2026-02-25T08:41:34.898 INFO:teuthology.orchestra.run.trial137.stderr: 'head': 'ubuntu@trial137.front.sepia.ceph.com', 2026-02-25T08:41:34.898 INFO:teuthology.orchestra.run.trial137.stderr: 'is_teuthology': True, 2026-02-25T08:41:34.898 INFO:teuthology.orchestra.run.trial137.stderr: 'iterations': 1, 2026-02-25T08:41:34.898 INFO:teuthology.orchestra.run.trial137.stderr: 'mons': ['ubuntu@trial137.front.sepia.ceph.com'], 2026-02-25T08:41:34.898 INFO:teuthology.orchestra.run.trial137.stderr: 'osds': ['ubuntu@trial137.front.sepia.ceph.com'], 2026-02-25T08:41:34.898 INFO:teuthology.orchestra.run.trial137.stderr: 'osds_per_node': 3, 2026-02-25T08:41:34.899 INFO:teuthology.orchestra.run.trial137.stderr: 'pid_dir': '/var/run/ceph', 2026-02-25T08:41:34.899 INFO:teuthology.orchestra.run.trial137.stderr: 'pool_profiles': {'replicated': {'pg_size': 256, 2026-02-25T08:41:34.899 INFO:teuthology.orchestra.run.trial137.stderr: 'pgp_size': 256, 2026-02-25T08:41:34.899 INFO:teuthology.orchestra.run.trial137.stderr: 'replication': 'replicated'}}, 2026-02-25T08:41:34.899 INFO:teuthology.orchestra.run.trial137.stderr: 'rebuild_every_test': False, 2026-02-25T08:41:34.899 INFO:teuthology.orchestra.run.trial137.stderr: 'rgws': {'ubuntu@trial137.front.sepia.ceph.com': None}, 2026-02-25T08:41:34.899 INFO:teuthology.orchestra.run.trial137.stderr: 'tmp_dir': '/tmp/cbt', 2026-02-25T08:41:34.899 INFO:teuthology.orchestra.run.trial137.stderr: 'use_existing': True, 2026-02-25T08:41:34.899 INFO:teuthology.orchestra.run.trial137.stderr: 'user': 'ubuntu'} 2026-02-25T08:41:34.899 INFO:teuthology.orchestra.run.trial137.stderr:08:41:34 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-9a441ab3 2026-02-25T08:41:34.899 INFO:teuthology.orchestra.run.trial137.stderr:08:41:34 - INFO - cbt - Pausing for 60s for idle monitoring. 2026-02-25T08:41:34.899 INFO:teuthology.orchestra.run.trial137.stderr:08:41:34 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/collectl 2026-02-25T08:41:35.065 INFO:teuthology.orchestra.run.trial137.stderr:08:41:35 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-25T08:41:35.067 INFO:teuthology.orchestra.run.trial137.stderr:08:41:35 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/perf 2026-02-25T08:41:35.231 INFO:teuthology.orchestra.run.trial137.stderr:08:41:35 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-25T08:41:35.231 INFO:teuthology.orchestra.run.trial137.stderr:08:41:35 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/perf/perf_stat.${pid} & done 2026-02-25T08:42:35.292 INFO:teuthology.orchestra.run.trial137.stderr:08:42:35 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-25T08:42:35.473 INFO:teuthology.orchestra.run.trial137.stderr:08:42:35 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-25T08:42:35.664 INFO:teuthology.orchestra.run.trial137.stderr:08:42:35 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/perf/perf.data 2026-02-25T08:42:35.666 INFO:teuthology.orchestra.run.trial137.stderr:08:42:35 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/perf/perf_stat.* 2026-02-25T08:42:35.668 INFO:teuthology.orchestra.run.trial137.stderr:08:42:35 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/* 2026-02-25T08:42:35.850 INFO:teuthology.orchestra.run.trial137.stderr:08:42:35 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial137.front.sepia.ceph.com -r /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-9a441ab3 2026-02-25T08:42:36.014 INFO:teuthology.orchestra.run.trial137.stderr:08:42:36 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo killall -9 rados 2026-02-25T08:42:36.191 INFO:teuthology.orchestra.run.trial137.stderr:08:42:36 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-9a441ab3 2026-02-25T08:42:36.191 INFO:teuthology.orchestra.run.trial137.stderr:08:42:36 - INFO - cbt - Skipping existing test in /home/ubuntu/cephtest/archive/cbt/results/00000000/id-9a441ab3. 2026-02-25T08:42:36.191 INFO:teuthology.orchestra.run.trial137.stderr:08:42:36 - INFO - cbt - Running benchmark /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004 2026-02-25T08:42:36.191 INFO:teuthology.orchestra.run.trial137.stderr:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-9a441ab3 2026-02-25T08:42:36.191 INFO:teuthology.orchestra.run.trial137.stderr:{'concurrent_ops': 4, 'concurrent_procs': 2, 'pool_profile': 'replicated', 'time': 60, 'write_only': True, 'op_size': 4194304, 'pool_monitoring_list': 'collectl', 'run_monitoring_list': 'collectl', 'benchmark': 'radosbench', 'iteration': 0} == iteration 0 == 2026-02-25T08:42:36.191 INFO:teuthology.orchestra.run.trial137.stderr:08:42:36 - INFO - cbt - Setting OSD Read Ahead to: 0 2026-02-25T08:42:36.192 INFO:teuthology.orchestra.run.trial137.stderr:08:42:36 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com find /dev/disk/by-partlabel/osd-device-*data -exec readlink {} \; | cut -d"/" -f 3 | sed "s/[0-9]$//" | xargs -I{} sudo sh -c "echo 0 > /sys/block/'{}'/queue/read_ahead_kb" 2026-02-25T08:42:36.192 INFO:teuthology.orchestra.run.trial137.stderr:08:42:36 - DEBUG - cbt - Cleaning existing temporary run directory: /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004 2026-02-25T08:42:36.193 INFO:teuthology.orchestra.run.trial137.stderr:08:42:36 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo rm -rf /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004 2026-02-25T08:42:36.368 INFO:teuthology.orchestra.run.trial137.stderr:08:42:36 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/collectl 2026-02-25T08:42:36.531 INFO:teuthology.orchestra.run.trial137.stderr:08:42:36 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-25T08:42:36.532 INFO:teuthology.orchestra.run.trial137.stderr:08:42:36 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/perf 2026-02-25T08:42:36.697 INFO:teuthology.orchestra.run.trial137.stderr:08:42:36 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-25T08:42:36.697 INFO:teuthology.orchestra.run.trial137.stderr:08:42:36 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/perf/perf_stat.${pid} & done 2026-02-25T08:42:36.699 INFO:teuthology.orchestra.run.trial137.stderr:08:42:36 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool delete rados-bench-cbt rados-bench-cbt --yes-i-really-really-mean-it 2026-02-25T08:42:37.039 INFO:teuthology.orchestra.run.trial137.stderr:08:42:37 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool create rados-bench-cbt 256 256 2026-02-25T08:42:37.431 INFO:teuthology.orchestra.run.trial137.stderr:08:42:37 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool application enable rados-bench-cbt radosbench 2026-02-25T08:42:38.436 INFO:teuthology.orchestra.run.trial137.stderr:08:42:38 - INFO - cbt - Checking Health after pool creation. 2026-02-25T08:42:43.439 INFO:teuthology.orchestra.run.trial137.stderr:08:42:43 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com /usr/bin/ceph -c /etc/ceph/ceph.conf health 2026-02-25T08:42:43.797 INFO:teuthology.orchestra.run.trial137.stderr:08:42:43 - INFO - cbt - Final Pool Health Check. 2026-02-25T08:42:48.803 INFO:teuthology.orchestra.run.trial137.stderr:08:42:48 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com /usr/bin/ceph -c /etc/ceph/ceph.conf health 2026-02-25T08:42:49.164 INFO:teuthology.orchestra.run.trial137.stderr:08:42:49 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-25T08:42:49.342 INFO:teuthology.orchestra.run.trial137.stderr:08:42:49 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-25T08:42:49.528 INFO:teuthology.orchestra.run.trial137.stderr:08:42:49 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/perf/perf.data 2026-02-25T08:42:49.530 INFO:teuthology.orchestra.run.trial137.stderr:08:42:49 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/perf/perf_stat.* 2026-02-25T08:42:49.531 INFO:teuthology.orchestra.run.trial137.stderr:08:42:49 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sync 2026-02-25T08:42:49.749 INFO:teuthology.orchestra.run.trial137.stderr:08:42:49 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com echo 3 | sudo tee /proc/sys/vm/drop_caches 2026-02-25T08:42:50.777 INFO:teuthology.orchestra.run.trial137.stderr:08:42:50 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -v 2026-02-25T08:42:50.989 INFO:teuthology.orchestra.run.trial137.stderr:08:42:50 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write 2026-02-25T08:42:51.147 INFO:teuthology.orchestra.run.trial137.stderr:08:42:51 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf daemon osd.0 config show > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/ceph_settings.out 2026-02-25T08:42:51.433 INFO:teuthology.orchestra.run.trial137.stderr:08:42:51 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/collectl 2026-02-25T08:42:51.595 INFO:teuthology.orchestra.run.trial137.stderr:08:42:51 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-25T08:42:51.597 INFO:teuthology.orchestra.run.trial137.stderr:08:42:51 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/perf 2026-02-25T08:42:51.763 INFO:teuthology.orchestra.run.trial137.stderr:08:42:51 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-25T08:42:51.763 INFO:teuthology.orchestra.run.trial137.stderr:08:42:51 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/perf/perf_stat.${pid} & done 2026-02-25T08:42:51.764 INFO:teuthology.orchestra.run.trial137.stderr:08:42:51 - INFO - cbt - Running radosbench write test. 2026-02-25T08:42:51.765 INFO:teuthology.orchestra.run.trial137.stderr:08:42:51 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench -b 4194304 60 write --concurrent-ios 4 --run-name `hostname -f`-0 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/objecter.0.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/output.0 2026-02-25T08:42:51.766 INFO:teuthology.orchestra.run.trial137.stderr:08:42:51 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench -b 4194304 60 write --concurrent-ios 4 --run-name `hostname -f`-1 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/objecter.1.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/output.1 2026-02-25T08:43:52.994 INFO:teuthology.orchestra.run.trial137.stderr:08:43:52 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-25T08:43:53.180 INFO:teuthology.orchestra.run.trial137.stderr:08:43:53 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-25T08:43:53.359 INFO:teuthology.orchestra.run.trial137.stderr:08:43:53 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/perf/perf.data 2026-02-25T08:43:53.361 INFO:teuthology.orchestra.run.trial137.stderr:08:43:53 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/perf/perf_stat.* 2026-02-25T08:43:53.363 INFO:teuthology.orchestra.run.trial137.stderr:08:43:53 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com find /var/run/ceph/ceph-osd*.asok -maxdepth 1 -exec sudo /usr/bin/ceph --admin-daemon {} dump_historic_ops \; > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/historic_ops.out 2026-02-25T08:43:53.731 INFO:teuthology.orchestra.run.trial137.stderr:08:43:53 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial137.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/* 2026-02-25T08:43:53.903 INFO:teuthology.orchestra.run.trial137.stderr:08:43:53 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial137.front.sepia.ceph.com -r /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-9a441ab3/write 2026-02-25T08:43:54.068 INFO:teuthology.orchestra.run.trial137.stderr:08:43:54 - INFO - cbt - Convert results to json format. 2026-02-25T08:43:54.080 DEBUG:teuthology.run_tasks:Unwinding manager cbt 2026-02-25T08:43:54.095 DEBUG:teuthology.orchestra.run.trial137:> rm --one-file-system -rf -- /home/ubuntu/cephtest/cbt 2026-02-25T08:43:54.122 INFO:tasks.cbt_performance:Collecting CBT performance data config 2026-02-25T08:43:54.122 INFO:tasks.cbt_performance:reading cbt preferences 2026-02-25T08:43:54.123 DEBUG:teuthology.orchestra.run.trial137:> find /home/ubuntu/cephtest -name 'json_output.*' 2026-02-25T08:43:54.181 INFO:teuthology.orchestra.run.trial137.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-9a441ab3/write/json_output.0.trial137.front.sepia.ceph.com 2026-02-25T08:43:54.181 INFO:teuthology.orchestra.run.trial137.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-9a441ab3/write/json_output.1.trial137.front.sepia.ceph.com 2026-02-25T08:43:54.182 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:43:54.182 DEBUG:teuthology.orchestra.run.trial137:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-9a441ab3/write/json_output.0.trial137.front.sepia.ceph.com of=/dev/stdout 2026-02-25T08:43:54.239 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-25T08:43:54.239 DEBUG:teuthology.orchestra.run.trial137:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-9a441ab3/write -name 'perf_stat.*' 2026-02-25T08:43:54.296 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-25T08:43:54.296 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-25T08:43:54.296 DEBUG:teuthology.orchestra.run.trial137:> set -ex 2026-02-25T08:43:54.296 DEBUG:teuthology.orchestra.run.trial137:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-9a441ab3/write/json_output.1.trial137.front.sepia.ceph.com of=/dev/stdout 2026-02-25T08:43:54.354 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-25T08:43:54.354 DEBUG:teuthology.orchestra.run.trial137:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-9a441ab3/write -name 'perf_stat.*' 2026-02-25T08:43:54.410 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-25T08:43:54.410 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-25T08:43:54.415 WARNING:tasks.cbt_performance:Could not send performance data to http://mira118.front.sepia.ceph.com:4000/cbt_performance: HTTPConnectionPool(host='mira118.front.sepia.ceph.com', port=4000): Max retries exceeded with url: /cbt_performance (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known')) 2026-02-25T08:43:54.415 WARNING:tasks.cbt_performance:Continuing without performance data collection 2026-02-25T08:43:54.416 DEBUG:teuthology.run_tasks:Unwinding manager ssh_keys 2026-02-25T08:43:54.438 INFO:teuthology.task.ssh_keys:Cleaning up SSH keys 2026-02-25T08:43:54.438 INFO:teuthology.task.ssh_keys:cleaning up keys added for testing 2026-02-25T08:43:54.438 INFO:teuthology.task.ssh_keys: cleaning up keys for user ubuntu on trial137.front.sepia.ceph.com 2026-02-25T08:43:54.438 DEBUG:teuthology.orchestra.run.trial137:> rm -- /home/ubuntu/.ssh/id_rsa 2026-02-25T08:43:54.467 DEBUG:teuthology.orchestra.run.trial137:> rm -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-25T08:43:54.523 DEBUG:teuthology.orchestra.run.trial137:> stat -c '"%a"' /home/ubuntu/.ssh/authorized_keys 2026-02-25T08:43:54.580 INFO:teuthology.orchestra.run.trial137.stdout:"600" 2026-02-25T08:43:54.581 DEBUG:teuthology.orchestra.run.trial137:> mv -- /home/ubuntu/.ssh/authorized_keys_2026-02-25_08:41:28:561671 /home/ubuntu/.ssh/authorized_keys 2026-02-25T08:43:54.638 DEBUG:teuthology.orchestra.run.trial137:> chmod 600 /home/ubuntu/.ssh/authorized_keys 2026-02-25T08:43:54.695 DEBUG:teuthology.orchestra.run.trial137:> rm /home/ubuntu/.ssh/config 2026-02-25T08:43:54.751 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-25T08:43:54.767 DEBUG:teuthology.orchestra.run.trial137:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-25T08:43:54.865 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:43:54.859+0000 7f946b10b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:54.866 INFO:teuthology.orchestra.run.trial137.stderr:2026-02-25T08:43:54.865+0000 7f946b10b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T08:43:54.988 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-25T08:43:54.989 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-25T08:43:54.989 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-25T08:43:54.989 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:43:55.079 INFO:tasks.ceph.osd.0:Stopped 2026-02-25T08:43:55.079 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-25T08:43:55.079 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:43:55.161 INFO:tasks.ceph.osd.1:Stopped 2026-02-25T08:43:55.162 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-25T08:43:55.162 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:43:55.241 INFO:tasks.ceph.osd.2:Stopped 2026-02-25T08:43:55.241 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-25T08:43:55.241 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-25T08:43:55.242 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:43:55.277 INFO:tasks.ceph.mgr.x:Stopped 2026-02-25T08:43:55.277 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-25T08:43:55.278 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-25T08:43:55.278 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T08:43:55.326 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2026-02-25T08:43:55.326 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2026-02-25T08:43:55.326 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2026-02-25T08:43:55.327 INFO:tasks.ceph.mon.a:Stopped 2026-02-25T08:43:55.327 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-25T08:43:55.327 DEBUG:teuthology.orchestra.run.trial137:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v '\(PG_' | grep -E -v '\(OSD_' | grep -E -v '\(OBJECT_' | grep -E -v 'overall HEALTH' | head -n 1 2026-02-25T08:43:55.392 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial137.front.sepia.ceph.com 2026-02-25T08:43:55.393 DEBUG:teuthology.orchestra.run.trial137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-25T08:43:55.469 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial137.front.sepia.ceph.com 2026-02-25T08:43:55.469 DEBUG:teuthology.orchestra.run.trial137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-25T08:43:55.539 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial137.front.sepia.ceph.com 2026-02-25T08:43:55.539 DEBUG:teuthology.orchestra.run.trial137:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-25T08:43:55.612 INFO:tasks.ceph:Archiving mon data... 2026-02-25T08:43:55.614 DEBUG:teuthology.misc:Transferring archived files from trial137:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69108/data/mon.a.tgz 2026-02-25T08:43:55.614 DEBUG:teuthology.orchestra.run.trial137:> mktemp 2026-02-25T08:43:55.668 INFO:teuthology.orchestra.run.trial137.stdout:/tmp/tmp.3PAwHsmMSM 2026-02-25T08:43:55.669 DEBUG:teuthology.orchestra.run.trial137:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.3PAwHsmMSM 2026-02-25T08:43:55.758 DEBUG:teuthology.orchestra.run.trial137:> sudo chmod 0666 /tmp/tmp.3PAwHsmMSM 2026-02-25T08:43:55.841 DEBUG:teuthology.orchestra.remote:trial137:/tmp/tmp.3PAwHsmMSM is 195KB 2026-02-25T08:43:55.897 DEBUG:teuthology.orchestra.run.trial137:> rm -fr /tmp/tmp.3PAwHsmMSM 2026-02-25T08:43:55.913 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-25T08:43:55.913 DEBUG:teuthology.orchestra.run.trial137:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-25T08:43:56.035 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-25T08:43:56.036 INFO:tasks.ceph:Archiving crash dumps... 2026-02-25T08:43:56.038 DEBUG:teuthology.misc:Transferring archived files from trial137:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69108/remote/trial137/crash 2026-02-25T08:43:56.039 DEBUG:teuthology.orchestra.run.trial137:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-25T08:43:56.099 INFO:tasks.ceph:Compressing logs... 2026-02-25T08:43:56.099 DEBUG:teuthology.orchestra.run.trial137:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T08:43:56.164 INFO:teuthology.orchestra.run.trial137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-25T08:43:56.172 INFO:teuthology.orchestra.run.trial137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41697.log 2026-02-25T08:43:56.172 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-25T08:43:56.172 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.41697.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-25T08:43:56.173 INFO:teuthology.orchestra.run.trial137.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41697.log.gz 2026-02-25T08:43:56.173 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45727.log 2026-02-25T08:43:56.173 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44492.log 2026-02-25T08:43:56.173 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.45727.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44434.log 2026-02-25T08:43:56.173 INFO:teuthology.orchestra.run.trial137.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.45727.log.gz 2026-02-25T08:43:56.173 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.44492.log: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-25T08:43:56.173 INFO:teuthology.orchestra.run.trial137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41305.log 2026-02-25T08:43:56.173 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.44434.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44492.log.gz 2026-02-25T08:43:56.173 INFO:teuthology.orchestra.run.trial137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40927.log 2026-02-25T08:43:56.173 INFO:teuthology.orchestra.run.trial137.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44434.log.gz 2026-02-25T08:43:56.173 INFO:teuthology.orchestra.run.trial137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41068.log 2026-02-25T08:43:56.173 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.41305.log: /var/log/ceph/ceph-client.admin.40927.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40998.log 2026-02-25T08:43:56.174 INFO:teuthology.orchestra.run.trial137.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41305.log.gz 2026-02-25T08:43:56.174 INFO:teuthology.orchestra.run.trial137.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40927.log.gz 2026-02-25T08:43:56.174 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.41068.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41554.log 2026-02-25T08:43:56.174 INFO:teuthology.orchestra.run.trial137.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41068.log.gz 2026-02-25T08:43:56.174 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.40998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40519.log 2026-02-25T08:43:56.174 INFO:teuthology.orchestra.run.trial137.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40998.log.gz 2026-02-25T08:43:56.174 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.41554.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40768.log 2026-02-25T08:43:56.174 INFO:teuthology.orchestra.run.trial137.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41554.log.gz 2026-02-25T08:43:56.174 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.40519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41553.log 2026-02-25T08:43:56.174 INFO:teuthology.orchestra.run.trial137.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40519.log.gz 2026-02-25T08:43:56.174 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.40768.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45132.log 2026-02-25T08:43:56.174 INFO:teuthology.orchestra.run.trial137.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40768.log.gz 2026-02-25T08:43:56.175 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.41553.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41406.log 2026-02-25T08:43:56.175 INFO:teuthology.orchestra.run.trial137.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41553.log.gz 2026-02-25T08:43:56.175 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.45132.log: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.45132.log.gz 2026-02-25T08:43:56.175 INFO:teuthology.orchestra.run.trial137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40472.log 2026-02-25T08:43:56.175 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.41406.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41207.log 2026-02-25T08:43:56.175 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.40472.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41406.log.gz 2026-02-25T08:43:56.175 INFO:teuthology.orchestra.run.trial137.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40472.log.gz 2026-02-25T08:43:56.175 INFO:teuthology.orchestra.run.trial137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44378.log 2026-02-25T08:43:56.175 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.41207.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.38856.log 2026-02-25T08:43:56.175 INFO:teuthology.orchestra.run.trial137.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41207.log.gz 2026-02-25T08:43:56.175 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.44378.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41407.log 2026-02-25T08:43:56.175 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph.tmp-client.admin.38856.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44378.log.gz 2026-02-25T08:43:56.175 INFO:teuthology.orchestra.run.trial137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44578.log 2026-02-25T08:43:56.176 INFO:teuthology.orchestra.run.trial137.stderr: 68.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.38856.log.gz 2026-02-25T08:43:56.176 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.41407.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-25T08:43:56.176 INFO:teuthology.orchestra.run.trial137.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41407.log.gz 2026-02-25T08:43:56.176 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.44578.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-25T08:43:56.176 INFO:teuthology.orchestra.run.trial137.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44578.log.gz 2026-02-25T08:43:56.176 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41158.log 2026-02-25T08:43:56.176 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45133.log 2026-02-25T08:43:56.176 INFO:teuthology.orchestra.run.trial137.stderr: 87.8% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-25T08:43:56.176 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.41158.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41256.log 2026-02-25T08:43:56.176 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.45133.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40288.log 2026-02-25T08:43:56.176 INFO:teuthology.orchestra.run.trial137.stderr: 66.4% -- replaced with /var/log/ceph/ceph-client.admin.45133.log.gz 2026-02-25T08:43:56.176 INFO:teuthology.orchestra.run.trial137.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41158.log.gz 2026-02-25T08:43:56.176 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.41256.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41746.log 2026-02-25T08:43:56.180 INFO:teuthology.orchestra.run.trial137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40425.log 2026-02-25T08:43:56.181 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.40288.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41256.log.gz 2026-02-25T08:43:56.181 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.41746.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40288.log.gz 2026-02-25T08:43:56.181 INFO:teuthology.orchestra.run.trial137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44524.log 2026-02-25T08:43:56.181 INFO:teuthology.orchestra.run.trial137.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41746.log.gz 2026-02-25T08:43:56.181 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.40425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41648.log 2026-02-25T08:43:56.181 INFO:teuthology.orchestra.run.trial137.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40425.log.gz 2026-02-25T08:43:56.181 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.44524.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-25T08:43:56.181 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.41648.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44524.log.gz 2026-02-25T08:43:56.181 INFO:teuthology.orchestra.run.trial137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40146.log 2026-02-25T08:43:56.181 INFO:teuthology.orchestra.run.trial137.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41648.log.gz 2026-02-25T08:43:56.181 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41555.log 2026-02-25T08:43:56.181 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.40146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41093.log 2026-02-25T08:43:56.182 INFO:teuthology.orchestra.run.trial137.stderr: 50.0%/var/log/ceph/ceph-client.admin.41555.log: -- replaced with /var/log/ceph/ceph-client.admin.40146.log.gz 2026-02-25T08:43:56.182 INFO:teuthology.orchestra.run.trial137.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40738.log 2026-02-25T08:43:56.182 INFO:teuthology.orchestra.run.trial137.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41555.log.gz 2026-02-25T08:43:56.182 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.41093.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-25T08:43:56.182 INFO:teuthology.orchestra.run.trial137.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.41093.log.gz 2026-02-25T08:43:56.182 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.40738.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41408.log 2026-02-25T08:43:56.182 INFO:teuthology.orchestra.run.trial137.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40738.log.gz 2026-02-25T08:43:56.182 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40241.log 2026-02-25T08:43:56.182 INFO:teuthology.orchestra.run.trial137.stderr: 84.8% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-25T08:43:56.182 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.41408.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41408.log.gz 2026-02-25T08:43:56.182 INFO:teuthology.orchestra.run.trial137.stderr:/var/log/ceph/ceph-client.admin.40241.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40241.log.gz 2026-02-25T08:43:56.183 INFO:teuthology.orchestra.run.trial137.stderr: 92.0% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-25T08:43:57.850 INFO:teuthology.orchestra.run.trial137.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-25T08:43:57.858 INFO:teuthology.orchestra.run.trial137.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-25T08:43:57.960 INFO:teuthology.orchestra.run.trial137.stderr: 92.0% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-25T08:43:57.961 INFO:teuthology.orchestra.run.trial137.stderr: 2026-02-25T08:43:57.961 INFO:teuthology.orchestra.run.trial137.stderr:real 0m1.806s 2026-02-25T08:43:57.961 INFO:teuthology.orchestra.run.trial137.stderr:user 0m4.887s 2026-02-25T08:43:57.961 INFO:teuthology.orchestra.run.trial137.stderr:sys 0m0.317s 2026-02-25T08:43:57.963 INFO:tasks.ceph:Archiving logs... 2026-02-25T08:43:57.963 DEBUG:teuthology.misc:Transferring archived files from trial137:/var/log/ceph to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69108/remote/trial137/log 2026-02-25T08:43:57.965 DEBUG:teuthology.orchestra.run.trial137:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-25T08:43:58.635 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-25T08:43:58.703 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-25T08:43:58.714 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-25T08:43:58.715 DEBUG:teuthology.orchestra.run.trial137:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-25T08:43:58.740 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-25T08:43:58.752 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-25T08:43:58.752 DEBUG:teuthology.orchestra.run.trial137:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T08:43:58.795 INFO:teuthology.orchestra.run.trial137.stderr:bash: line 1: ntpq: command not found 2026-02-25T08:43:58.818 INFO:teuthology.orchestra.run.trial137.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T08:43:58.818 INFO:teuthology.orchestra.run.trial137.stdout:=============================================================================== 2026-02-25T08:43:58.819 INFO:teuthology.orchestra.run.trial137.stdout:^* ip-64-111-99-224.nodes.d> 2 6 377 22 +697us[ +676us] +/- 25ms 2026-02-25T08:43:58.819 INFO:teuthology.orchestra.run.trial137.stdout:^+ 144.202.66.214.vultruser> 3 8 377 22 +434us[ +413us] +/- 29ms 2026-02-25T08:43:58.819 INFO:teuthology.orchestra.run.trial137.stdout:^+ time2.trtnw.net 2 6 377 22 -637us[ -658us] +/- 40ms 2026-02-25T08:43:58.819 INFO:teuthology.orchestra.run.trial137.stdout:^- 23.186.168.132 2 7 177 77 +482us[ +465us] +/- 56ms 2026-02-25T08:43:58.821 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-25T08:43:58.834 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-25T08:43:58.834 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-25T08:43:58.847 DEBUG:teuthology.orchestra.run.trial137:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-25T08:43:58.919 DEBUG:teuthology.orchestra.run.trial137:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T08:43:58.945 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:43:58.945 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-25T08:43:58.962 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-25T08:43:58.978 INFO:teuthology.task.internal:Duration was 2312.510292 seconds 2026-02-25T08:43:58.979 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-25T08:43:58.996 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-25T08:43:58.996 DEBUG:teuthology.orchestra.run.trial137:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-25T08:43:59.034 INFO:teuthology.orchestra.run.trial137.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T08:43:59.208 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-25T08:43:59.208 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial137.front.sepia.ceph.com 2026-02-25T08:43:59.208 DEBUG:teuthology.orchestra.run.trial137:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-25T08:43:59.268 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-25T08:43:59.268 DEBUG:teuthology.orchestra.run.trial137:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T08:43:59.373 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-25T08:43:59.374 DEBUG:teuthology.orchestra.run.trial137:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T08:43:59.398 INFO:teuthology.orchestra.run.trial137.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T08:43:59.398 INFO:teuthology.orchestra.run.trial137.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T08:43:59.398 INFO:teuthology.orchestra.run.trial137.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T08:43:59.398 INFO:teuthology.orchestra.run.trial137.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-25T08:43:59.399 INFO:teuthology.orchestra.run.trial137.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-25T08:43:59.409 INFO:teuthology.orchestra.run.trial137.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.6% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-25T08:43:59.411 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-25T08:43:59.430 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-25T08:43:59.430 DEBUG:teuthology.orchestra.run.trial137:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-25T08:43:59.474 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-25T08:43:59.492 DEBUG:teuthology.orchestra.run.trial137:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-25T08:43:59.538 INFO:teuthology.orchestra.run.trial137.stdout:kernel.core_pattern = core 2026-02-25T08:43:59.554 DEBUG:teuthology.orchestra.run.trial137:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-25T08:43:59.609 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T08:43:59.610 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-25T08:43:59.627 INFO:teuthology.task.internal:Transferring archived files... 2026-02-25T08:43:59.629 DEBUG:teuthology.misc:Transferring archived files from trial137:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:55:35-crimson-rados-wip-logstoremanager-distro-debug-trial/69108/remote/trial137 2026-02-25T08:43:59.629 DEBUG:teuthology.orchestra.run.trial137:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-25T08:43:59.707 INFO:teuthology.task.internal:Removing archive directory... 2026-02-25T08:43:59.708 DEBUG:teuthology.orchestra.run.trial137:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-25T08:43:59.740 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-25T08:43:59.762 INFO:teuthology.task.internal:Not uploading archives. 2026-02-25T08:43:59.762 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-25T08:43:59.780 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-25T08:43:59.780 DEBUG:teuthology.orchestra.run.trial137:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-25T08:43:59.796 INFO:teuthology.orchestra.run.trial137.stdout: 920834 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 25 08:43 /home/ubuntu/cephtest 2026-02-25T08:43:59.798 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-25T08:43:59.816 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-25T08:43:59.857 INFO:teuthology.run:Summary data: description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} perf_ignorelist settings/optimized workloads/radosbench_4M_write} duration: 2312.510291814804 flavor: debug owner: scheduled_matan@soko04.front.sepia.ceph.com success: true 2026-02-25T08:43:59.857 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-25T08:43:59.971 INFO:teuthology.run:pass