2026-02-21T00:11:52.456 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-21T00:11:52.472 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T00:11:52.551 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61964 branch: wip-liucm-osd-shard-redesign-crimson-only description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} perf_ignorelist settings/optimized workloads/radosbench_4M_seq_read} email: null first_in_suite: false flavor: debug job_id: '61964' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-liucm-osd-shard-redesign-crimson-only ceph: conf: global: auth client required: none auth cluster required: none auth service required: none auth supported: none debug asok: 0/0 debug auth: 0/0 debug bdev: 0/0 debug bluefs: 0/0 debug bluestore: 0/0 debug buffer: 0/0 debug civetweb: 0/0 debug client: 0/0 debug compressor: 0/0 debug context: 0/0 debug crush: 0/0 debug crypto: 0/0 debug dpdk: 0/0 debug eventtrace: 0/0 debug filer: 0/0 debug filestore: 0/0 debug finisher: 0/0 debug fuse: 0/0 debug heartbeatmap: 0/0 debug javaclient: 0/0 debug journal: 0/0 debug journaler: 0/0 debug kstore: 0/0 debug leveldb: 0/0 debug lockdep: 0/0 debug mds: 0/0 debug mds balancer: 0/0 debug mds locker: 0/0 debug mds log: 0/0 debug mds log expire: 0/0 debug mds migrator: 0/0 debug memdb: 0/0 debug mgr: 0/0 debug mgrc: 0/0 debug mon: 0/0 debug monc: 0/0 debug ms: 0/0 debug objclass: 0/0 debug objectcacher: 0/0 debug objecter: 0/0 debug optracker: 0/0 debug osd: 0/0 debug paxos: 0/0 debug perfcounter: 0/0 debug rados: 0/0 debug rbd: 0/0 debug rbd mirror: 0/0 debug rbd replay: 0/0 debug refs: 0/0 debug rgw: 0/0 debug rgw lifecycle: 0/0 debug rgw notification: 0/0 debug rgw sync: 0/0 debug rocksdb: 0/0 debug striper: 0/0 debug throttle: 0/0 debug timer: 0/0 debug tp: 0/0 enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 0/0 debug ms: 0/0 debug paxos: 0/0 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug filestore: 0/0 debug journal: 0/0 debug ms: 0/0 debug osd: 0/0 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore main device type: RANDOM_BLOCK_SSD seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - \(PG_ - \(OSD_ - \(OBJECT_ - overall HEALTH sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: flavor: debug sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f workunit: branch: wip-liucm-osd-shard-redesign-crimson-only env: CRIMSON_COMPAT: '1' sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f owner: scheduled_liucm@soko04.front.sepia.ceph.com priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 6283 sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f sleep_before_teardown: 0 suite: crimson-rados suite_branch: wip-liucm-osd-shard-redesign-crimson-only suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f targets: trial120.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG66fx57biwyrVPWfPX1FEs1YTPkdGF7zw1pBf+DBHObEX0zicQsyu9EALHHsKoW1jjYBtcvgrO0uIL3ltaCQM0= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: osd: debug monc: 20 wait-for-scrub: false - ssh_keys: null - cbt: benchmarks: radosbench: concurrent_ops: 4 concurrent_procs: 2 op_size: - 4194304 pool_monitoring_list: - collectl pool_profile: replicated run_monitoring_list: - collectl time: 60 write_only: false cluster: iterations: 1 osds_per_node: 3 pool_profiles: replicated: pg_size: 256 pgp_size: 256 replication: replicated user: ubuntu monitoring_profiles: perf: args: stat -p {pid} -o {perf_dir}/perf_stat.{pid} nodes: - osds perf_cmd: perf pid_glob: crimson-osd.*.pid teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-20_22:16:48 tube: trial user: liucm verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-21T00:11:52.551 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/qa; will attempt to use it 2026-02-21T00:11:52.552 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/qa/tasks 2026-02-21T00:11:52.552 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-21T00:11:52.553 INFO:teuthology.task.internal:Checking packages... 2026-02-21T00:11:52.554 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f' 2026-02-21T00:11:52.554 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-21T00:11:52.554 INFO:teuthology.packaging:ref: None 2026-02-21T00:11:52.554 INFO:teuthology.packaging:tag: None 2026-02-21T00:11:52.554 INFO:teuthology.packaging:branch: wip-liucm-osd-shard-redesign-crimson-only 2026-02-21T00:11:52.554 INFO:teuthology.packaging:sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-21T00:11:52.554 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=wip-liucm-osd-shard-redesign-crimson-only 2026-02-21T00:11:52.673 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5343.gbcdd429b 2026-02-21T00:11:52.675 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-21T00:11:52.687 INFO:teuthology.task.internal:no buildpackages task found 2026-02-21T00:11:52.687 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-21T00:11:52.701 INFO:teuthology.task.internal:Saving configuration 2026-02-21T00:11:52.709 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-21T00:11:52.720 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-21T00:11:52.788 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial120.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61964', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-21 00:08:29.320402', 'locked_by': 'scheduled_liucm@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG66fx57biwyrVPWfPX1FEs1YTPkdGF7zw1pBf+DBHObEX0zicQsyu9EALHHsKoW1jjYBtcvgrO0uIL3ltaCQM0='} 2026-02-21T00:11:52.789 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-21T00:11:52.800 INFO:teuthology.task.internal:roles: ubuntu@trial120.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-21T00:11:52.800 INFO:teuthology.run_tasks:Running task console_log... 2026-02-21T00:11:52.881 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x766267c2e660>, signals=[15]) 2026-02-21T00:11:52.881 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-21T00:11:52.893 INFO:teuthology.task.internal:Opening connections... 2026-02-21T00:11:52.893 DEBUG:teuthology.task.internal:connecting to ubuntu@trial120.front.sepia.ceph.com 2026-02-21T00:11:52.894 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T00:11:52.960 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-21T00:11:52.972 DEBUG:teuthology.orchestra.run.trial120:> uname -m 2026-02-21T00:11:53.036 INFO:teuthology.orchestra.run.trial120.stdout:x86_64 2026-02-21T00:11:53.036 DEBUG:teuthology.orchestra.run.trial120:> cat /etc/os-release 2026-02-21T00:11:53.098 INFO:teuthology.orchestra.run.trial120.stdout:NAME="CentOS Stream" 2026-02-21T00:11:53.098 INFO:teuthology.orchestra.run.trial120.stdout:VERSION="9" 2026-02-21T00:11:53.098 INFO:teuthology.orchestra.run.trial120.stdout:ID="centos" 2026-02-21T00:11:53.098 INFO:teuthology.orchestra.run.trial120.stdout:ID_LIKE="rhel fedora" 2026-02-21T00:11:53.098 INFO:teuthology.orchestra.run.trial120.stdout:VERSION_ID="9" 2026-02-21T00:11:53.099 INFO:teuthology.orchestra.run.trial120.stdout:PLATFORM_ID="platform:el9" 2026-02-21T00:11:53.099 INFO:teuthology.orchestra.run.trial120.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-21T00:11:53.099 INFO:teuthology.orchestra.run.trial120.stdout:ANSI_COLOR="0;31" 2026-02-21T00:11:53.099 INFO:teuthology.orchestra.run.trial120.stdout:LOGO="fedora-logo-icon" 2026-02-21T00:11:53.099 INFO:teuthology.orchestra.run.trial120.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-21T00:11:53.099 INFO:teuthology.orchestra.run.trial120.stdout:HOME_URL="https://centos.org/" 2026-02-21T00:11:53.099 INFO:teuthology.orchestra.run.trial120.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-21T00:11:53.099 INFO:teuthology.orchestra.run.trial120.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-21T00:11:53.100 INFO:teuthology.orchestra.run.trial120.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-21T00:11:53.101 INFO:teuthology.lock.ops:Updating trial120.front.sepia.ceph.com on lock server 2026-02-21T00:11:53.190 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-21T00:11:53.205 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-21T00:11:53.216 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-21T00:11:53.216 DEBUG:teuthology.orchestra.run.trial120:> test '!' -e /home/ubuntu/cephtest 2026-02-21T00:11:53.231 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-21T00:11:53.242 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-21T00:11:53.242 DEBUG:teuthology.orchestra.run.trial120:> test -z $(ls -A /var/lib/ceph) 2026-02-21T00:11:53.290 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-21T00:11:53.361 INFO:teuthology.run_tasks:Running task kernel... 2026-02-21T00:11:53.383 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-21T00:11:53.383 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-21T00:11:53.383 DEBUG:teuthology.orchestra.run.trial120:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-21T00:11:53.397 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T00:11:53.398 DEBUG:teuthology.orchestra.run.trial120:> uname -r 2026-02-21T00:11:53.456 INFO:teuthology.orchestra.run.trial120.stdout:5.14.0-681.el9.x86_64 2026-02-21T00:11:53.456 INFO:teuthology.task.kernel:Running kernel on trial120: 5.14.0-681.el9.x86_64 2026-02-21T00:11:53.456 DEBUG:teuthology.orchestra.run.trial120:> sudo yum install -y kernel 2026-02-21T00:11:55.318 INFO:teuthology.orchestra.run.trial120.stdout:CentOS Stream 9 - BaseOS 6.3 MB/s | 8.9 MB 00:01 2026-02-21T00:11:56.185 INFO:teuthology.orchestra.run.trial120.stdout:CentOS Stream 9 - AppStream 79 MB/s | 27 MB 00:00 2026-02-21T00:11:59.763 INFO:teuthology.orchestra.run.trial120.stdout:CentOS Stream 9 - CRB 6.5 MB/s | 7.8 MB 00:01 2026-02-21T00:12:00.621 INFO:teuthology.orchestra.run.trial120.stdout:CentOS Stream 9 - Extras packages 149 kB/s | 20 kB 00:00 2026-02-21T00:12:00.925 INFO:teuthology.orchestra.run.trial120.stdout:Extra Packages for Enterprise Linux 86 MB/s | 20 MB 00:00 2026-02-21T00:12:04.172 INFO:teuthology.orchestra.run.trial120.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-21T00:12:05.176 INFO:teuthology.orchestra.run.trial120.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-21T00:12:05.176 INFO:teuthology.orchestra.run.trial120.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-21T00:12:05.195 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:12:05.196 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:12:05.196 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:12:05.220 DEBUG:teuthology.orchestra.run.trial120:> echo no | sudo yum reinstall kernel || true 2026-02-21T00:12:05.518 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:01 ago on Sat 21 Feb 2026 12:12:04 AM UTC. 2026-02-21T00:12:05.599 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:12:05.599 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:12:05.600 INFO:teuthology.orchestra.run.trial120.stdout: Package Architecture Version Repository Size 2026-02-21T00:12:05.600 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:12:05.600 INFO:teuthology.orchestra.run.trial120.stdout:Reinstalling: 2026-02-21T00:12:05.600 INFO:teuthology.orchestra.run.trial120.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-21T00:12:05.600 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:12:05.600 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:12:05.600 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:12:05.600 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:12:05.600 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 803 k 2026-02-21T00:12:05.600 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 0 2026-02-21T00:12:05.600 INFO:teuthology.orchestra.run.trial120.stderr:Operation aborted. 2026-02-21T00:12:05.619 INFO:teuthology.orchestra.run.trial120.stdout:Is this ok [y/N]: 2026-02-21T00:12:05.620 DEBUG:teuthology.orchestra.run.trial120:> sudo yum reinstall -y kernel || true 2026-02-21T00:12:05.917 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:01 ago on Sat 21 Feb 2026 12:12:04 AM UTC. 2026-02-21T00:12:05.997 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:12:05.997 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:12:05.998 INFO:teuthology.orchestra.run.trial120.stdout: Package Architecture Version Repository Size 2026-02-21T00:12:05.998 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:12:05.998 INFO:teuthology.orchestra.run.trial120.stdout:Reinstalling: 2026-02-21T00:12:05.998 INFO:teuthology.orchestra.run.trial120.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-21T00:12:05.998 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:12:05.998 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:12:05.998 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:12:05.998 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:12:05.998 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 803 k 2026-02-21T00:12:05.998 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 0 2026-02-21T00:12:05.998 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:12:06.123 INFO:teuthology.orchestra.run.trial120.stdout:kernel-5.14.0-681.el9.x86_64.rpm 22 MB/s | 803 kB 00:00 2026-02-21T00:12:06.123 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:12:06.123 INFO:teuthology.orchestra.run.trial120.stdout:Total 6.3 MB/s | 803 kB 00:00 2026-02-21T00:12:06.133 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:12:06.167 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:12:06.167 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:12:06.171 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:12:06.171 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:12:06.195 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:12:06.199 INFO:teuthology.orchestra.run.trial120.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-21T00:12:06.199 INFO:teuthology.orchestra.run.trial120.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-21T00:12:06.311 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-21T00:12:06.311 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-21T00:12:06.364 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-21T00:12:06.364 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:12:06.364 INFO:teuthology.orchestra.run.trial120.stdout:Reinstalled: 2026-02-21T00:12:06.364 INFO:teuthology.orchestra.run.trial120.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-21T00:12:06.364 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:12:06.364 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:12:06.391 DEBUG:teuthology.orchestra.run.trial120:> rpm -q kernel | sort -rV | head -n 1 2026-02-21T00:12:06.414 INFO:teuthology.orchestra.run.trial120.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-21T00:12:06.416 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-21T00:12:06.416 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-21T00:12:06.416 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-21T00:12:06.416 DEBUG:teuthology.orchestra.run.trial120:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-21T00:12:06.479 INFO:teuthology.orchestra.run.trial120.stdout:ttyS1 2026-02-21T00:12:06.503 DEBUG:teuthology.parallel:result is None 2026-02-21T00:12:06.503 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-21T00:12:06.515 INFO:teuthology.task.internal:Creating test directory... 2026-02-21T00:12:06.515 DEBUG:teuthology.orchestra.run.trial120:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-21T00:12:06.559 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-21T00:12:06.573 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-21T00:12:06.585 INFO:teuthology.task.internal:Creating archive directory... 2026-02-21T00:12:06.585 DEBUG:teuthology.orchestra.run.trial120:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-21T00:12:06.620 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-21T00:12:06.634 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-21T00:12:06.635 DEBUG:teuthology.orchestra.run.trial120:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-21T00:12:06.681 INFO:teuthology.orchestra.run.trial120.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T00:12:06.689 INFO:teuthology.orchestra.run.trial120.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-21T00:12:06.691 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-21T00:12:06.703 INFO:teuthology.task.internal:Configuring sudo... 2026-02-21T00:12:06.704 DEBUG:teuthology.orchestra.run.trial120:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-21T00:12:06.756 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-21T00:12:06.773 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-21T00:12:06.774 DEBUG:teuthology.orchestra.run.trial120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-21T00:12:06.812 DEBUG:teuthology.orchestra.run.trial120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T00:12:06.943 DEBUG:teuthology.orchestra.run.trial120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T00:12:06.968 DEBUG:teuthology.orchestra.run.trial120:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T00:12:07.120 DEBUG:teuthology.orchestra.run.trial120:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T00:12:07.143 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:12:07.144 DEBUG:teuthology.orchestra.run.trial120:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-21T00:12:07.208 DEBUG:teuthology.orchestra.run.trial120:> sudo service rsyslog restart 2026-02-21T00:12:07.275 INFO:teuthology.orchestra.run.trial120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-21T00:12:07.662 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-21T00:12:07.674 INFO:teuthology.task.internal:Starting timer... 2026-02-21T00:12:07.674 INFO:teuthology.run_tasks:Running task pcp... 2026-02-21T00:12:07.695 INFO:teuthology.run_tasks:Running task selinux... 2026-02-21T00:12:07.713 DEBUG:teuthology.orchestra.run.trial120:> sudo service auditd rotate 2026-02-21T00:12:07.741 INFO:teuthology.orchestra.run.trial120.stdout:Rotating logs: 2026-02-21T00:12:07.742 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-21T00:12:07.742 DEBUG:teuthology.orchestra.run.trial120:> /usr/sbin/getenforce 2026-02-21T00:12:07.798 INFO:teuthology.orchestra.run.trial120.stdout:Permissive 2026-02-21T00:12:07.799 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial120.front.sepia.ceph.com': 'permissive'} 2026-02-21T00:12:07.799 DEBUG:teuthology.orchestra.run.trial120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-21T00:12:07.862 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T00:12:07.863 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-21T00:12:07.863 DEBUG:teuthology.orchestra.run.trial120:> sudo /usr/sbin/setenforce permissive 2026-02-21T00:12:07.927 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-21T00:12:07.944 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-21T00:12:07.957 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-21T00:12:07.959 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial120.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-21T00:14:47.707 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial120.front.sepia.ceph.com')] 2026-02-21T00:14:47.709 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial120.front.sepia.ceph.com' 2026-02-21T00:14:47.711 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial120.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-21T00:14:47.821 DEBUG:teuthology.orchestra.run.trial120:> true 2026-02-21T00:14:47.898 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial120.front.sepia.ceph.com' 2026-02-21T00:14:47.898 INFO:teuthology.run_tasks:Running task clock... 2026-02-21T00:14:47.912 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-21T00:14:47.912 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-21T00:14:47.913 DEBUG:teuthology.orchestra.run.trial120:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T00:14:47.967 INFO:teuthology.orchestra.run.trial120.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-21T00:14:47.979 INFO:teuthology.orchestra.run.trial120.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-21T00:14:47.998 INFO:teuthology.orchestra.run.trial120.stderr:sudo: ntpd: command not found 2026-02-21T00:14:48.008 INFO:teuthology.orchestra.run.trial120.stdout:506 Cannot talk to daemon 2026-02-21T00:14:48.018 INFO:teuthology.orchestra.run.trial120.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-21T00:14:48.030 INFO:teuthology.orchestra.run.trial120.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-21T00:14:48.079 INFO:teuthology.orchestra.run.trial120.stderr:bash: line 1: ntpq: command not found 2026-02-21T00:14:48.101 INFO:teuthology.orchestra.run.trial120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-21T00:14:48.101 INFO:teuthology.orchestra.run.trial120.stdout:=============================================================================== 2026-02-21T00:14:48.101 INFO:teuthology.orchestra.run.trial120.stdout:^? acheron.bitsrc.net 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-21T00:14:48.101 INFO:teuthology.orchestra.run.trial120.stdout:^? ntp.netlinkify.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-21T00:14:48.101 INFO:teuthology.orchestra.run.trial120.stdout:^? 159.203.158.197 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-21T00:14:48.101 INFO:teuthology.orchestra.run.trial120.stdout:^? triton.ellipse.net 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-21T00:14:48.103 INFO:teuthology.run_tasks:Running task install... 2026-02-21T00:14:48.116 DEBUG:teuthology.task.install:project ceph 2026-02-21T00:14:48.117 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}} 2026-02-21T00:14:48.117 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'} 2026-02-21T00:14:48.117 INFO:teuthology.task.install:Using flavor: debug 2026-02-21T00:14:48.126 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-21T00:14:48.126 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-21T00:14:48.126 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'tag': None, 'wait_for_package': False} 2026-02-21T00:14:48.127 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-21T00:14:48.289 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/ 2026-02-21T00:14:48.289 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5343.gbcdd429b 2026-02-21T00:14:48.627 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-21T00:14:48.627 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:14:48.627 DEBUG:teuthology.orchestra.run.trial120:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-21T00:14:48.652 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-21T00:14:48.652 DEBUG:teuthology.orchestra.run.trial120:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-21T00:14:48.716 DEBUG:teuthology.orchestra.run.trial120:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-21T00:14:48.789 DEBUG:teuthology.orchestra.run.trial120:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-21T00:14:48.851 INFO:teuthology.orchestra.run.trial120.stdout:check_obsoletes = 1 2026-02-21T00:14:48.852 DEBUG:teuthology.orchestra.run.trial120:> sudo yum clean all 2026-02-21T00:14:49.010 INFO:teuthology.orchestra.run.trial120.stdout:41 files removed 2026-02-21T00:14:49.025 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-radosgw 2026-02-21T00:14:49.866 INFO:teuthology.orchestra.run.trial120.stdout:ceph packages for x86_64 131 kB/s | 92 kB 00:00 2026-02-21T00:14:50.472 INFO:teuthology.orchestra.run.trial120.stdout:ceph noarch packages 41 kB/s | 24 kB 00:00 2026-02-21T00:14:51.051 INFO:teuthology.orchestra.run.trial120.stdout:ceph source packages 3.7 kB/s | 2.1 kB 00:00 2026-02-21T00:14:51.788 INFO:teuthology.orchestra.run.trial120.stdout:CentOS Stream 9 - BaseOS 12 MB/s | 8.9 MB 00:00 2026-02-21T00:14:55.558 INFO:teuthology.orchestra.run.trial120.stdout:CentOS Stream 9 - AppStream 8.3 MB/s | 27 MB 00:03 2026-02-21T00:14:58.848 INFO:teuthology.orchestra.run.trial120.stdout:CentOS Stream 9 - CRB 8.7 MB/s | 7.8 MB 00:00 2026-02-21T00:15:00.687 INFO:teuthology.orchestra.run.trial120.stdout:CentOS Stream 9 - Extras packages 18 kB/s | 20 kB 00:01 2026-02-21T00:15:01.046 INFO:teuthology.orchestra.run.trial120.stdout:Extra Packages for Enterprise Linux 69 MB/s | 20 MB 00:00 2026-02-21T00:15:04.272 INFO:teuthology.orchestra.run.trial120.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-21T00:15:05.463 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:15:05.464 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:15:05.464 INFO:teuthology.orchestra.run.trial120.stdout: Package Arch Version Repository Size 2026-02-21T00:15:05.464 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:15:05.464 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:15:05.465 INFO:teuthology.orchestra.run.trial120.stdout: ceph-radosgw x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 11 M 2026-02-21T00:15:05.465 INFO:teuthology.orchestra.run.trial120.stdout:Upgrading: 2026-02-21T00:15:05.465 INFO:teuthology.orchestra.run.trial120.stdout: librados2 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 3.3 M 2026-02-21T00:15:05.465 INFO:teuthology.orchestra.run.trial120.stdout: librbd1 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 2.7 M 2026-02-21T00:15:05.465 INFO:teuthology.orchestra.run.trial120.stdout:Installing dependencies: 2026-02-21T00:15:05.465 INFO:teuthology.orchestra.run.trial120.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-21T00:15:05.466 INFO:teuthology.orchestra.run.trial120.stdout: ceph-base x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 5.1 M 2026-02-21T00:15:05.466 INFO:teuthology.orchestra.run.trial120.stdout: ceph-common x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 21 M 2026-02-21T00:15:05.466 INFO:teuthology.orchestra.run.trial120.stdout: ceph-selinux x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 25 k 2026-02-21T00:15:05.466 INFO:teuthology.orchestra.run.trial120.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-21T00:15:05.466 INFO:teuthology.orchestra.run.trial120.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-21T00:15:05.466 INFO:teuthology.orchestra.run.trial120.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-21T00:15:05.467 INFO:teuthology.orchestra.run.trial120.stdout: libcephfs2 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 860 k 2026-02-21T00:15:05.467 INFO:teuthology.orchestra.run.trial120.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-21T00:15:05.467 INFO:teuthology.orchestra.run.trial120.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-21T00:15:05.467 INFO:teuthology.orchestra.run.trial120.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-21T00:15:05.467 INFO:teuthology.orchestra.run.trial120.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-21T00:15:05.467 INFO:teuthology.orchestra.run.trial120.stdout: libradosstriper1 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 263 k 2026-02-21T00:15:05.468 INFO:teuthology.orchestra.run.trial120.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-21T00:15:05.468 INFO:teuthology.orchestra.run.trial120.stdout: librgw2 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 5.6 M 2026-02-21T00:15:05.468 INFO:teuthology.orchestra.run.trial120.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-21T00:15:05.468 INFO:teuthology.orchestra.run.trial120.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-21T00:15:05.468 INFO:teuthology.orchestra.run.trial120.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-21T00:15:05.468 INFO:teuthology.orchestra.run.trial120.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-21T00:15:05.468 INFO:teuthology.orchestra.run.trial120.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-21T00:15:05.469 INFO:teuthology.orchestra.run.trial120.stdout: python3-ceph-argparse x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 45 k 2026-02-21T00:15:05.469 INFO:teuthology.orchestra.run.trial120.stdout: python3-ceph-common x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 185 k 2026-02-21T00:15:05.469 INFO:teuthology.orchestra.run.trial120.stdout: python3-cephfs x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 203 k 2026-02-21T00:15:05.469 INFO:teuthology.orchestra.run.trial120.stdout: python3-rados x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 324 k 2026-02-21T00:15:05.469 INFO:teuthology.orchestra.run.trial120.stdout: python3-rbd x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 305 k 2026-02-21T00:15:05.469 INFO:teuthology.orchestra.run.trial120.stdout: python3-rgw x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 100 k 2026-02-21T00:15:05.469 INFO:teuthology.orchestra.run.trial120.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-21T00:15:05.470 INFO:teuthology.orchestra.run.trial120.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-21T00:15:05.470 INFO:teuthology.orchestra.run.trial120.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-21T00:15:05.470 INFO:teuthology.orchestra.run.trial120.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-21T00:15:05.470 INFO:teuthology.orchestra.run.trial120.stdout:Installing weak dependencies: 2026-02-21T00:15:05.470 INFO:teuthology.orchestra.run.trial120.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-21T00:15:05.470 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:15:05.470 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:15:05.471 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:15:05.471 INFO:teuthology.orchestra.run.trial120.stdout:Install 32 Packages 2026-02-21T00:15:05.471 INFO:teuthology.orchestra.run.trial120.stdout:Upgrade 2 Packages 2026-02-21T00:15:05.471 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:15:05.472 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 60 M 2026-02-21T00:15:05.472 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:15:09.196 INFO:teuthology.orchestra.run.trial120.stdout:(1/34): ceph-base-20.3.0-5343.gbcdd429b.el9.x86 1.6 MB/s | 5.1 MB 00:03 2026-02-21T00:15:09.261 INFO:teuthology.orchestra.run.trial120.stdout:(2/34): ceph-selinux-20.3.0-5343.gbcdd429b.el9. 375 kB/s | 25 kB 00:00 2026-02-21T00:15:09.727 INFO:teuthology.orchestra.run.trial120.stdout:(3/34): libcephfs2-20.3.0-5343.gbcdd429b.el9.x8 1.8 MB/s | 860 kB 00:00 2026-02-21T00:15:09.863 INFO:teuthology.orchestra.run.trial120.stdout:(4/34): libradosstriper1-20.3.0-5343.gbcdd429b. 1.9 MB/s | 263 kB 00:00 2026-02-21T00:15:13.490 INFO:teuthology.orchestra.run.trial120.stdout:(5/34): librgw2-20.3.0-5343.gbcdd429b.el9.x86_6 1.6 MB/s | 5.6 MB 00:03 2026-02-21T00:15:13.560 INFO:teuthology.orchestra.run.trial120.stdout:(6/34): python3-ceph-argparse-20.3.0-5343.gbcdd 648 kB/s | 45 kB 00:00 2026-02-21T00:15:13.701 INFO:teuthology.orchestra.run.trial120.stdout:(7/34): python3-ceph-common-20.3.0-5343.gbcdd42 1.3 MB/s | 185 kB 00:00 2026-02-21T00:15:13.904 INFO:teuthology.orchestra.run.trial120.stdout:(8/34): python3-cephfs-20.3.0-5343.gbcdd429b.el 997 kB/s | 203 kB 00:00 2026-02-21T00:15:14.156 INFO:teuthology.orchestra.run.trial120.stdout:(9/34): python3-rados-20.3.0-5343.gbcdd429b.el9 1.3 MB/s | 324 kB 00:00 2026-02-21T00:15:14.356 INFO:teuthology.orchestra.run.trial120.stdout:(10/34): python3-rbd-20.3.0-5343.gbcdd429b.el9. 1.5 MB/s | 305 kB 00:00 2026-02-21T00:15:14.427 INFO:teuthology.orchestra.run.trial120.stdout:(11/34): python3-rgw-20.3.0-5343.gbcdd429b.el9. 1.4 MB/s | 100 kB 00:00 2026-02-21T00:15:14.872 INFO:teuthology.orchestra.run.trial120.stdout:(12/34): mailcap-2.1.49-5.el9.noarch.rpm 74 kB/s | 33 kB 00:00 2026-02-21T00:15:14.939 INFO:teuthology.orchestra.run.trial120.stdout:(13/34): ceph-radosgw-20.3.0-5343.gbcdd429b.el9 1.2 MB/s | 11 MB 00:08 2026-02-21T00:15:15.090 INFO:teuthology.orchestra.run.trial120.stdout:(14/34): pciutils-3.7.0-7.el9.x86_64.rpm 427 kB/s | 93 kB 00:00 2026-02-21T00:15:15.254 INFO:teuthology.orchestra.run.trial120.stdout:(15/34): boost-program-options-1.75.0-13.el9.x8 330 kB/s | 104 kB 00:00 2026-02-21T00:15:15.413 INFO:teuthology.orchestra.run.trial120.stdout:(16/34): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.0 MB/s | 160 kB 00:00 2026-02-21T00:15:15.571 INFO:teuthology.orchestra.run.trial120.stdout:(17/34): librabbitmq-0.11.0-7.el9.x86_64.rpm 287 kB/s | 45 kB 00:00 2026-02-21T00:15:15.808 INFO:teuthology.orchestra.run.trial120.stdout:(18/34): librdkafka-1.6.1-102.el9.x86_64.rpm 2.7 MB/s | 662 kB 00:00 2026-02-21T00:15:15.968 INFO:teuthology.orchestra.run.trial120.stdout:(19/34): lttng-ust-2.12.0-6.el9.x86_64.rpm 1.8 MB/s | 292 kB 00:00 2026-02-21T00:15:16.117 INFO:teuthology.orchestra.run.trial120.stdout:(20/34): libnbd-1.20.3-4.el9.x86_64.rpm 160 kB/s | 164 kB 00:01 2026-02-21T00:15:16.127 INFO:teuthology.orchestra.run.trial120.stdout:(21/34): qatlib-25.08.0-2.el9.x86_64.rpm 1.5 MB/s | 240 kB 00:00 2026-02-21T00:15:16.274 INFO:teuthology.orchestra.run.trial120.stdout:(22/34): qatlib-service-25.08.0-2.el9.x86_64.rp 235 kB/s | 37 kB 00:00 2026-02-21T00:15:16.443 INFO:teuthology.orchestra.run.trial120.stdout:(23/34): qatzip-libs-1.3.1-1.el9.x86_64.rpm 210 kB/s | 66 kB 00:00 2026-02-21T00:15:16.478 INFO:teuthology.orchestra.run.trial120.stdout:(24/34): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.5 MB/s | 308 kB 00:00 2026-02-21T00:15:16.504 INFO:teuthology.orchestra.run.trial120.stdout:(25/34): libarrow-doc-9.0.0-15.el9.noarch.rpm 946 kB/s | 25 kB 00:00 2026-02-21T00:15:16.530 INFO:teuthology.orchestra.run.trial120.stdout:(26/34): liboath-2.6.12-1.el9.x86_64.rpm 1.8 MB/s | 49 kB 00:00 2026-02-21T00:15:16.557 INFO:teuthology.orchestra.run.trial120.stdout:(27/34): libunwind-1.6.2-1.el9.x86_64.rpm 2.5 MB/s | 67 kB 00:00 2026-02-21T00:15:16.610 INFO:teuthology.orchestra.run.trial120.stdout:(28/34): parquet-libs-9.0.0-15.el9.x86_64.rpm 16 MB/s | 838 kB 00:00 2026-02-21T00:15:16.640 INFO:teuthology.orchestra.run.trial120.stdout:(29/34): re2-20211101-20.el9.x86_64.rpm 6.3 MB/s | 191 kB 00:00 2026-02-21T00:15:16.676 INFO:teuthology.orchestra.run.trial120.stdout:(30/34): libarrow-9.0.0-15.el9.x86_64.rpm 19 MB/s | 4.4 MB 00:00 2026-02-21T00:15:16.695 INFO:teuthology.orchestra.run.trial120.stdout:(31/34): thrift-0.15.0-4.el9.x86_64.rpm 28 MB/s | 1.6 MB 00:00 2026-02-21T00:15:18.359 INFO:teuthology.orchestra.run.trial120.stdout:(32/34): librados2-20.3.0-5343.gbcdd429b.el9.x8 2.0 MB/s | 3.3 MB 00:01 2026-02-21T00:15:18.526 INFO:teuthology.orchestra.run.trial120.stdout:(33/34): librbd1-20.3.0-5343.gbcdd429b.el9.x86_ 1.5 MB/s | 2.7 MB 00:01 2026-02-21T00:15:19.475 INFO:teuthology.orchestra.run.trial120.stdout:(34/34): ceph-common-20.3.0-5343.gbcdd429b.el9. 1.5 MB/s | 21 MB 00:13 2026-02-21T00:15:19.478 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:15:19.478 INFO:teuthology.orchestra.run.trial120.stdout:Total 4.3 MB/s | 60 MB 00:14 2026-02-21T00:15:19.568 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:15:19.586 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:15:19.586 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:15:19.830 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:15:19.830 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:15:20.109 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:15:20.249 INFO:teuthology.orchestra.run.trial120.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/36 2026-02-21T00:15:20.254 INFO:teuthology.orchestra.run.trial120.stdout: Upgrading : librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/36 2026-02-21T00:15:20.285 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/36 2026-02-21T00:15:20.322 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 3/36 2026-02-21T00:15:20.323 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 4/36 2026-02-21T00:15:20.334 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 4/36 2026-02-21T00:15:20.342 INFO:teuthology.orchestra.run.trial120.stdout: Installing : liboath-2.6.12-1.el9.x86_64 5/36 2026-02-21T00:15:20.345 INFO:teuthology.orchestra.run.trial120.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/36 2026-02-21T00:15:20.348 INFO:teuthology.orchestra.run.trial120.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/36 2026-02-21T00:15:20.352 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 8/36 2026-02-21T00:15:20.363 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-ceph-argparse-2:20.3.0-5343.gbcdd429b.el9. 9/36 2026-02-21T00:15:20.375 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 10/36 2026-02-21T00:15:20.376 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libradosstriper1-2:20.3.0-5343.gbcdd429b.el9.x86_6 11/36 2026-02-21T00:15:20.396 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5343.gbcdd429b.el9.x86_6 11/36 2026-02-21T00:15:20.400 INFO:teuthology.orchestra.run.trial120.stdout: Installing : thrift-0.15.0-4.el9.x86_64 12/36 2026-02-21T00:15:20.430 INFO:teuthology.orchestra.run.trial120.stdout: Installing : re2-1:20211101-20.el9.x86_64 13/36 2026-02-21T00:15:20.434 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 14/36 2026-02-21T00:15:20.440 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 15/36 2026-02-21T00:15:20.442 INFO:teuthology.orchestra.run.trial120.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 16/36 2026-02-21T00:15:20.447 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 17/36 2026-02-21T00:15:20.561 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/36 2026-02-21T00:15:20.562 INFO:teuthology.orchestra.run.trial120.stdout: Upgrading : librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 19/36 2026-02-21T00:15:20.588 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 19/36 2026-02-21T00:15:20.590 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 20/36 2026-02-21T00:15:20.597 INFO:teuthology.orchestra.run.trial120.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/36 2026-02-21T00:15:20.843 INFO:teuthology.orchestra.run.trial120.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/36 2026-02-21T00:15:20.845 INFO:teuthology.orchestra.run.trial120.stdout: Installing : librgw2-2:20.3.0-5343.gbcdd429b.el9.x86_64 23/36 2026-02-21T00:15:20.860 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: librgw2-2:20.3.0-5343.gbcdd429b.el9.x86_64 23/36 2026-02-21T00:15:20.865 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 24/36 2026-02-21T00:15:20.870 INFO:teuthology.orchestra.run.trial120.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/36 2026-02-21T00:15:20.872 INFO:teuthology.orchestra.run.trial120.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/36 2026-02-21T00:15:20.873 INFO:teuthology.orchestra.run.trial120.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-21T00:15:20.891 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-21T00:15:20.894 INFO:teuthology.orchestra.run.trial120.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/36 2026-02-21T00:15:20.929 INFO:teuthology.orchestra.run.trial120.stdout: Installing : mailcap-2.1.49-5.el9.noarch 29/36 2026-02-21T00:15:20.931 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-ceph-common-2:20.3.0-5343.gbcdd429b.el9.x8 30/36 2026-02-21T00:15:21.868 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 31/36 2026-02-21T00:15:21.870 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 31/36 2026-02-21T00:15:22.137 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 31/36 2026-02-21T00:15:22.140 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 32/36 2026-02-21T00:15:22.182 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 32/36 2026-02-21T00:15:22.183 INFO:teuthology.orchestra.run.trial120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-21T00:15:22.183 INFO:teuthology.orchestra.run.trial120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-21T00:15:22.183 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:15:22.186 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-selinux-2:20.3.0-5343.gbcdd429b.el9.x86_64 33/36 2026-02-21T00:15:26.708 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5343.gbcdd429b.el9.x86_64 33/36 2026-02-21T00:15:26.709 INFO:teuthology.orchestra.run.trial120.stdout:skipping the directory /sys 2026-02-21T00:15:26.709 INFO:teuthology.orchestra.run.trial120.stdout:skipping the directory /proc 2026-02-21T00:15:26.709 INFO:teuthology.orchestra.run.trial120.stdout:skipping the directory /mnt 2026-02-21T00:15:26.709 INFO:teuthology.orchestra.run.trial120.stdout:skipping the directory /var/tmp 2026-02-21T00:15:26.709 INFO:teuthology.orchestra.run.trial120.stdout:skipping the directory /home 2026-02-21T00:15:26.709 INFO:teuthology.orchestra.run.trial120.stdout:skipping the directory /root 2026-02-21T00:15:26.709 INFO:teuthology.orchestra.run.trial120.stdout:skipping the directory /tmp 2026-02-21T00:15:26.709 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:15:27.199 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 34/36 2026-02-21T00:15:27.222 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 34/36 2026-02-21T00:15:27.222 INFO:teuthology.orchestra.run.trial120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T00:15:27.222 INFO:teuthology.orchestra.run.trial120.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-21T00:15:27.222 INFO:teuthology.orchestra.run.trial120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-21T00:15:27.222 INFO:teuthology.orchestra.run.trial120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-21T00:15:27.222 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:15:27.223 INFO:teuthology.orchestra.run.trial120.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-21T00:15:27.235 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-21T00:15:27.235 INFO:teuthology.orchestra.run.trial120.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-21T00:15:27.290 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-21T00:15:27.290 INFO:teuthology.orchestra.run.trial120.stdout:Creating group 'qat' with GID 992. 2026-02-21T00:15:27.290 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:15:27.588 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/36 2026-02-21T00:15:27.589 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/36 2026-02-21T00:15:27.589 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 3/36 2026-02-21T00:15:27.589 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-selinux-2:20.3.0-5343.gbcdd429b.el9.x86_64 4/36 2026-02-21T00:15:27.589 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 5/36 2026-02-21T00:15:27.589 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libradosstriper1-2:20.3.0-5343.gbcdd429b.el9.x86_6 6/36 2026-02-21T00:15:27.589 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : librgw2-2:20.3.0-5343.gbcdd429b.el9.x86_64 7/36 2026-02-21T00:15:27.590 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5343.gbcdd429b.el9. 8/36 2026-02-21T00:15:27.590 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-ceph-common-2:20.3.0-5343.gbcdd429b.el9.x8 9/36 2026-02-21T00:15:27.590 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 10/36 2026-02-21T00:15:27.590 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 11/36 2026-02-21T00:15:27.590 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 12/36 2026-02-21T00:15:27.590 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 13/36 2026-02-21T00:15:27.590 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/36 2026-02-21T00:15:27.590 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 15/36 2026-02-21T00:15:27.590 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 16/36 2026-02-21T00:15:27.590 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 17/36 2026-02-21T00:15:27.590 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 18/36 2026-02-21T00:15:27.591 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/36 2026-02-21T00:15:27.591 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 20/36 2026-02-21T00:15:27.591 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 21/36 2026-02-21T00:15:27.591 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 22/36 2026-02-21T00:15:27.591 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 23/36 2026-02-21T00:15:27.591 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 24/36 2026-02-21T00:15:27.591 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 25/36 2026-02-21T00:15:27.591 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 26/36 2026-02-21T00:15:27.591 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 27/36 2026-02-21T00:15:27.591 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 28/36 2026-02-21T00:15:27.591 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 29/36 2026-02-21T00:15:27.592 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 30/36 2026-02-21T00:15:27.592 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : re2-1:20211101-20.el9.x86_64 31/36 2026-02-21T00:15:27.592 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 32/36 2026-02-21T00:15:27.592 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 33/36 2026-02-21T00:15:27.592 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 34/36 2026-02-21T00:15:27.592 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 35/36 2026-02-21T00:15:27.648 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 36/36 2026-02-21T00:15:27.648 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:15:27.648 INFO:teuthology.orchestra.run.trial120.stdout:Upgraded: 2026-02-21T00:15:27.649 INFO:teuthology.orchestra.run.trial120.stdout: librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:15:27.649 INFO:teuthology.orchestra.run.trial120.stdout: librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:15:27.649 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:15:27.649 INFO:teuthology.orchestra.run.trial120.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-21T00:15:27.649 INFO:teuthology.orchestra.run.trial120.stdout: ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:15:27.649 INFO:teuthology.orchestra.run.trial120.stdout: ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:15:27.649 INFO:teuthology.orchestra.run.trial120.stdout: ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:15:27.649 INFO:teuthology.orchestra.run.trial120.stdout: ceph-selinux-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:15:27.649 INFO:teuthology.orchestra.run.trial120.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-21T00:15:27.649 INFO:teuthology.orchestra.run.trial120.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-21T00:15:27.649 INFO:teuthology.orchestra.run.trial120.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-21T00:15:27.650 INFO:teuthology.orchestra.run.trial120.stdout: libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:15:27.650 INFO:teuthology.orchestra.run.trial120.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-21T00:15:27.650 INFO:teuthology.orchestra.run.trial120.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-21T00:15:27.650 INFO:teuthology.orchestra.run.trial120.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-21T00:15:27.650 INFO:teuthology.orchestra.run.trial120.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-21T00:15:27.650 INFO:teuthology.orchestra.run.trial120.stdout: libradosstriper1-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:15:27.650 INFO:teuthology.orchestra.run.trial120.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-21T00:15:27.650 INFO:teuthology.orchestra.run.trial120.stdout: librgw2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:15:27.650 INFO:teuthology.orchestra.run.trial120.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-21T00:15:27.650 INFO:teuthology.orchestra.run.trial120.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-21T00:15:27.650 INFO:teuthology.orchestra.run.trial120.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-21T00:15:27.650 INFO:teuthology.orchestra.run.trial120.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-21T00:15:27.651 INFO:teuthology.orchestra.run.trial120.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-21T00:15:27.651 INFO:teuthology.orchestra.run.trial120.stdout: python3-ceph-argparse-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:15:27.651 INFO:teuthology.orchestra.run.trial120.stdout: python3-ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:15:27.651 INFO:teuthology.orchestra.run.trial120.stdout: python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:15:27.651 INFO:teuthology.orchestra.run.trial120.stdout: python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:15:27.651 INFO:teuthology.orchestra.run.trial120.stdout: python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:15:27.651 INFO:teuthology.orchestra.run.trial120.stdout: python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:15:27.651 INFO:teuthology.orchestra.run.trial120.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-21T00:15:27.651 INFO:teuthology.orchestra.run.trial120.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-21T00:15:27.651 INFO:teuthology.orchestra.run.trial120.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-21T00:15:27.651 INFO:teuthology.orchestra.run.trial120.stdout: re2-1:20211101-20.el9.x86_64 2026-02-21T00:15:27.651 INFO:teuthology.orchestra.run.trial120.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-21T00:15:27.652 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:15:27.652 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:15:27.691 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-test 2026-02-21T00:15:28.001 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:23 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:15:28.083 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:15:28.084 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:15:28.084 INFO:teuthology.orchestra.run.trial120.stdout: Package Arch Version Repository Size 2026-02-21T00:15:28.084 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:15:28.084 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:15:28.084 INFO:teuthology.orchestra.run.trial120.stdout: ceph-test x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 126 M 2026-02-21T00:15:28.084 INFO:teuthology.orchestra.run.trial120.stdout:Installing dependencies: 2026-02-21T00:15:28.084 INFO:teuthology.orchestra.run.trial120.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-21T00:15:28.084 INFO:teuthology.orchestra.run.trial120.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-21T00:15:28.084 INFO:teuthology.orchestra.run.trial120.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-21T00:15:28.084 INFO:teuthology.orchestra.run.trial120.stdout: libcephsqlite x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 153 k 2026-02-21T00:15:28.084 INFO:teuthology.orchestra.run.trial120.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-21T00:15:28.084 INFO:teuthology.orchestra.run.trial120.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-21T00:15:28.084 INFO:teuthology.orchestra.run.trial120.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-21T00:15:28.085 INFO:teuthology.orchestra.run.trial120.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-21T00:15:28.085 INFO:teuthology.orchestra.run.trial120.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-21T00:15:28.085 INFO:teuthology.orchestra.run.trial120.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-21T00:15:28.085 INFO:teuthology.orchestra.run.trial120.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-21T00:15:28.085 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:15:28.085 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:15:28.085 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:15:28.085 INFO:teuthology.orchestra.run.trial120.stdout:Install 12 Packages 2026-02-21T00:15:28.085 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:15:28.086 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 131 M 2026-02-21T00:15:28.086 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 981 M 2026-02-21T00:15:28.086 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:15:29.018 INFO:teuthology.orchestra.run.trial120.stdout:(1/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 2.9 MB/s | 2.1 MB 00:00 2026-02-21T00:15:29.096 INFO:teuthology.orchestra.run.trial120.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 293 kB/s | 23 kB 00:00 2026-02-21T00:15:29.122 INFO:teuthology.orchestra.run.trial120.stdout:(3/12): libcephsqlite-20.3.0-5343.gbcdd429b.el9 183 kB/s | 153 kB 00:00 2026-02-21T00:15:29.147 INFO:teuthology.orchestra.run.trial120.stdout:(4/12): libasan8-15.2.1-7.el9.x86_64.rpm 20 MB/s | 525 kB 00:00 2026-02-21T00:15:29.153 INFO:teuthology.orchestra.run.trial120.stdout:(5/12): libubsan-11.5.0-14.el9.x86_64.rpm 28 MB/s | 176 kB 00:00 2026-02-21T00:15:29.159 INFO:teuthology.orchestra.run.trial120.stdout:(6/12): libxslt-1.1.34-12.el9.x86_64.rpm 38 MB/s | 233 kB 00:00 2026-02-21T00:15:29.175 INFO:teuthology.orchestra.run.trial120.stdout:(7/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 1.2 MB/s | 94 kB 00:00 2026-02-21T00:15:29.186 INFO:teuthology.orchestra.run.trial120.stdout:(8/12): protobuf-3.14.0-17.el9.x86_64.rpm 37 MB/s | 1.0 MB 00:00 2026-02-21T00:15:29.191 INFO:teuthology.orchestra.run.trial120.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 13 MB/s | 64 kB 00:00 2026-02-21T00:15:29.196 INFO:teuthology.orchestra.run.trial120.stdout:(10/12): socat-1.7.4.1-8.el9.x86_64.rpm 14 MB/s | 303 kB 00:00 2026-02-21T00:15:29.327 INFO:teuthology.orchestra.run.trial120.stdout:(11/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 900 kB/s | 122 kB 00:00 2026-02-21T00:16:22.242 INFO:teuthology.orchestra.run.trial120.stdout:(12/12): ceph-test-20.3.0-5343.gbcdd429b.el9.x8 2.3 MB/s | 126 MB 00:53 2026-02-21T00:16:22.243 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:16:22.243 INFO:teuthology.orchestra.run.trial120.stdout:Total 2.4 MB/s | 131 MB 00:54 2026-02-21T00:16:22.311 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:16:22.322 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:16:22.322 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:16:22.753 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:16:22.753 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:16:23.209 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:16:23.218 INFO:teuthology.orchestra.run.trial120.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-21T00:16:23.226 INFO:teuthology.orchestra.run.trial120.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-21T00:16:23.231 INFO:teuthology.orchestra.run.trial120.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-21T00:16:23.235 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-21T00:16:23.238 INFO:teuthology.orchestra.run.trial120.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-21T00:16:23.242 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-21T00:16:23.243 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-21T00:16:23.258 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-21T00:16:23.261 INFO:teuthology.orchestra.run.trial120.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-21T00:16:23.271 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-21T00:16:23.280 INFO:teuthology.orchestra.run.trial120.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-21T00:16:23.280 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libcephsqlite-2:20.3.0-5343.gbcdd429b.el9.x86_64 11/12 2026-02-21T00:16:30.178 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5343.gbcdd429b.el9.x86_64 11/12 2026-02-21T00:16:30.197 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 12/12 2026-02-21T00:16:30.971 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 12/12 2026-02-21T00:16:30.971 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/12 2026-02-21T00:16:30.971 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libcephsqlite-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/12 2026-02-21T00:16:30.971 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-21T00:16:30.971 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-21T00:16:30.971 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-21T00:16:30.971 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-21T00:16:30.971 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-21T00:16:30.971 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-21T00:16:30.971 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-21T00:16:30.971 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-21T00:16:30.971 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout: ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout: libcephsqlite-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:16:31.025 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:16:31.068 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph 2026-02-21T00:16:31.372 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:01:27 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:16:31.456 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:16:31.457 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:16:31.457 INFO:teuthology.orchestra.run.trial120.stdout: Package Arch Version Repository Size 2026-02-21T00:16:31.457 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:16:31.457 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:16:31.458 INFO:teuthology.orchestra.run.trial120.stdout: ceph x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 6.5 k 2026-02-21T00:16:31.458 INFO:teuthology.orchestra.run.trial120.stdout:Installing dependencies: 2026-02-21T00:16:31.458 INFO:teuthology.orchestra.run.trial120.stdout: ceph-mds x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 2.0 M 2026-02-21T00:16:31.458 INFO:teuthology.orchestra.run.trial120.stdout: ceph-mgr x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 937 k 2026-02-21T00:16:31.458 INFO:teuthology.orchestra.run.trial120.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 297 k 2026-02-21T00:16:31.458 INFO:teuthology.orchestra.run.trial120.stdout: ceph-mon x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 4.9 M 2026-02-21T00:16:31.458 INFO:teuthology.orchestra.run.trial120.stdout: ceph-osd x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 163 k 2026-02-21T00:16:31.458 INFO:teuthology.orchestra.run.trial120.stdout: ceph-osd-classic x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 15 M 2026-02-21T00:16:31.458 INFO:teuthology.orchestra.run.trial120.stdout: ceph-osd-crimson x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 88 M 2026-02-21T00:16:31.458 INFO:teuthology.orchestra.run.trial120.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-21T00:16:31.458 INFO:teuthology.orchestra.run.trial120.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-21T00:16:31.458 INFO:teuthology.orchestra.run.trial120.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-21T00:16:31.458 INFO:teuthology.orchestra.run.trial120.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-21T00:16:31.459 INFO:teuthology.orchestra.run.trial120.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-21T00:16:31.459 INFO:teuthology.orchestra.run.trial120.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-21T00:16:31.459 INFO:teuthology.orchestra.run.trial120.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-21T00:16:31.459 INFO:teuthology.orchestra.run.trial120.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-21T00:16:31.459 INFO:teuthology.orchestra.run.trial120.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-21T00:16:31.459 INFO:teuthology.orchestra.run.trial120.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-21T00:16:31.459 INFO:teuthology.orchestra.run.trial120.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-21T00:16:31.459 INFO:teuthology.orchestra.run.trial120.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-21T00:16:31.459 INFO:teuthology.orchestra.run.trial120.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-21T00:16:31.459 INFO:teuthology.orchestra.run.trial120.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-21T00:16:31.459 INFO:teuthology.orchestra.run.trial120.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-21T00:16:31.459 INFO:teuthology.orchestra.run.trial120.stdout: python3-jaraco-collections 2026-02-21T00:16:31.459 INFO:teuthology.orchestra.run.trial120.stdout: noarch 3.0.0-8.el9 epel 23 k 2026-02-21T00:16:31.460 INFO:teuthology.orchestra.run.trial120.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-21T00:16:31.460 INFO:teuthology.orchestra.run.trial120.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-21T00:16:31.460 INFO:teuthology.orchestra.run.trial120.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-21T00:16:31.460 INFO:teuthology.orchestra.run.trial120.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-21T00:16:31.460 INFO:teuthology.orchestra.run.trial120.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-21T00:16:31.460 INFO:teuthology.orchestra.run.trial120.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-21T00:16:31.460 INFO:teuthology.orchestra.run.trial120.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-21T00:16:31.460 INFO:teuthology.orchestra.run.trial120.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-21T00:16:31.460 INFO:teuthology.orchestra.run.trial120.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-21T00:16:31.460 INFO:teuthology.orchestra.run.trial120.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-21T00:16:31.460 INFO:teuthology.orchestra.run.trial120.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-21T00:16:31.460 INFO:teuthology.orchestra.run.trial120.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-21T00:16:31.460 INFO:teuthology.orchestra.run.trial120.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-21T00:16:31.460 INFO:teuthology.orchestra.run.trial120.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-21T00:16:31.461 INFO:teuthology.orchestra.run.trial120.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-21T00:16:31.461 INFO:teuthology.orchestra.run.trial120.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-21T00:16:31.461 INFO:teuthology.orchestra.run.trial120.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-21T00:16:31.461 INFO:teuthology.orchestra.run.trial120.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-21T00:16:31.461 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:16:31.461 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:16:31.461 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:16:31.461 INFO:teuthology.orchestra.run.trial120.stdout:Install 42 Packages 2026-02-21T00:16:31.461 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:16:31.462 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 115 M 2026-02-21T00:16:31.462 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 974 M 2026-02-21T00:16:31.462 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:16:32.460 INFO:teuthology.orchestra.run.trial120.stdout:(1/42): ceph-20.3.0-5343.gbcdd429b.el9.x86_64.r 24 kB/s | 6.5 kB 00:00 2026-02-21T00:16:33.498 INFO:teuthology.orchestra.run.trial120.stdout:(2/42): ceph-mgr-20.3.0-5343.gbcdd429b.el9.x86_ 717 kB/s | 937 kB 00:01 2026-02-21T00:16:33.691 INFO:teuthology.orchestra.run.trial120.stdout:(3/42): ceph-osd-20.3.0-5343.gbcdd429b.el9.x86_ 844 kB/s | 163 kB 00:00 2026-02-21T00:16:34.096 INFO:teuthology.orchestra.run.trial120.stdout:(4/42): ceph-mds-20.3.0-5343.gbcdd429b.el9.x86_ 1.1 MB/s | 2.0 MB 00:01 2026-02-21T00:16:39.075 INFO:teuthology.orchestra.run.trial120.stdout:(5/42): ceph-mon-20.3.0-5343.gbcdd429b.el9.x86_ 754 kB/s | 4.9 MB 00:06 2026-02-21T00:16:39.533 INFO:teuthology.orchestra.run.trial120.stdout:(6/42): ceph-mgr-modules-core-20.3.0-5343.gbcdd 647 kB/s | 297 kB 00:00 2026-02-21T00:16:39.601 INFO:teuthology.orchestra.run.trial120.stdout:(7/42): ledmon-libs-1.1.0-3.el9.x86_64.rpm 601 kB/s | 40 kB 00:00 2026-02-21T00:16:39.646 INFO:teuthology.orchestra.run.trial120.stdout:(8/42): libconfig-1.7.2-9.el9.x86_64.rpm 1.6 MB/s | 72 kB 00:00 2026-02-21T00:16:39.694 INFO:teuthology.orchestra.run.trial120.stdout:(9/42): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.2 MB/s | 253 kB 00:00 2026-02-21T00:16:39.790 INFO:teuthology.orchestra.run.trial120.stdout:(10/42): python3-cryptography-36.0.1-5.el9.x86_ 13 MB/s | 1.2 MB 00:00 2026-02-21T00:16:39.840 INFO:teuthology.orchestra.run.trial120.stdout:(11/42): python3-ply-3.11-14.el9.noarch.rpm 2.1 MB/s | 106 kB 00:00 2026-02-21T00:16:39.866 INFO:teuthology.orchestra.run.trial120.stdout:(12/42): python3-pycparser-2.20-6.el9.noarch.rp 5.1 MB/s | 135 kB 00:00 2026-02-21T00:16:39.894 INFO:teuthology.orchestra.run.trial120.stdout:(13/42): python3-pyparsing-2.4.7-9.el9.noarch.r 5.2 MB/s | 150 kB 00:00 2026-02-21T00:16:39.920 INFO:teuthology.orchestra.run.trial120.stdout:(14/42): python3-requests-2.25.1-10.el9.noarch. 4.8 MB/s | 126 kB 00:00 2026-02-21T00:16:39.946 INFO:teuthology.orchestra.run.trial120.stdout:(15/42): python3-urllib3-1.26.5-7.el9.noarch.rp 8.1 MB/s | 218 kB 00:00 2026-02-21T00:16:39.973 INFO:teuthology.orchestra.run.trial120.stdout:(16/42): unzip-6.0-59.el9.x86_64.rpm 6.6 MB/s | 182 kB 00:00 2026-02-21T00:16:39.998 INFO:teuthology.orchestra.run.trial120.stdout:(17/42): zip-3.0-35.el9.x86_64.rpm 10 MB/s | 266 kB 00:00 2026-02-21T00:16:40.058 INFO:teuthology.orchestra.run.trial120.stdout:(18/42): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 4.0 MB/s | 246 kB 00:00 2026-02-21T00:16:40.073 INFO:teuthology.orchestra.run.trial120.stdout:(19/42): lua-5.4.4-4.el9.x86_64.rpm 12 MB/s | 188 kB 00:00 2026-02-21T00:16:40.086 INFO:teuthology.orchestra.run.trial120.stdout:(20/42): python3-libstoragemgmt-1.10.1-1.el9.x8 13 MB/s | 177 kB 00:00 2026-02-21T00:16:40.097 INFO:teuthology.orchestra.run.trial120.stdout:(21/42): python3-packaging-20.9-5.el9.noarch.rp 6.7 MB/s | 77 kB 00:00 2026-02-21T00:16:40.107 INFO:teuthology.orchestra.run.trial120.stdout:(22/42): python3-toml-0.10.2-6.el9.noarch.rpm 4.1 MB/s | 42 kB 00:00 2026-02-21T00:16:40.117 INFO:teuthology.orchestra.run.trial120.stdout:(23/42): lua-devel-5.4.4-4.el9.x86_64.rpm 2.3 MB/s | 22 kB 00:00 2026-02-21T00:16:40.674 INFO:teuthology.orchestra.run.trial120.stdout:(24/42): luarocks-3.9.2-5.el9.noarch.rpm 272 kB/s | 151 kB 00:00 2026-02-21T00:16:40.760 INFO:teuthology.orchestra.run.trial120.stdout:(25/42): python3-autocommand-2.2.2-8.el9.noarch 343 kB/s | 29 kB 00:00 2026-02-21T00:16:40.841 INFO:teuthology.orchestra.run.trial120.stdout:(26/42): python3-backports-tarfile-1.2.0-1.el9. 747 kB/s | 60 kB 00:00 2026-02-21T00:16:40.938 INFO:teuthology.orchestra.run.trial120.stdout:(27/42): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 450 kB/s | 43 kB 00:00 2026-02-21T00:16:41.174 INFO:teuthology.orchestra.run.trial120.stdout:(28/42): python3-cheroot-10.0.1-4.el9.noarch.rp 734 kB/s | 173 kB 00:00 2026-02-21T00:16:41.667 INFO:teuthology.orchestra.run.trial120.stdout:(29/42): python3-cherrypy-18.6.1-2.el9.noarch.r 727 kB/s | 358 kB 00:00 2026-02-21T00:16:41.747 INFO:teuthology.orchestra.run.trial120.stdout:(30/42): python3-jaraco-8.2.1-3.el9.noarch.rpm 134 kB/s | 11 kB 00:00 2026-02-21T00:16:41.839 INFO:teuthology.orchestra.run.trial120.stdout:(31/42): python3-jaraco-classes-3.2.1-5.el9.noa 192 kB/s | 18 kB 00:00 2026-02-21T00:16:41.919 INFO:teuthology.orchestra.run.trial120.stdout:(32/42): python3-jaraco-collections-3.0.0-8.el9 289 kB/s | 23 kB 00:00 2026-02-21T00:16:41.999 INFO:teuthology.orchestra.run.trial120.stdout:(33/42): python3-jaraco-context-6.0.1-3.el9.noa 247 kB/s | 20 kB 00:00 2026-02-21T00:16:42.085 INFO:teuthology.orchestra.run.trial120.stdout:(34/42): python3-jaraco-functools-3.5.0-2.el9.n 225 kB/s | 19 kB 00:00 2026-02-21T00:16:42.164 INFO:teuthology.orchestra.run.trial120.stdout:(35/42): python3-jaraco-text-4.0.0-2.el9.noarch 333 kB/s | 26 kB 00:00 2026-02-21T00:16:42.210 INFO:teuthology.orchestra.run.trial120.stdout:(36/42): ceph-osd-classic-20.3.0-5343.gbcdd429b 1.7 MB/s | 15 MB 00:08 2026-02-21T00:16:42.330 INFO:teuthology.orchestra.run.trial120.stdout:(37/42): python3-more-itertools-8.12.0-2.el9.no 476 kB/s | 79 kB 00:00 2026-02-21T00:16:42.488 INFO:teuthology.orchestra.run.trial120.stdout:(38/42): python3-pyOpenSSL-21.0.0-1.el9.noarch. 570 kB/s | 90 kB 00:00 2026-02-21T00:16:42.574 INFO:teuthology.orchestra.run.trial120.stdout:(39/42): python3-tempora-5.0.0-2.el9.noarch.rpm 415 kB/s | 36 kB 00:00 2026-02-21T00:16:42.609 INFO:teuthology.orchestra.run.trial120.stdout:(40/42): python3-portend-3.1.0-2.el9.noarch.rpm 41 kB/s | 16 kB 00:00 2026-02-21T00:16:42.664 INFO:teuthology.orchestra.run.trial120.stdout:(41/42): python3-zc-lockfile-2.0-10.el9.noarch. 222 kB/s | 20 kB 00:00 2026-02-21T00:17:46.167 INFO:teuthology.orchestra.run.trial120.stdout:(42/42): ceph-osd-crimson-20.3.0-5343.gbcdd429b 1.2 MB/s | 88 MB 01:12 2026-02-21T00:17:46.170 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:17:46.170 INFO:teuthology.orchestra.run.trial120.stdout:Total 1.5 MB/s | 115 MB 01:14 2026-02-21T00:17:46.293 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:17:46.313 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:17:46.313 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:17:46.710 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:17:46.711 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:17:47.150 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:17:47.152 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/42 2026-02-21T00:17:47.156 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/42 2026-02-21T00:17:47.161 INFO:teuthology.orchestra.run.trial120.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/42 2026-02-21T00:17:47.163 INFO:teuthology.orchestra.run.trial120.stdout: Installing : unzip-6.0-59.el9.x86_64 4/42 2026-02-21T00:17:47.169 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 5/42 2026-02-21T00:17:47.192 INFO:teuthology.orchestra.run.trial120.stdout: Installing : zip-3.0-35.el9.x86_64 6/42 2026-02-21T00:17:47.196 INFO:teuthology.orchestra.run.trial120.stdout: Installing : luarocks-3.9.2-5.el9.noarch 7/42 2026-02-21T00:17:47.201 INFO:teuthology.orchestra.run.trial120.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 8/42 2026-02-21T00:17:47.205 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 9/42 2026-02-21T00:17:47.211 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 10/42 2026-02-21T00:17:47.215 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 11/42 2026-02-21T00:17:47.221 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 12/42 2026-02-21T00:17:47.226 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 13/42 2026-02-21T00:17:47.230 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 14/42 2026-02-21T00:17:47.236 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 15/42 2026-02-21T00:17:47.239 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 16/42 2026-02-21T00:17:47.246 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 17/42 2026-02-21T00:17:47.249 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 18/42 2026-02-21T00:17:47.272 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 19/42 2026-02-21T00:17:47.285 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 20/42 2026-02-21T00:17:47.290 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 21/42 2026-02-21T00:17:47.300 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 22/42 2026-02-21T00:17:47.307 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-21T00:17:47.320 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/42 2026-02-21T00:17:47.336 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/42 2026-02-21T00:17:47.411 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/42 2026-02-21T00:17:47.419 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 27/42 2026-02-21T00:17:47.442 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/42 2026-02-21T00:17:47.471 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 29/42 2026-02-21T00:17:47.476 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/42 2026-02-21T00:17:47.528 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/42 2026-02-21T00:17:47.569 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5343.gbcdd429b.el9. 32/42 2026-02-21T00:17:47.570 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 33/42 2026-02-21T00:17:47.593 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 33/42 2026-02-21T00:17:47.593 INFO:teuthology.orchestra.run.trial120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T00:17:47.593 INFO:teuthology.orchestra.run.trial120.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-21T00:17:47.593 INFO:teuthology.orchestra.run.trial120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-21T00:17:47.593 INFO:teuthology.orchestra.run.trial120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-21T00:17:47.593 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:47.598 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 34/42 2026-02-21T00:17:47.614 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-21T00:17:47.615 INFO:teuthology.orchestra.run.trial120.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-21T00:17:47.615 INFO:teuthology.orchestra.run.trial120.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-21T00:17:47.615 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:47.622 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-21T00:17:47.644 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-21T00:17:47.644 INFO:teuthology.orchestra.run.trial120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-21T00:17:47.645 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:48.256 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 36/42 2026-02-21T00:17:53.753 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_6 37/42 2026-02-21T00:17:53.763 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86_6 38/42 2026-02-21T00:17:53.764 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 39/42 2026-02-21T00:17:53.788 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 39/42 2026-02-21T00:17:53.788 INFO:teuthology.orchestra.run.trial120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T00:17:53.788 INFO:teuthology.orchestra.run.trial120.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-21T00:17:53.788 INFO:teuthology.orchestra.run.trial120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-21T00:17:53.789 INFO:teuthology.orchestra.run.trial120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-21T00:17:53.789 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:53.996 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-mon-2:20.3.0-5343.gbcdd429b.el9.x86_64 40/42 2026-02-21T00:17:54.017 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-mon-2:20.3.0-5343.gbcdd429b.el9.x86_64 40/42 2026-02-21T00:17:54.017 INFO:teuthology.orchestra.run.trial120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T00:17:54.017 INFO:teuthology.orchestra.run.trial120.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-21T00:17:54.017 INFO:teuthology.orchestra.run.trial120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-21T00:17:54.017 INFO:teuthology.orchestra.run.trial120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-21T00:17:54.017 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:54.101 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-mds-2:20.3.0-5343.gbcdd429b.el9.x86_64 41/42 2026-02-21T00:17:54.121 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-mds-2:20.3.0-5343.gbcdd429b.el9.x86_64 41/42 2026-02-21T00:17:54.121 INFO:teuthology.orchestra.run.trial120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T00:17:54.121 INFO:teuthology.orchestra.run.trial120.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-21T00:17:54.121 INFO:teuthology.orchestra.run.trial120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-21T00:17:54.121 INFO:teuthology.orchestra.run.trial120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-21T00:17:54.121 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:54.122 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 42/42 2026-02-21T00:17:54.128 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_6 42/42 2026-02-21T00:17:54.150 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86_6 42/42 2026-02-21T00:17:55.080 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 42/42 2026-02-21T00:17:55.081 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/42 2026-02-21T00:17:55.081 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-mds-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/42 2026-02-21T00:17:55.081 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 3/42 2026-02-21T00:17:55.081 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-mon-2:20.3.0-5343.gbcdd429b.el9.x86_64 4/42 2026-02-21T00:17:55.081 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 5/42 2026-02-21T00:17:55.081 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_6 6/42 2026-02-21T00:17:55.081 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86_6 7/42 2026-02-21T00:17:55.081 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5343.gbcdd429b.el9. 8/42 2026-02-21T00:17:55.081 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 9/42 2026-02-21T00:17:55.081 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 10/42 2026-02-21T00:17:55.082 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/42 2026-02-21T00:17:55.082 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 12/42 2026-02-21T00:17:55.082 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/42 2026-02-21T00:17:55.082 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/42 2026-02-21T00:17:55.082 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 15/42 2026-02-21T00:17:55.082 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 16/42 2026-02-21T00:17:55.082 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 17/42 2026-02-21T00:17:55.082 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : unzip-6.0-59.el9.x86_64 18/42 2026-02-21T00:17:55.082 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : zip-3.0-35.el9.x86_64 19/42 2026-02-21T00:17:55.082 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 20/42 2026-02-21T00:17:55.082 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : lua-5.4.4-4.el9.x86_64 21/42 2026-02-21T00:17:55.082 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 22/42 2026-02-21T00:17:55.083 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-21T00:17:55.083 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 24/42 2026-02-21T00:17:55.083 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 25/42 2026-02-21T00:17:55.083 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 26/42 2026-02-21T00:17:55.083 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 27/42 2026-02-21T00:17:55.083 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 28/42 2026-02-21T00:17:55.083 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 29/42 2026-02-21T00:17:55.083 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 30/42 2026-02-21T00:17:55.083 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 31/42 2026-02-21T00:17:55.083 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 32/42 2026-02-21T00:17:55.083 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 33/42 2026-02-21T00:17:55.083 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 34/42 2026-02-21T00:17:55.083 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 35/42 2026-02-21T00:17:55.084 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 36/42 2026-02-21T00:17:55.084 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 37/42 2026-02-21T00:17:55.084 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 38/42 2026-02-21T00:17:55.084 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 39/42 2026-02-21T00:17:55.084 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 40/42 2026-02-21T00:17:55.084 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 41/42 2026-02-21T00:17:55.136 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 42/42 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: ceph-mds-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: ceph-mgr-modules-core-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: ceph-mon-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: lua-5.4.4-4.el9.x86_64 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-21T00:17:55.140 INFO:teuthology.orchestra.run.trial120.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: python3-ply-3.11-14.el9.noarch 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: unzip-6.0-59.el9.x86_64 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: zip-3.0-35.el9.x86_64 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:55.141 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:17:55.179 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-base 2026-02-21T00:17:55.487 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:02:51 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:17:55.551 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:17:55.568 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:17:55.569 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:17:55.569 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:17:55.590 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install cephadm 2026-02-21T00:17:55.891 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:02:51 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:17:55.971 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:17:55.972 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:17:55.972 INFO:teuthology.orchestra.run.trial120.stdout: Package Arch Version Repository Size 2026-02-21T00:17:55.972 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:17:55.972 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:17:55.972 INFO:teuthology.orchestra.run.trial120.stdout: cephadm noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 1.0 M 2026-02-21T00:17:55.972 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:55.972 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:17:55.972 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:17:55.972 INFO:teuthology.orchestra.run.trial120.stdout:Install 1 Package 2026-02-21T00:17:55.972 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:55.972 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 1.0 M 2026-02-21T00:17:55.972 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 1.0 M 2026-02-21T00:17:55.972 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:17:56.656 INFO:teuthology.orchestra.run.trial120.stdout:cephadm-20.3.0-5343.gbcdd429b.el9.noarch.rpm 1.5 MB/s | 1.0 MB 00:00 2026-02-21T00:17:56.656 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:17:56.657 INFO:teuthology.orchestra.run.trial120.stdout:Total 1.5 MB/s | 1.0 MB 00:00 2026-02-21T00:17:56.657 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:17:56.660 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:17:56.660 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:17:56.664 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:17:56.664 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:17:56.682 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:17:56.741 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 1/1 2026-02-21T00:17:56.755 INFO:teuthology.orchestra.run.trial120.stdout: Installing : cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 1/1 2026-02-21T00:17:56.817 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 1/1 2026-02-21T00:17:56.871 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 1/1 2026-02-21T00:17:56.871 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:56.872 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:17:56.872 INFO:teuthology.orchestra.run.trial120.stdout: cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-21T00:17:56.872 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:56.872 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:17:56.899 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-immutable-object-cache 2026-02-21T00:17:57.211 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:02:53 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:17:57.292 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:17:57.293 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:17:57.293 INFO:teuthology.orchestra.run.trial120.stdout: Package Arch Version Repo Size 2026-02-21T00:17:57.293 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:17:57.293 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:17:57.293 INFO:teuthology.orchestra.run.trial120.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 149 k 2026-02-21T00:17:57.293 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:57.293 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:17:57.293 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:17:57.293 INFO:teuthology.orchestra.run.trial120.stdout:Install 1 Package 2026-02-21T00:17:57.293 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:57.293 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 149 k 2026-02-21T00:17:57.294 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 455 k 2026-02-21T00:17:57.294 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:17:58.061 INFO:teuthology.orchestra.run.trial120.stdout:ceph-immutable-object-cache-20.3.0-5343.gbcdd42 195 kB/s | 149 kB 00:00 2026-02-21T00:17:58.063 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:17:58.063 INFO:teuthology.orchestra.run.trial120.stdout:Total 194 kB/s | 149 kB 00:00 2026-02-21T00:17:58.064 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:17:58.068 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:17:58.068 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:17:58.094 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:17:58.095 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:17:58.143 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:17:58.145 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b. 1/1 2026-02-21T00:17:58.167 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b. 1/1 2026-02-21T00:17:58.167 INFO:teuthology.orchestra.run.trial120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T00:17:58.167 INFO:teuthology.orchestra.run.trial120.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-21T00:17:58.167 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:58.475 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b. 1/1 2026-02-21T00:17:58.475 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:58.475 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:17:58.475 INFO:teuthology.orchestra.run.trial120.stdout: ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:17:58.475 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:58.475 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:17:58.502 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-mgr 2026-02-21T00:17:58.815 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:02:54 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:17:58.878 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:17:58.896 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:17:58.896 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:17:58.896 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:17:58.917 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-mgr-dashboard 2026-02-21T00:17:59.222 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:02:55 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:17:59.305 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:17:59.305 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:17:59.305 INFO:teuthology.orchestra.run.trial120.stdout: Package Arch Version Repository Size 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 51 M 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout:Installing dependencies: 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 57 k 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 17 k 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout:Install 13 Packages 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 55 M 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 262 M 2026-02-21T00:17:59.306 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:18:00.088 INFO:teuthology.orchestra.run.trial120.stdout:(1/13): ceph-prometheus-alerts-20.3.0-5343.gbcd 50 kB/s | 17 kB 00:00 2026-02-21T00:18:00.103 INFO:teuthology.orchestra.run.trial120.stdout:(2/13): python3-jmespath-1.0.1-1.el9.noarch.rpm 3.2 MB/s | 48 kB 00:00 2026-02-21T00:18:00.116 INFO:teuthology.orchestra.run.trial120.stdout:(3/13): python3-protobuf-3.14.0-17.el9.noarch.r 20 MB/s | 267 kB 00:00 2026-02-21T00:18:00.158 INFO:teuthology.orchestra.run.trial120.stdout:(4/13): ceph-grafana-dashboards-20.3.0-5343.gbc 137 kB/s | 57 kB 00:00 2026-02-21T00:18:00.316 INFO:teuthology.orchestra.run.trial120.stdout:(5/13): protobuf-compiler-3.14.0-17.el9.x86_64. 4.2 MB/s | 862 kB 00:00 2026-02-21T00:18:00.336 INFO:teuthology.orchestra.run.trial120.stdout:(6/13): abseil-cpp-20211102.0-4.el9.x86_64.rpm 3.0 MB/s | 551 kB 00:00 2026-02-21T00:18:00.400 INFO:teuthology.orchestra.run.trial120.stdout:(7/13): python3-grpcio-1.46.7-10.el9.x86_64.rpm 33 MB/s | 2.0 MB 00:00 2026-02-21T00:18:00.420 INFO:teuthology.orchestra.run.trial120.stdout:(8/13): grpc-data-1.46.7-10.el9.noarch.rpm 188 kB/s | 19 kB 00:00 2026-02-21T00:18:00.421 INFO:teuthology.orchestra.run.trial120.stdout:(9/13): python3-grpcio-tools-1.46.7-10.el9.x86_ 7.0 MB/s | 144 kB 00:00 2026-02-21T00:18:00.440 INFO:teuthology.orchestra.run.trial120.stdout:(10/13): python3-repoze-lru-0.7-16.el9.noarch.r 1.5 MB/s | 31 kB 00:00 2026-02-21T00:18:00.441 INFO:teuthology.orchestra.run.trial120.stdout:(11/13): python3-routes-2.5.1-5.el9.noarch.rpm 9.1 MB/s | 188 kB 00:00 2026-02-21T00:18:00.460 INFO:teuthology.orchestra.run.trial120.stdout:(12/13): python3-xmltodict-0.12.0-15.el9.noarch 1.1 MB/s | 22 kB 00:00 2026-02-21T00:18:27.919 INFO:teuthology.orchestra.run.trial120.stdout:(13/13): ceph-mgr-dashboard-20.3.0-5343.gbcdd42 1.8 MB/s | 51 MB 00:28 2026-02-21T00:18:27.921 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:18:27.922 INFO:teuthology.orchestra.run.trial120.stdout:Total 1.9 MB/s | 55 MB 00:28 2026-02-21T00:18:27.947 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:18:27.958 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:18:27.958 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:18:28.170 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:18:28.170 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:18:28.398 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:18:28.404 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 1/13 2026-02-21T00:18:28.415 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 2/13 2026-02-21T00:18:28.417 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 3/13 2026-02-21T00:18:28.446 INFO:teuthology.orchestra.run.trial120.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 4/13 2026-02-21T00:18:28.492 INFO:teuthology.orchestra.run.trial120.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 5/13 2026-02-21T00:18:28.500 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 6/13 2026-02-21T00:18:28.529 INFO:teuthology.orchestra.run.trial120.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 7/13 2026-02-21T00:18:28.539 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 8/13 2026-02-21T00:18:28.547 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/13 2026-02-21T00:18:28.549 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 10/13 2026-02-21T00:18:28.559 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5343.gbcdd429b.el9 11/13 2026-02-21T00:18:31.583 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5343.gbcdd429b.el 12/13 2026-02-21T00:18:31.590 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noa 13/13 2026-02-21T00:18:31.844 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noa 13/13 2026-02-21T00:18:31.844 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5343.gbcdd429b.el 1/13 2026-02-21T00:18:31.844 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noa 2/13 2026-02-21T00:18:31.844 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5343.gbcdd429b.el9 3/13 2026-02-21T00:18:31.845 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 4/13 2026-02-21T00:18:31.845 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 5/13 2026-02-21T00:18:31.845 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 6/13 2026-02-21T00:18:31.845 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 7/13 2026-02-21T00:18:31.845 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/13 2026-02-21T00:18:31.845 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/13 2026-02-21T00:18:31.845 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/13 2026-02-21T00:18:31.845 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/13 2026-02-21T00:18:31.845 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/13 2026-02-21T00:18:31.899 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 13/13 2026-02-21T00:18:31.899 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:31.899 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:18:31.899 INFO:teuthology.orchestra.run.trial120.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-21T00:18:31.899 INFO:teuthology.orchestra.run.trial120.stdout: ceph-grafana-dashboards-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-21T00:18:31.899 INFO:teuthology.orchestra.run.trial120.stdout: ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-21T00:18:31.899 INFO:teuthology.orchestra.run.trial120.stdout: ceph-prometheus-alerts-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-21T00:18:31.899 INFO:teuthology.orchestra.run.trial120.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-21T00:18:31.899 INFO:teuthology.orchestra.run.trial120.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-21T00:18:31.899 INFO:teuthology.orchestra.run.trial120.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-21T00:18:31.900 INFO:teuthology.orchestra.run.trial120.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-21T00:18:31.900 INFO:teuthology.orchestra.run.trial120.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-21T00:18:31.900 INFO:teuthology.orchestra.run.trial120.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-21T00:18:31.900 INFO:teuthology.orchestra.run.trial120.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-21T00:18:31.900 INFO:teuthology.orchestra.run.trial120.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-21T00:18:31.900 INFO:teuthology.orchestra.run.trial120.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-21T00:18:31.900 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:31.900 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:31.936 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-21T00:18:32.242 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:28 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:32.325 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:32.326 INFO:teuthology.orchestra.run.trial120.stdout:======================================================================================= 2026-02-21T00:18:32.326 INFO:teuthology.orchestra.run.trial120.stdout: Package Arch Version Repository Size 2026-02-21T00:18:32.326 INFO:teuthology.orchestra.run.trial120.stdout:======================================================================================= 2026-02-21T00:18:32.326 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:18:32.327 INFO:teuthology.orchestra.run.trial120.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 7.4 M 2026-02-21T00:18:32.327 INFO:teuthology.orchestra.run.trial120.stdout:Installing dependencies: 2026-02-21T00:18:32.327 INFO:teuthology.orchestra.run.trial120.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-21T00:18:32.327 INFO:teuthology.orchestra.run.trial120.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-21T00:18:32.327 INFO:teuthology.orchestra.run.trial120.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-21T00:18:32.327 INFO:teuthology.orchestra.run.trial120.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-21T00:18:32.327 INFO:teuthology.orchestra.run.trial120.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-21T00:18:32.327 INFO:teuthology.orchestra.run.trial120.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-21T00:18:32.327 INFO:teuthology.orchestra.run.trial120.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-21T00:18:32.328 INFO:teuthology.orchestra.run.trial120.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-21T00:18:32.328 INFO:teuthology.orchestra.run.trial120.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-21T00:18:32.328 INFO:teuthology.orchestra.run.trial120.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-21T00:18:32.328 INFO:teuthology.orchestra.run.trial120.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-21T00:18:32.328 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:32.328 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:18:32.328 INFO:teuthology.orchestra.run.trial120.stdout:======================================================================================= 2026-02-21T00:18:32.328 INFO:teuthology.orchestra.run.trial120.stdout:Install 12 Packages 2026-02-21T00:18:32.328 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:32.328 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 43 M 2026-02-21T00:18:32.328 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 234 M 2026-02-21T00:18:32.328 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:18:32.605 INFO:teuthology.orchestra.run.trial120.stdout:(1/12): libquadmath-11.5.0-14.el9.x86_64.rpm 1.5 MB/s | 184 kB 00:00 2026-02-21T00:18:32.686 INFO:teuthology.orchestra.run.trial120.stdout:(2/12): flexiblas-3.0.4-9.el9.x86_64.rpm 366 kB/s | 30 kB 00:00 2026-02-21T00:18:32.905 INFO:teuthology.orchestra.run.trial120.stdout:(3/12): flexiblas-netlib-3.0.4-9.el9.x86_64.rpm 14 MB/s | 3.0 MB 00:00 2026-02-21T00:18:32.936 INFO:teuthology.orchestra.run.trial120.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-9.el9.x 483 kB/s | 15 kB 00:00 2026-02-21T00:18:32.967 INFO:teuthology.orchestra.run.trial120.stdout:(5/12): openblas-0.3.29-1.el9.x86_64.rpm 1.4 MB/s | 42 kB 00:00 2026-02-21T00:18:33.042 INFO:teuthology.orchestra.run.trial120.stdout:(6/12): openblas-openmp-0.3.29-1.el9.x86_64.rpm 70 MB/s | 5.3 MB 00:00 2026-02-21T00:18:33.076 INFO:teuthology.orchestra.run.trial120.stdout:(7/12): python3-devel-3.9.25-3.el9.x86_64.rpm 7.1 MB/s | 244 kB 00:00 2026-02-21T00:18:33.134 INFO:teuthology.orchestra.run.trial120.stdout:(8/12): libgfortran-11.5.0-14.el9.x86_64.rpm 1.2 MB/s | 794 kB 00:00 2026-02-21T00:18:33.179 INFO:teuthology.orchestra.run.trial120.stdout:(9/12): python3-numpy-1.23.5-2.el9.x86_64.rpm 59 MB/s | 6.1 MB 00:00 2026-02-21T00:18:33.333 INFO:teuthology.orchestra.run.trial120.stdout:(10/12): python3-numpy-f2py-1.23.5-2.el9.x86_64 2.2 MB/s | 442 kB 00:00 2026-02-21T00:18:33.499 INFO:teuthology.orchestra.run.trial120.stdout:(11/12): python3-scipy-1.9.3-2.el9.x86_64.rpm 60 MB/s | 19 MB 00:00 2026-02-21T00:18:39.528 INFO:teuthology.orchestra.run.trial120.stdout:(12/12): ceph-mgr-diskprediction-local-20.3.0-5 1.0 MB/s | 7.4 MB 00:07 2026-02-21T00:18:39.530 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:18:39.530 INFO:teuthology.orchestra.run.trial120.stdout:Total 5.9 MB/s | 43 MB 00:07 2026-02-21T00:18:39.693 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:18:39.708 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:18:39.709 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:18:39.911 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:18:39.911 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:18:40.129 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:18:40.131 INFO:teuthology.orchestra.run.trial120.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 1/12 2026-02-21T00:18:40.137 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 2/12 2026-02-21T00:18:40.173 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 3/12 2026-02-21T00:18:40.178 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 4/12 2026-02-21T00:18:40.237 INFO:teuthology.orchestra.run.trial120.stdout: Installing : openblas-0.3.29-1.el9.x86_64 5/12 2026-02-21T00:18:40.240 INFO:teuthology.orchestra.run.trial120.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 6/12 2026-02-21T00:18:40.261 INFO:teuthology.orchestra.run.trial120.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 7/12 2026-02-21T00:18:40.594 INFO:teuthology.orchestra.run.trial120.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 8/12 2026-02-21T00:18:40.668 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 9/12 2026-02-21T00:18:41.351 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 10/12 2026-02-21T00:18:41.805 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 11/12 2026-02-21T00:18:41.806 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd4 12/12 2026-02-21T00:18:42.085 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd4 12/12 2026-02-21T00:18:42.085 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd4 1/12 2026-02-21T00:18:42.085 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 2/12 2026-02-21T00:18:42.086 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 3/12 2026-02-21T00:18:42.086 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 4/12 2026-02-21T00:18:42.086 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 5/12 2026-02-21T00:18:42.086 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 6/12 2026-02-21T00:18:42.086 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 7/12 2026-02-21T00:18:42.086 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 8/12 2026-02-21T00:18:42.086 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 9/12 2026-02-21T00:18:42.086 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 10/12 2026-02-21T00:18:42.086 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 11/12 2026-02-21T00:18:42.142 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 12/12 2026-02-21T00:18:42.142 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:42.142 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:18:42.142 INFO:teuthology.orchestra.run.trial120.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-21T00:18:42.142 INFO:teuthology.orchestra.run.trial120.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-21T00:18:42.143 INFO:teuthology.orchestra.run.trial120.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-21T00:18:42.143 INFO:teuthology.orchestra.run.trial120.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-21T00:18:42.143 INFO:teuthology.orchestra.run.trial120.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-21T00:18:42.143 INFO:teuthology.orchestra.run.trial120.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-21T00:18:42.143 INFO:teuthology.orchestra.run.trial120.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-21T00:18:42.143 INFO:teuthology.orchestra.run.trial120.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-21T00:18:42.143 INFO:teuthology.orchestra.run.trial120.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-21T00:18:42.143 INFO:teuthology.orchestra.run.trial120.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-21T00:18:42.143 INFO:teuthology.orchestra.run.trial120.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-21T00:18:42.143 INFO:teuthology.orchestra.run.trial120.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-21T00:18:42.144 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:42.144 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:42.180 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-mgr-rook 2026-02-21T00:18:42.489 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:38 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:42.571 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:42.572 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:42.572 INFO:teuthology.orchestra.run.trial120.stdout: Package Arch Version Repository Size 2026-02-21T00:18:42.572 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:42.573 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:18:42.573 INFO:teuthology.orchestra.run.trial120.stdout: ceph-mgr-rook noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 50 k 2026-02-21T00:18:42.573 INFO:teuthology.orchestra.run.trial120.stdout:Installing dependencies: 2026-02-21T00:18:42.573 INFO:teuthology.orchestra.run.trial120.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-21T00:18:42.573 INFO:teuthology.orchestra.run.trial120.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-21T00:18:42.573 INFO:teuthology.orchestra.run.trial120.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-21T00:18:42.573 INFO:teuthology.orchestra.run.trial120.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-21T00:18:42.573 INFO:teuthology.orchestra.run.trial120.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-21T00:18:42.573 INFO:teuthology.orchestra.run.trial120.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-21T00:18:42.573 INFO:teuthology.orchestra.run.trial120.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-21T00:18:42.573 INFO:teuthology.orchestra.run.trial120.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-21T00:18:42.573 INFO:teuthology.orchestra.run.trial120.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-21T00:18:42.574 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:42.574 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:18:42.574 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:42.574 INFO:teuthology.orchestra.run.trial120.stdout:Install 10 Packages 2026-02-21T00:18:42.574 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:42.574 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 2.0 M 2026-02-21T00:18:42.574 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 25 M 2026-02-21T00:18:42.574 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:18:43.375 INFO:teuthology.orchestra.run.trial120.stdout:(1/10): ceph-mgr-rook-20.3.0-5343.gbcdd429b.el9 118 kB/s | 50 kB 00:00 2026-02-21T00:18:43.629 INFO:teuthology.orchestra.run.trial120.stdout:(2/10): python3-pyasn1-0.4.8-6.el9.noarch.rpm 233 kB/s | 159 kB 00:00 2026-02-21T00:18:43.706 INFO:teuthology.orchestra.run.trial120.stdout:(3/10): python3-pyasn1-modules-0.4.8-6.el9.noar 368 kB/s | 279 kB 00:00 2026-02-21T00:18:43.822 INFO:teuthology.orchestra.run.trial120.stdout:(4/10): python3-cachetools-4.2.4-1.el9.noarch.r 167 kB/s | 32 kB 00:00 2026-02-21T00:18:43.859 INFO:teuthology.orchestra.run.trial120.stdout:(5/10): python3-certifi-2023.05.07-4.el9.noarch 92 kB/s | 14 kB 00:00 2026-02-21T00:18:43.874 INFO:teuthology.orchestra.run.trial120.stdout:(6/10): python3-requests-oauthlib-1.3.0-12.el9. 107 kB/s | 54 kB 00:00 2026-02-21T00:18:44.044 INFO:teuthology.orchestra.run.trial120.stdout:(7/10): python3-google-auth-2.45.0-1.el9.noarch 1.1 MB/s | 254 kB 00:00 2026-02-21T00:18:44.097 INFO:teuthology.orchestra.run.trial120.stdout:(8/10): python3-rsa-4.9-2.el9.noarch.rpm 266 kB/s | 59 kB 00:00 2026-02-21T00:18:44.108 INFO:teuthology.orchestra.run.trial120.stdout:(9/10): python3-websocket-client-1.2.3-2.el9.no 1.4 MB/s | 90 kB 00:00 2026-02-21T00:18:44.196 INFO:teuthology.orchestra.run.trial120.stdout:(10/10): python3-kubernetes-26.1.0-3.el9.noarch 3.0 MB/s | 1.0 MB 00:00 2026-02-21T00:18:44.197 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:18:44.197 INFO:teuthology.orchestra.run.trial120.stdout:Total 1.2 MB/s | 2.0 MB 00:01 2026-02-21T00:18:44.221 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:18:44.229 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:18:44.230 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:18:44.268 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:18:44.268 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:18:44.342 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:18:44.393 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/10 2026-02-21T00:18:44.406 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/10 2026-02-21T00:18:44.421 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/10 2026-02-21T00:18:44.425 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/10 2026-02-21T00:18:44.433 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/10 2026-02-21T00:18:44.472 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/10 2026-02-21T00:18:44.484 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-21T00:18:44.803 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 8/10 2026-02-21T00:18:44.818 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 9/10 2026-02-21T00:18:44.819 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch 10/10 2026-02-21T00:18:44.929 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch 10/10 2026-02-21T00:18:44.929 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch 1/10 2026-02-21T00:18:44.929 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 2/10 2026-02-21T00:18:44.929 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 3/10 2026-02-21T00:18:44.929 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 4/10 2026-02-21T00:18:44.929 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 5/10 2026-02-21T00:18:44.929 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 6/10 2026-02-21T00:18:44.929 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-21T00:18:44.929 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 8/10 2026-02-21T00:18:44.930 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 9/10 2026-02-21T00:18:44.985 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 10/10 2026-02-21T00:18:44.985 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:44.985 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:18:44.986 INFO:teuthology.orchestra.run.trial120.stdout: ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-21T00:18:44.986 INFO:teuthology.orchestra.run.trial120.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-21T00:18:44.986 INFO:teuthology.orchestra.run.trial120.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-21T00:18:44.986 INFO:teuthology.orchestra.run.trial120.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-21T00:18:44.986 INFO:teuthology.orchestra.run.trial120.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-21T00:18:44.986 INFO:teuthology.orchestra.run.trial120.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-21T00:18:44.986 INFO:teuthology.orchestra.run.trial120.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-21T00:18:44.986 INFO:teuthology.orchestra.run.trial120.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-21T00:18:44.986 INFO:teuthology.orchestra.run.trial120.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-21T00:18:44.986 INFO:teuthology.orchestra.run.trial120.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-21T00:18:44.986 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:44.986 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:45.016 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-mgr-cephadm 2026-02-21T00:18:45.325 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:41 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:45.408 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:45.409 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:45.409 INFO:teuthology.orchestra.run.trial120.stdout: Package Arch Version Repository Size 2026-02-21T00:18:45.409 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:45.409 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:18:45.409 INFO:teuthology.orchestra.run.trial120.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 184 k 2026-02-21T00:18:45.409 INFO:teuthology.orchestra.run.trial120.stdout:Installing dependencies: 2026-02-21T00:18:45.409 INFO:teuthology.orchestra.run.trial120.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-21T00:18:45.410 INFO:teuthology.orchestra.run.trial120.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-21T00:18:45.410 INFO:teuthology.orchestra.run.trial120.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-21T00:18:45.410 INFO:teuthology.orchestra.run.trial120.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-21T00:18:45.410 INFO:teuthology.orchestra.run.trial120.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-21T00:18:45.410 INFO:teuthology.orchestra.run.trial120.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-21T00:18:45.410 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:45.410 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:18:45.410 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:45.410 INFO:teuthology.orchestra.run.trial120.stdout:Install 7 Packages 2026-02-21T00:18:45.410 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:45.410 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 7.1 M 2026-02-21T00:18:45.410 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 34 M 2026-02-21T00:18:45.411 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:18:45.644 INFO:teuthology.orchestra.run.trial120.stdout:(1/7): python3-jinja2-2.11.3-8.el9.noarch.rpm 4.2 MB/s | 249 kB 00:00 2026-02-21T00:18:45.655 INFO:teuthology.orchestra.run.trial120.stdout:(2/7): python3-markupsafe-1.1.1-12.el9.x86_64.r 3.2 MB/s | 35 kB 00:00 2026-02-21T00:18:45.696 INFO:teuthology.orchestra.run.trial120.stdout:(3/7): python3-babel-2.9.1-2.el9.noarch.rpm 54 MB/s | 6.0 MB 00:00 2026-02-21T00:18:45.716 INFO:teuthology.orchestra.run.trial120.stdout:(4/7): python3-asyncssh-2.13.2-5.el9.noarch.rpm 8.8 MB/s | 548 kB 00:00 2026-02-21T00:18:45.726 INFO:teuthology.orchestra.run.trial120.stdout:(5/7): python3-typing-extensions-4.15.0-1.el9.n 7.9 MB/s | 86 kB 00:00 2026-02-21T00:18:45.733 INFO:teuthology.orchestra.run.trial120.stdout:(6/7): python3-natsort-7.1.1-5.el9.noarch.rpm 1.5 MB/s | 58 kB 00:00 2026-02-21T00:18:46.127 INFO:teuthology.orchestra.run.trial120.stdout:(7/7): ceph-mgr-cephadm-20.3.0-5343.gbcdd429b.e 340 kB/s | 184 kB 00:00 2026-02-21T00:18:46.128 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:18:46.128 INFO:teuthology.orchestra.run.trial120.stdout:Total 9.9 MB/s | 7.1 MB 00:00 2026-02-21T00:18:46.163 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:18:46.170 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:18:46.170 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:18:46.241 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:18:46.241 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:18:46.331 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:18:46.379 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 1/7 2026-02-21T00:18:46.389 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/7 2026-02-21T00:18:46.395 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/7 2026-02-21T00:18:46.612 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-21T00:18:46.636 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 5/7 2026-02-21T00:18:46.660 INFO:teuthology.orchestra.run.trial120.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 6/7 2026-02-21T00:18:46.661 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 7/7 2026-02-21T00:18:46.769 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 7/7 2026-02-21T00:18:46.769 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 1/7 2026-02-21T00:18:46.769 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/7 2026-02-21T00:18:46.769 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 3/7 2026-02-21T00:18:46.769 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-21T00:18:46.769 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 5/7 2026-02-21T00:18:46.770 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 6/7 2026-02-21T00:18:46.826 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 7/7 2026-02-21T00:18:46.826 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:46.826 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:18:46.826 INFO:teuthology.orchestra.run.trial120.stdout: ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-21T00:18:46.826 INFO:teuthology.orchestra.run.trial120.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-21T00:18:46.827 INFO:teuthology.orchestra.run.trial120.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-21T00:18:46.827 INFO:teuthology.orchestra.run.trial120.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-21T00:18:46.827 INFO:teuthology.orchestra.run.trial120.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-21T00:18:46.827 INFO:teuthology.orchestra.run.trial120.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-21T00:18:46.827 INFO:teuthology.orchestra.run.trial120.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-21T00:18:46.827 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:46.827 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:46.856 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-osd 2026-02-21T00:18:47.160 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:43 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:47.223 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:18:47.241 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:47.242 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:18:47.242 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:47.262 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-osd-classic 2026-02-21T00:18:47.568 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:43 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:47.632 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:18:47.650 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:47.651 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:18:47.651 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:47.671 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-fuse 2026-02-21T00:18:47.974 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:43 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:48.058 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:48.059 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:48.060 INFO:teuthology.orchestra.run.trial120.stdout: Package Arch Version Repo Size 2026-02-21T00:18:48.060 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:48.060 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:18:48.060 INFO:teuthology.orchestra.run.trial120.stdout: ceph-fuse x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 914 k 2026-02-21T00:18:48.060 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:48.060 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:18:48.060 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:48.060 INFO:teuthology.orchestra.run.trial120.stdout:Install 1 Package 2026-02-21T00:18:48.060 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:48.060 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 914 k 2026-02-21T00:18:48.061 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 2.9 M 2026-02-21T00:18:48.061 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:18:50.213 INFO:teuthology.orchestra.run.trial120.stdout:ceph-fuse-20.3.0-5343.gbcdd429b.el9.x86_64.rpm 424 kB/s | 914 kB 00:02 2026-02-21T00:18:50.214 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:18:50.214 INFO:teuthology.orchestra.run.trial120.stdout:Total 424 kB/s | 914 kB 00:02 2026-02-21T00:18:50.214 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:18:50.219 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:18:50.219 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:18:50.251 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:18:50.251 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:18:50.340 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:18:50.356 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-21T00:18:50.576 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-21T00:18:50.633 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-21T00:18:50.633 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:50.633 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:18:50.633 INFO:teuthology.orchestra.run.trial120.stdout: ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:18:50.633 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:50.634 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:50.661 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-volume 2026-02-21T00:18:50.968 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:46 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:51.051 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:51.052 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:51.052 INFO:teuthology.orchestra.run.trial120.stdout: Package Arch Version Repository Size 2026-02-21T00:18:51.052 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:51.052 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:18:51.052 INFO:teuthology.orchestra.run.trial120.stdout: ceph-volume noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 301 k 2026-02-21T00:18:51.052 INFO:teuthology.orchestra.run.trial120.stdout:Installing dependencies: 2026-02-21T00:18:51.053 INFO:teuthology.orchestra.run.trial120.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-21T00:18:51.053 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:51.053 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:18:51.053 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:51.053 INFO:teuthology.orchestra.run.trial120.stdout:Install 2 Packages 2026-02-21T00:18:51.053 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:51.053 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 652 k 2026-02-21T00:18:51.053 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 2.2 M 2026-02-21T00:18:51.053 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:18:51.462 INFO:teuthology.orchestra.run.trial120.stdout:(1/2): cryptsetup-2.8.1-3.el9.x86_64.rpm 3.2 MB/s | 351 kB 00:00 2026-02-21T00:18:52.212 INFO:teuthology.orchestra.run.trial120.stdout:(2/2): ceph-volume-20.3.0-5343.gbcdd429b.el9.no 352 kB/s | 301 kB 00:00 2026-02-21T00:18:52.212 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:18:52.212 INFO:teuthology.orchestra.run.trial120.stdout:Total 562 kB/s | 652 kB 00:01 2026-02-21T00:18:52.222 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:18:52.228 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:18:52.228 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:18:52.271 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:18:52.271 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:18:52.351 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:18:52.413 INFO:teuthology.orchestra.run.trial120.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 1/2 2026-02-21T00:18:52.416 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch 2/2 2026-02-21T00:18:52.430 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch 2/2 2026-02-21T00:18:52.430 INFO:teuthology.orchestra.run.trial120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T00:18:52.430 INFO:teuthology.orchestra.run.trial120.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-21T00:18:52.430 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:52.654 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch 1/2 2026-02-21T00:18:52.711 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 2/2 2026-02-21T00:18:52.711 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:52.711 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:18:52.711 INFO:teuthology.orchestra.run.trial120.stdout: ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch cryptsetup-2.8.1-3.el9.x86_64 2026-02-21T00:18:52.711 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:52.711 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:52.741 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install librados-devel 2026-02-21T00:18:53.055 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:49 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:53.138 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:53.138 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:53.139 INFO:teuthology.orchestra.run.trial120.stdout: Package Arch Version Repo Size 2026-02-21T00:18:53.139 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:53.139 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:18:53.139 INFO:teuthology.orchestra.run.trial120.stdout: librados-devel x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 124 k 2026-02-21T00:18:53.139 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:53.139 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:18:53.139 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:53.139 INFO:teuthology.orchestra.run.trial120.stdout:Install 1 Package 2026-02-21T00:18:53.139 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:53.139 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 124 k 2026-02-21T00:18:53.139 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 434 k 2026-02-21T00:18:53.140 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:18:53.599 INFO:teuthology.orchestra.run.trial120.stdout:librados-devel-20.3.0-5343.gbcdd429b.el9.x86_64 269 kB/s | 124 kB 00:00 2026-02-21T00:18:53.600 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:18:53.601 INFO:teuthology.orchestra.run.trial120.stdout:Total 267 kB/s | 124 kB 00:00 2026-02-21T00:18:53.601 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:18:53.604 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:18:53.604 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:18:53.635 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:18:53.635 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:18:53.688 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:18:53.705 INFO:teuthology.orchestra.run.trial120.stdout: Installing : librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-21T00:18:53.785 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-21T00:18:53.843 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-21T00:18:53.843 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:53.843 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:18:53.843 INFO:teuthology.orchestra.run.trial120.stdout: librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:18:53.843 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:53.843 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:53.871 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install libcephfs2 2026-02-21T00:18:54.180 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:50 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:54.245 INFO:teuthology.orchestra.run.trial120.stdout:Package libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:18:54.264 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:54.264 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:18:54.264 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:54.286 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install libcephfs-devel 2026-02-21T00:18:54.589 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:50 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:54.673 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:54.674 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:54.674 INFO:teuthology.orchestra.run.trial120.stdout: Package Arch Version Repo Size 2026-02-21T00:18:54.674 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:54.674 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:18:54.674 INFO:teuthology.orchestra.run.trial120.stdout: libcephfs-devel x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 37 k 2026-02-21T00:18:54.674 INFO:teuthology.orchestra.run.trial120.stdout:Installing dependencies: 2026-02-21T00:18:54.674 INFO:teuthology.orchestra.run.trial120.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 26 k 2026-02-21T00:18:54.674 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:54.674 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:18:54.674 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:54.675 INFO:teuthology.orchestra.run.trial120.stdout:Install 2 Packages 2026-02-21T00:18:54.675 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:54.675 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 63 k 2026-02-21T00:18:54.675 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 227 k 2026-02-21T00:18:54.675 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:18:55.035 INFO:teuthology.orchestra.run.trial120.stdout:(1/2): libcephfs-proxy2-20.3.0-5343.gbcdd429b.e 71 kB/s | 26 kB 00:00 2026-02-21T00:18:55.097 INFO:teuthology.orchestra.run.trial120.stdout:(2/2): libcephfs-devel-20.3.0-5343.gbcdd429b.el 88 kB/s | 37 kB 00:00 2026-02-21T00:18:55.098 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:18:55.098 INFO:teuthology.orchestra.run.trial120.stdout:Total 148 kB/s | 63 kB 00:00 2026-02-21T00:18:55.098 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:18:55.101 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:18:55.101 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:18:55.134 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:18:55.134 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:18:55.183 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:18:55.184 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libcephfs-proxy2-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/2 2026-02-21T00:18:55.199 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/2 2026-02-21T00:18:55.215 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/2 2026-02-21T00:18:55.271 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/2 2026-02-21T00:18:55.271 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/2 2026-02-21T00:18:55.327 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/2 2026-02-21T00:18:55.327 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:55.328 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:18:55.328 INFO:teuthology.orchestra.run.trial120.stdout: libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:18:55.328 INFO:teuthology.orchestra.run.trial120.stdout: libcephfs-proxy2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:18:55.328 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:55.328 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:55.354 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install librados2 2026-02-21T00:18:55.664 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:51 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:55.728 INFO:teuthology.orchestra.run.trial120.stdout:Package librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:18:55.746 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:55.747 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:18:55.747 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:55.768 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install librbd1 2026-02-21T00:18:56.078 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:52 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:56.143 INFO:teuthology.orchestra.run.trial120.stdout:Package librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:18:56.161 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:56.162 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:18:56.162 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:56.183 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install python3-rados 2026-02-21T00:18:56.494 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:52 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:56.561 INFO:teuthology.orchestra.run.trial120.stdout:Package python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:18:56.579 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:56.580 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:18:56.580 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:56.601 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install python3-rgw 2026-02-21T00:18:56.906 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:52 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:56.971 INFO:teuthology.orchestra.run.trial120.stdout:Package python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:18:56.989 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:56.990 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:18:56.990 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:57.011 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install python3-cephfs 2026-02-21T00:18:57.317 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:53 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:57.383 INFO:teuthology.orchestra.run.trial120.stdout:Package python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:18:57.401 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:57.402 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:18:57.402 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:57.425 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install python3-rbd 2026-02-21T00:18:57.735 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:53 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:57.800 INFO:teuthology.orchestra.run.trial120.stdout:Package python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:18:57.818 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:57.819 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:18:57.819 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:57.840 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install rbd-fuse 2026-02-21T00:18:58.151 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:54 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:58.235 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:58.236 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:58.236 INFO:teuthology.orchestra.run.trial120.stdout: Package Architecture Version Repository Size 2026-02-21T00:18:58.236 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:58.236 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:18:58.236 INFO:teuthology.orchestra.run.trial120.stdout: rbd-fuse x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 86 k 2026-02-21T00:18:58.237 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:58.237 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:18:58.237 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:58.237 INFO:teuthology.orchestra.run.trial120.stdout:Install 1 Package 2026-02-21T00:18:58.237 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:58.237 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 86 k 2026-02-21T00:18:58.237 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 236 k 2026-02-21T00:18:58.237 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:18:58.722 INFO:teuthology.orchestra.run.trial120.stdout:rbd-fuse-20.3.0-5343.gbcdd429b.el9.x86_64.rpm 178 kB/s | 86 kB 00:00 2026-02-21T00:18:58.723 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:18:58.723 INFO:teuthology.orchestra.run.trial120.stdout:Total 178 kB/s | 86 kB 00:00 2026-02-21T00:18:58.723 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:18:58.727 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:18:58.727 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:18:58.757 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:18:58.757 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:18:58.808 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:18:58.825 INFO:teuthology.orchestra.run.trial120.stdout: Installing : rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-21T00:18:58.904 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-21T00:18:58.962 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-21T00:18:58.962 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:58.962 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:18:58.963 INFO:teuthology.orchestra.run.trial120.stdout: rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:18:58.963 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:58.963 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:18:58.991 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install rbd-mirror 2026-02-21T00:18:59.302 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:55 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:18:59.385 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:18:59.386 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:59.386 INFO:teuthology.orchestra.run.trial120.stdout: Package Arch Version Repo Size 2026-02-21T00:18:59.386 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:59.386 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:18:59.386 INFO:teuthology.orchestra.run.trial120.stdout: rbd-mirror x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 2.6 M 2026-02-21T00:18:59.386 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:59.386 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:18:59.386 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:18:59.386 INFO:teuthology.orchestra.run.trial120.stdout:Install 1 Package 2026-02-21T00:18:59.386 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:18:59.386 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 2.6 M 2026-02-21T00:18:59.386 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 11 M 2026-02-21T00:18:59.386 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:19:01.698 INFO:teuthology.orchestra.run.trial120.stdout:rbd-mirror-20.3.0-5343.gbcdd429b.el9.x86_64.rpm 1.1 MB/s | 2.6 MB 00:02 2026-02-21T00:19:01.698 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:19:01.698 INFO:teuthology.orchestra.run.trial120.stdout:Total 1.1 MB/s | 2.6 MB 00:02 2026-02-21T00:19:01.698 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:19:01.703 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:19:01.703 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:19:01.741 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:19:01.741 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:19:01.903 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:19:01.905 INFO:teuthology.orchestra.run.trial120.stdout: Installing : rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-21T00:19:01.928 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-21T00:19:01.928 INFO:teuthology.orchestra.run.trial120.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-21T00:19:01.928 INFO:teuthology.orchestra.run.trial120.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-21T00:19:01.929 INFO:teuthology.orchestra.run.trial120.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-21T00:19:01.929 INFO:teuthology.orchestra.run.trial120.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-21T00:19:01.929 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:19:02.245 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-21T00:19:02.245 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:19:02.246 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:19:02.246 INFO:teuthology.orchestra.run.trial120.stdout: rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:19:02.246 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:19:02.246 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:19:02.274 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install rbd-nbd 2026-02-21T00:19:02.586 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:58 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:19:02.669 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:19:02.669 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:19:02.670 INFO:teuthology.orchestra.run.trial120.stdout: Package Architecture Version Repository Size 2026-02-21T00:19:02.670 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:19:02.670 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:19:02.670 INFO:teuthology.orchestra.run.trial120.stdout: rbd-nbd x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 166 k 2026-02-21T00:19:02.670 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:19:02.670 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:19:02.670 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:19:02.670 INFO:teuthology.orchestra.run.trial120.stdout:Install 1 Package 2026-02-21T00:19:02.670 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:19:02.671 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 166 k 2026-02-21T00:19:02.671 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 496 k 2026-02-21T00:19:02.671 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:19:03.352 INFO:teuthology.orchestra.run.trial120.stdout:rbd-nbd-20.3.0-5343.gbcdd429b.el9.x86_64.rpm 244 kB/s | 166 kB 00:00 2026-02-21T00:19:03.353 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:19:03.353 INFO:teuthology.orchestra.run.trial120.stdout:Total 243 kB/s | 166 kB 00:00 2026-02-21T00:19:03.353 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:19:03.358 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:19:03.358 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:19:03.388 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:19:03.388 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:19:03.443 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:19:03.460 INFO:teuthology.orchestra.run.trial120.stdout: Installing : rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-21T00:19:03.544 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-21T00:19:03.603 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-21T00:19:03.603 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:19:03.604 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:19:03.604 INFO:teuthology.orchestra.run.trial120.stdout: rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:19:03.604 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:19:03.604 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:19:03.631 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-osd-crimson 2026-02-21T00:19:03.943 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:03:59 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:19:04.008 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:19:04.026 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:19:04.026 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:19:04.026 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:19:04.048 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-21T00:19:04.358 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:04:00 ago on Sat 21 Feb 2026 12:15:04 AM UTC. 2026-02-21T00:19:04.443 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:19:04.443 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:19:04.444 INFO:teuthology.orchestra.run.trial120.stdout: Package Arch Version Repo Size 2026-02-21T00:19:04.444 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:19:04.444 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:19:04.444 INFO:teuthology.orchestra.run.trial120.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 433 M 2026-02-21T00:19:04.444 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:19:04.444 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:19:04.444 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:19:04.444 INFO:teuthology.orchestra.run.trial120.stdout:Install 1 Package 2026-02-21T00:19:04.444 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:19:04.444 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 433 M 2026-02-21T00:19:04.445 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 2.6 G 2026-02-21T00:19:04.445 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:23:02.150 INFO:teuthology.orchestra.run.trial120.stdout:ceph-osd-crimson-debuginfo-20.3.0-5343.gbcdd429 1.8 MB/s | 433 MB 03:57 2026-02-21T00:23:02.151 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:23:02.151 INFO:teuthology.orchestra.run.trial120.stdout:Total 1.8 MB/s | 433 MB 03:57 2026-02-21T00:23:02.151 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:23:02.152 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:23:02.152 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:23:03.439 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:23:03.439 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:23:27.060 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:23:27.077 INFO:teuthology.orchestra.run.trial120.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5343.gbcdd429b.e 1/1 2026-02-21T00:23:29.010 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5343.gbcdd429b.e 1/1 2026-02-21T00:23:29.066 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5343.gbcdd429b.e 1/1 2026-02-21T00:23:29.066 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:23:29.066 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:23:29.066 INFO:teuthology.orchestra.run.trial120.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:23:29.067 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:23:29.067 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:23:29.145 DEBUG:teuthology.parallel:result is None 2026-02-21T00:23:29.147 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-21T00:23:29.315 DEBUG:teuthology.orchestra.run.trial120:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-21T00:23:29.336 INFO:teuthology.orchestra.run.trial120.stdout:20.3.0-5343.gbcdd429b.el9 2026-02-21T00:23:29.336 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5343.gbcdd429b.el9 2026-02-21T00:23:29.336 INFO:teuthology.task.install:The correct ceph version 20.3.0-5343.gbcdd429b is installed. 2026-02-21T00:23:29.338 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-21T00:23:29.339 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:23:29.339 DEBUG:teuthology.orchestra.run.trial120:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-21T00:23:29.401 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-21T00:23:29.402 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:23:29.402 DEBUG:teuthology.orchestra.run.trial120:> sudo dd of=/usr/bin/daemon-helper 2026-02-21T00:23:29.465 DEBUG:teuthology.orchestra.run.trial120:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-21T00:23:29.530 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-21T00:23:29.530 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:23:29.531 DEBUG:teuthology.orchestra.run.trial120:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-21T00:23:29.595 DEBUG:teuthology.orchestra.run.trial120:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-21T00:23:29.659 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-21T00:23:29.660 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:23:29.660 DEBUG:teuthology.orchestra.run.trial120:> sudo dd of=/usr/bin/stdin-killer 2026-02-21T00:23:29.724 DEBUG:teuthology.orchestra.run.trial120:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-21T00:23:29.789 INFO:teuthology.run_tasks:Running task pexec... 2026-02-21T00:23:29.811 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-21T00:23:29.811 DEBUG:teuthology.orchestra.run.trial120:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-21T00:23:29.832 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial120.front.sepia.ceph.com 2026-02-21T00:23:29.832 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-21T00:23:29.883 DEBUG:teuthology.parallel:result is None 2026-02-21T00:23:29.883 INFO:teuthology.run_tasks:Running task install... 2026-02-21T00:23:29.897 DEBUG:teuthology.task.install:project ceph 2026-02-21T00:23:29.898 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}} 2026-02-21T00:23:29.898 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'} 2026-02-21T00:23:29.898 INFO:teuthology.task.install:Using flavor: debug 2026-02-21T00:23:29.908 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-21T00:23:29.908 INFO:teuthology.task.install:extra packages: [] 2026-02-21T00:23:29.909 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'tag': None, 'wait_for_package': False} 2026-02-21T00:23:29.909 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-21T00:23:30.063 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/ 2026-02-21T00:23:30.063 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5343.gbcdd429b 2026-02-21T00:23:30.448 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-21T00:23:30.448 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:23:30.449 DEBUG:teuthology.orchestra.run.trial120:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-21T00:23:30.474 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-21T00:23:30.474 DEBUG:teuthology.orchestra.run.trial120:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-21T00:23:30.541 DEBUG:teuthology.orchestra.run.trial120:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-21T00:23:30.605 DEBUG:teuthology.orchestra.run.trial120:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-21T00:23:30.662 INFO:teuthology.orchestra.run.trial120.stdout:check_obsoletes = 1 2026-02-21T00:23:30.672 DEBUG:teuthology.orchestra.run.trial120:> sudo yum clean all 2026-02-21T00:23:30.834 INFO:teuthology.orchestra.run.trial120.stdout:56 files removed 2026-02-21T00:23:30.850 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-radosgw 2026-02-21T00:23:31.710 INFO:teuthology.orchestra.run.trial120.stdout:ceph packages for x86_64 129 kB/s | 92 kB 00:00 2026-02-21T00:23:32.315 INFO:teuthology.orchestra.run.trial120.stdout:ceph noarch packages 41 kB/s | 24 kB 00:00 2026-02-21T00:23:32.883 INFO:teuthology.orchestra.run.trial120.stdout:ceph source packages 3.8 kB/s | 2.1 kB 00:00 2026-02-21T00:23:34.269 INFO:teuthology.orchestra.run.trial120.stdout:CentOS Stream 9 - BaseOS 6.5 MB/s | 8.9 MB 00:01 2026-02-21T00:23:59.358 INFO:teuthology.orchestra.run.trial120.stdout:CentOS Stream 9 - AppStream 1.1 MB/s | 27 MB 00:24 2026-02-21T00:24:03.101 INFO:teuthology.orchestra.run.trial120.stdout:CentOS Stream 9 - CRB 6.1 MB/s | 7.8 MB 00:01 2026-02-21T00:24:04.818 INFO:teuthology.orchestra.run.trial120.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 20 kB 00:00 2026-02-21T00:24:06.379 INFO:teuthology.orchestra.run.trial120.stdout:Extra Packages for Enterprise Linux 14 MB/s | 20 MB 00:01 2026-02-21T00:24:09.542 INFO:teuthology.orchestra.run.trial120.stdout:lab-extras 3.0 MB/s | 50 kB 00:00 2026-02-21T00:24:10.713 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:10.731 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:10.732 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:10.732 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:10.755 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-test 2026-02-21T00:24:11.066 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:02 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:11.130 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:11.149 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:11.149 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:11.149 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:11.171 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph 2026-02-21T00:24:11.477 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:02 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:11.543 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:11.561 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:11.562 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:11.562 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:11.583 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-base 2026-02-21T00:24:11.889 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:02 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:11.953 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:11.971 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:11.972 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:11.972 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:11.993 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install cephadm 2026-02-21T00:24:12.305 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:03 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:12.371 INFO:teuthology.orchestra.run.trial120.stdout:Package cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:24:12.390 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:12.390 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:12.390 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:12.412 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-immutable-object-cache 2026-02-21T00:24:12.722 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:03 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:12.788 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:12.807 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:12.807 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:12.807 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:12.828 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-mgr 2026-02-21T00:24:13.136 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:04 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:13.201 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:13.219 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:13.219 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:13.220 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:13.241 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-mgr-dashboard 2026-02-21T00:24:13.553 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:04 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:13.619 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:24:13.638 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:13.638 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:13.638 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:13.660 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-21T00:24:13.969 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:04 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:14.034 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:24:14.052 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:14.053 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:14.053 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:14.075 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-mgr-rook 2026-02-21T00:24:14.383 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:05 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:14.449 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:24:14.468 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:14.469 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:14.469 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:14.490 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-mgr-cephadm 2026-02-21T00:24:14.800 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:05 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:14.867 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:24:14.885 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:14.886 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:14.886 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:14.908 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-osd 2026-02-21T00:24:15.222 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:06 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:15.286 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:15.304 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:15.305 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:15.305 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:15.326 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-osd-classic 2026-02-21T00:24:15.636 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:06 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:15.701 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:15.719 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:15.720 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:15.720 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:15.741 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-fuse 2026-02-21T00:24:16.049 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:07 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:16.114 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:16.133 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:16.133 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:16.134 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:16.155 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install ceph-volume 2026-02-21T00:24:16.463 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:07 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:16.528 INFO:teuthology.orchestra.run.trial120.stdout:Package ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:24:16.547 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:16.547 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:16.548 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:16.569 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install librados-devel 2026-02-21T00:24:16.876 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:07 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:16.940 INFO:teuthology.orchestra.run.trial120.stdout:Package librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:16.958 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:16.959 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:16.959 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:16.980 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install libcephfs2 2026-02-21T00:24:17.286 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:08 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:17.352 INFO:teuthology.orchestra.run.trial120.stdout:Package libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:17.370 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:17.371 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:17.371 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:17.392 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install libcephfs-devel 2026-02-21T00:24:17.700 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:08 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:17.764 INFO:teuthology.orchestra.run.trial120.stdout:Package libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:17.782 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:17.783 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:17.783 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:17.805 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install librados2 2026-02-21T00:24:18.117 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:09 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:18.182 INFO:teuthology.orchestra.run.trial120.stdout:Package librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:18.200 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:18.200 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:18.201 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:18.223 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install librbd1 2026-02-21T00:24:18.539 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:09 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:18.603 INFO:teuthology.orchestra.run.trial120.stdout:Package librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:18.621 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:18.621 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:18.622 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:18.643 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install python3-rados 2026-02-21T00:24:18.953 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:09 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:19.018 INFO:teuthology.orchestra.run.trial120.stdout:Package python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:19.037 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:19.037 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:19.037 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:19.059 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install python3-rgw 2026-02-21T00:24:19.372 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:10 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:19.437 INFO:teuthology.orchestra.run.trial120.stdout:Package python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:19.455 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:19.456 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:19.456 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:19.478 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install python3-cephfs 2026-02-21T00:24:19.788 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:10 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:19.853 INFO:teuthology.orchestra.run.trial120.stdout:Package python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:19.871 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:19.872 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:19.872 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:19.894 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install python3-rbd 2026-02-21T00:24:20.203 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:11 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:20.270 INFO:teuthology.orchestra.run.trial120.stdout:Package python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:20.289 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:20.289 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:20.289 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:20.311 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install rbd-fuse 2026-02-21T00:24:20.620 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:11 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:20.683 INFO:teuthology.orchestra.run.trial120.stdout:Package rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:20.702 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:20.702 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:20.702 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:20.723 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install rbd-mirror 2026-02-21T00:24:21.033 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:12 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:21.099 INFO:teuthology.orchestra.run.trial120.stdout:Package rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:21.117 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:21.118 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:21.118 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:21.138 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install rbd-nbd 2026-02-21T00:24:21.443 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:12 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:24:21.508 INFO:teuthology.orchestra.run.trial120.stdout:Package rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:24:21.527 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:24:21.527 INFO:teuthology.orchestra.run.trial120.stdout:Nothing to do. 2026-02-21T00:24:21.528 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:24:21.551 DEBUG:teuthology.parallel:result is None 2026-02-21T00:24:21.552 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-21T00:24:21.716 DEBUG:teuthology.orchestra.run.trial120:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-21T00:24:21.736 INFO:teuthology.orchestra.run.trial120.stdout:20.3.0-5343.gbcdd429b.el9 2026-02-21T00:24:21.737 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5343.gbcdd429b.el9 2026-02-21T00:24:21.737 INFO:teuthology.task.install:The correct ceph version 20.3.0-5343.gbcdd429b is installed. 2026-02-21T00:24:21.739 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-21T00:24:21.739 INFO:teuthology.run_tasks:Running task ceph... 2026-02-21T00:24:21.844 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-21T00:24:21.844 DEBUG:teuthology.orchestra.run.trial120:> sudo chmod 777 /var/log/ceph 2026-02-21T00:24:21.868 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-21T00:24:21.868 DEBUG:teuthology.orchestra.run.trial120:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-21T00:24:21.931 INFO:tasks.ceph:Creating extra log directories... 2026-02-21T00:24:21.931 DEBUG:teuthology.orchestra.run.trial120:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-21T00:24:22.007 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-21T00:24:22.007 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-21T00:24:22.007 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61964', 'branch': 'wip-liucm-osd-shard-redesign-crimson-only', 'description': 'crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} perf_ignorelist settings/optimized workloads/radosbench_4M_seq_read}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '61964', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-liucm-osd-shard-redesign-crimson-only'}, 'ceph': {'conf': {'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}}, 'workunit': {'branch': 'wip-liucm-osd-shard-redesign-crimson-only', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}}, 'owner': 'scheduled_liucm@soko04.front.sepia.ceph.com', 'priority': 120, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 6283, 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'wip-liucm-osd-shard-redesign-crimson-only', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'targets': {'trial120.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG66fx57biwyrVPWfPX1FEs1YTPkdGF7zw1pBf+DBHObEX0zicQsyu9EALHHsKoW1jjYBtcvgrO0uIL3ltaCQM0='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'wait-for-scrub': False, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'cluster': 'ceph'}}, {'ssh_keys': None}, {'cbt': {'benchmarks': {'radosbench': {'concurrent_ops': 4, 'concurrent_procs': 2, 'op_size': [4194304], 'pool_monitoring_list': ['collectl'], 'pool_profile': 'replicated', 'run_monitoring_list': ['collectl'], 'time': 60, 'write_only': False}}, 'cluster': {'iterations': 1, 'osds_per_node': 3, 'pool_profiles': {'replicated': {'pg_size': 256, 'pgp_size': 256, 'replication': 'replicated'}}, 'user': 'ubuntu'}, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-20_22:16:48', 'tube': 'trial', 'user': 'liucm', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699'} 2026-02-21T00:24:22.008 DEBUG:teuthology.orchestra.run.trial120:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-21T00:24:22.052 DEBUG:teuthology.orchestra.run.trial120:> sudo install -d -m0777 -- /var/run/ceph 2026-02-21T00:24:22.116 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:24:22.116 DEBUG:teuthology.orchestra.run.trial120:> dd if=/scratch_devs of=/dev/stdout 2026-02-21T00:24:22.172 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-21T00:24:22.172 DEBUG:teuthology.orchestra.run.trial120:> stat /dev/vg_nvme/lv_1 2026-02-21T00:24:22.228 INFO:teuthology.orchestra.run.trial120.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-21T00:24:22.229 INFO:teuthology.orchestra.run.trial120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T00:24:22.229 INFO:teuthology.orchestra.run.trial120.stdout:Device: 6h/6d Inode: 906 Links: 1 2026-02-21T00:24:22.229 INFO:teuthology.orchestra.run.trial120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T00:24:22.229 INFO:teuthology.orchestra.run.trial120.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T00:24:22.229 INFO:teuthology.orchestra.run.trial120.stdout:Access: 2026-02-21 00:19:02.071246690 +0000 2026-02-21T00:24:22.229 INFO:teuthology.orchestra.run.trial120.stdout:Modify: 2026-02-21 00:14:37.485121341 +0000 2026-02-21T00:24:22.229 INFO:teuthology.orchestra.run.trial120.stdout:Change: 2026-02-21 00:14:37.485121341 +0000 2026-02-21T00:24:22.229 INFO:teuthology.orchestra.run.trial120.stdout: Birth: 2026-02-21 00:14:37.485121341 +0000 2026-02-21T00:24:22.230 DEBUG:teuthology.orchestra.run.trial120:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-21T00:24:22.293 INFO:teuthology.orchestra.run.trial120.stderr:1+0 records in 2026-02-21T00:24:22.293 INFO:teuthology.orchestra.run.trial120.stderr:1+0 records out 2026-02-21T00:24:22.294 INFO:teuthology.orchestra.run.trial120.stderr:512 bytes copied, 0.000160568 s, 3.2 MB/s 2026-02-21T00:24:22.295 DEBUG:teuthology.orchestra.run.trial120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-21T00:24:22.352 DEBUG:teuthology.orchestra.run.trial120:> stat /dev/vg_nvme/lv_2 2026-02-21T00:24:22.407 INFO:teuthology.orchestra.run.trial120.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-21T00:24:22.407 INFO:teuthology.orchestra.run.trial120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T00:24:22.407 INFO:teuthology.orchestra.run.trial120.stdout:Device: 6h/6d Inode: 938 Links: 1 2026-02-21T00:24:22.407 INFO:teuthology.orchestra.run.trial120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T00:24:22.407 INFO:teuthology.orchestra.run.trial120.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T00:24:22.407 INFO:teuthology.orchestra.run.trial120.stdout:Access: 2026-02-21 00:19:02.071246690 +0000 2026-02-21T00:24:22.408 INFO:teuthology.orchestra.run.trial120.stdout:Modify: 2026-02-21 00:14:37.708112730 +0000 2026-02-21T00:24:22.408 INFO:teuthology.orchestra.run.trial120.stdout:Change: 2026-02-21 00:14:37.708112730 +0000 2026-02-21T00:24:22.408 INFO:teuthology.orchestra.run.trial120.stdout: Birth: 2026-02-21 00:14:37.708112730 +0000 2026-02-21T00:24:22.408 DEBUG:teuthology.orchestra.run.trial120:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-21T00:24:22.469 INFO:teuthology.orchestra.run.trial120.stderr:1+0 records in 2026-02-21T00:24:22.469 INFO:teuthology.orchestra.run.trial120.stderr:1+0 records out 2026-02-21T00:24:22.469 INFO:teuthology.orchestra.run.trial120.stderr:512 bytes copied, 0.00013442 s, 3.8 MB/s 2026-02-21T00:24:22.470 DEBUG:teuthology.orchestra.run.trial120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-21T00:24:22.528 DEBUG:teuthology.orchestra.run.trial120:> stat /dev/vg_nvme/lv_3 2026-02-21T00:24:22.584 INFO:teuthology.orchestra.run.trial120.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-21T00:24:22.585 INFO:teuthology.orchestra.run.trial120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T00:24:22.585 INFO:teuthology.orchestra.run.trial120.stdout:Device: 6h/6d Inode: 966 Links: 1 2026-02-21T00:24:22.585 INFO:teuthology.orchestra.run.trial120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T00:24:22.585 INFO:teuthology.orchestra.run.trial120.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T00:24:22.585 INFO:teuthology.orchestra.run.trial120.stdout:Access: 2026-02-21 00:19:02.071246690 +0000 2026-02-21T00:24:22.585 INFO:teuthology.orchestra.run.trial120.stdout:Modify: 2026-02-21 00:14:37.962102928 +0000 2026-02-21T00:24:22.585 INFO:teuthology.orchestra.run.trial120.stdout:Change: 2026-02-21 00:14:37.962102928 +0000 2026-02-21T00:24:22.585 INFO:teuthology.orchestra.run.trial120.stdout: Birth: 2026-02-21 00:14:37.962102928 +0000 2026-02-21T00:24:22.586 DEBUG:teuthology.orchestra.run.trial120:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-21T00:24:22.646 INFO:teuthology.orchestra.run.trial120.stderr:1+0 records in 2026-02-21T00:24:22.647 INFO:teuthology.orchestra.run.trial120.stderr:1+0 records out 2026-02-21T00:24:22.647 INFO:teuthology.orchestra.run.trial120.stderr:512 bytes copied, 0.000174414 s, 2.9 MB/s 2026-02-21T00:24:22.648 DEBUG:teuthology.orchestra.run.trial120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-21T00:24:22.705 DEBUG:teuthology.orchestra.run.trial120:> stat /dev/vg_nvme/lv_4 2026-02-21T00:24:22.762 INFO:teuthology.orchestra.run.trial120.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-21T00:24:22.762 INFO:teuthology.orchestra.run.trial120.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T00:24:22.762 INFO:teuthology.orchestra.run.trial120.stdout:Device: 6h/6d Inode: 995 Links: 1 2026-02-21T00:24:22.762 INFO:teuthology.orchestra.run.trial120.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T00:24:22.762 INFO:teuthology.orchestra.run.trial120.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T00:24:22.763 INFO:teuthology.orchestra.run.trial120.stdout:Access: 2026-02-21 00:19:02.071246690 +0000 2026-02-21T00:24:22.763 INFO:teuthology.orchestra.run.trial120.stdout:Modify: 2026-02-21 00:14:38.190094130 +0000 2026-02-21T00:24:22.763 INFO:teuthology.orchestra.run.trial120.stdout:Change: 2026-02-21 00:14:38.190094130 +0000 2026-02-21T00:24:22.763 INFO:teuthology.orchestra.run.trial120.stdout: Birth: 2026-02-21 00:14:38.190094130 +0000 2026-02-21T00:24:22.763 DEBUG:teuthology.orchestra.run.trial120:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-21T00:24:22.826 INFO:teuthology.orchestra.run.trial120.stderr:1+0 records in 2026-02-21T00:24:22.826 INFO:teuthology.orchestra.run.trial120.stderr:1+0 records out 2026-02-21T00:24:22.826 INFO:teuthology.orchestra.run.trial120.stderr:512 bytes copied, 0.000130994 s, 3.9 MB/s 2026-02-21T00:24:22.827 DEBUG:teuthology.orchestra.run.trial120:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-21T00:24:22.884 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T00:24:22.884 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial120.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-21T00:24:22.885 INFO:tasks.ceph:Generating config... 2026-02-21T00:24:22.887 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-21T00:24:22.888 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-21T00:24:22.888 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-21T00:24:22.888 INFO:tasks.ceph:[osd] debug filestore = 0/0 2026-02-21T00:24:22.888 INFO:tasks.ceph:[osd] debug journal = 0/0 2026-02-21T00:24:22.888 INFO:tasks.ceph:[osd] debug ms = 0/0 2026-02-21T00:24:22.888 INFO:tasks.ceph:[osd] debug osd = 0/0 2026-02-21T00:24:22.888 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-21T00:24:22.888 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-21T00:24:22.888 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-21T00:24:22.889 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-21T00:24:22.889 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-21T00:24:22.889 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-21T00:24:22.889 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-21T00:24:22.889 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-21T00:24:22.889 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-21T00:24:22.889 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-21T00:24:22.889 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-21T00:24:22.889 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-21T00:24:22.889 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-21T00:24:22.889 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-21T00:24:22.890 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-21T00:24:22.890 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-21T00:24:22.890 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-21T00:24:22.890 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-21T00:24:22.890 INFO:tasks.ceph:[osd] seastore main device type = RANDOM_BLOCK_SSD 2026-02-21T00:24:22.890 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-21T00:24:22.890 INFO:tasks.ceph:[global] auth client required = none 2026-02-21T00:24:22.890 INFO:tasks.ceph:[global] auth cluster required = none 2026-02-21T00:24:22.890 INFO:tasks.ceph:[global] auth service required = none 2026-02-21T00:24:22.890 INFO:tasks.ceph:[global] auth supported = none 2026-02-21T00:24:22.891 INFO:tasks.ceph:[global] debug asok = 0/0 2026-02-21T00:24:22.891 INFO:tasks.ceph:[global] debug auth = 0/0 2026-02-21T00:24:22.891 INFO:tasks.ceph:[global] debug bdev = 0/0 2026-02-21T00:24:22.891 INFO:tasks.ceph:[global] debug bluefs = 0/0 2026-02-21T00:24:22.891 INFO:tasks.ceph:[global] debug bluestore = 0/0 2026-02-21T00:24:22.891 INFO:tasks.ceph:[global] debug buffer = 0/0 2026-02-21T00:24:22.891 INFO:tasks.ceph:[global] debug civetweb = 0/0 2026-02-21T00:24:22.891 INFO:tasks.ceph:[global] debug client = 0/0 2026-02-21T00:24:22.891 INFO:tasks.ceph:[global] debug compressor = 0/0 2026-02-21T00:24:22.891 INFO:tasks.ceph:[global] debug context = 0/0 2026-02-21T00:24:22.891 INFO:tasks.ceph:[global] debug crush = 0/0 2026-02-21T00:24:22.891 INFO:tasks.ceph:[global] debug crypto = 0/0 2026-02-21T00:24:22.892 INFO:tasks.ceph:[global] debug dpdk = 0/0 2026-02-21T00:24:22.892 INFO:tasks.ceph:[global] debug eventtrace = 0/0 2026-02-21T00:24:22.892 INFO:tasks.ceph:[global] debug filer = 0/0 2026-02-21T00:24:22.892 INFO:tasks.ceph:[global] debug filestore = 0/0 2026-02-21T00:24:22.892 INFO:tasks.ceph:[global] debug finisher = 0/0 2026-02-21T00:24:22.892 INFO:tasks.ceph:[global] debug fuse = 0/0 2026-02-21T00:24:22.892 INFO:tasks.ceph:[global] debug heartbeatmap = 0/0 2026-02-21T00:24:22.892 INFO:tasks.ceph:[global] debug javaclient = 0/0 2026-02-21T00:24:22.892 INFO:tasks.ceph:[global] debug journal = 0/0 2026-02-21T00:24:22.892 INFO:tasks.ceph:[global] debug journaler = 0/0 2026-02-21T00:24:22.892 INFO:tasks.ceph:[global] debug kstore = 0/0 2026-02-21T00:24:22.893 INFO:tasks.ceph:[global] debug leveldb = 0/0 2026-02-21T00:24:22.893 INFO:tasks.ceph:[global] debug lockdep = 0/0 2026-02-21T00:24:22.893 INFO:tasks.ceph:[global] debug mds = 0/0 2026-02-21T00:24:22.893 INFO:tasks.ceph:[global] debug mds balancer = 0/0 2026-02-21T00:24:22.893 INFO:tasks.ceph:[global] debug mds locker = 0/0 2026-02-21T00:24:22.893 INFO:tasks.ceph:[global] debug mds log = 0/0 2026-02-21T00:24:22.893 INFO:tasks.ceph:[global] debug mds log expire = 0/0 2026-02-21T00:24:22.893 INFO:tasks.ceph:[global] debug mds migrator = 0/0 2026-02-21T00:24:22.893 INFO:tasks.ceph:[global] debug memdb = 0/0 2026-02-21T00:24:22.893 INFO:tasks.ceph:[global] debug mgr = 0/0 2026-02-21T00:24:22.893 INFO:tasks.ceph:[global] debug mgrc = 0/0 2026-02-21T00:24:22.894 INFO:tasks.ceph:[global] debug mon = 0/0 2026-02-21T00:24:22.894 INFO:tasks.ceph:[global] debug monc = 0/0 2026-02-21T00:24:22.894 INFO:tasks.ceph:[global] debug ms = 0/0 2026-02-21T00:24:22.894 INFO:tasks.ceph:[global] debug objclass = 0/0 2026-02-21T00:24:22.894 INFO:tasks.ceph:[global] debug objectcacher = 0/0 2026-02-21T00:24:22.894 INFO:tasks.ceph:[global] debug objecter = 0/0 2026-02-21T00:24:22.894 INFO:tasks.ceph:[global] debug optracker = 0/0 2026-02-21T00:24:22.894 INFO:tasks.ceph:[global] debug osd = 0/0 2026-02-21T00:24:22.894 INFO:tasks.ceph:[global] debug paxos = 0/0 2026-02-21T00:24:22.894 INFO:tasks.ceph:[global] debug perfcounter = 0/0 2026-02-21T00:24:22.894 INFO:tasks.ceph:[global] debug rados = 0/0 2026-02-21T00:24:22.894 INFO:tasks.ceph:[global] debug rbd = 0/0 2026-02-21T00:24:22.895 INFO:tasks.ceph:[global] debug rbd mirror = 0/0 2026-02-21T00:24:22.895 INFO:tasks.ceph:[global] debug rbd replay = 0/0 2026-02-21T00:24:22.895 INFO:tasks.ceph:[global] debug refs = 0/0 2026-02-21T00:24:22.895 INFO:tasks.ceph:[global] debug rgw = 0/0 2026-02-21T00:24:22.895 INFO:tasks.ceph:[global] debug rgw lifecycle = 0/0 2026-02-21T00:24:22.895 INFO:tasks.ceph:[global] debug rgw notification = 0/0 2026-02-21T00:24:22.895 INFO:tasks.ceph:[global] debug rgw sync = 0/0 2026-02-21T00:24:22.895 INFO:tasks.ceph:[global] debug rocksdb = 0/0 2026-02-21T00:24:22.895 INFO:tasks.ceph:[global] debug striper = 0/0 2026-02-21T00:24:22.895 INFO:tasks.ceph:[global] debug throttle = 0/0 2026-02-21T00:24:22.895 INFO:tasks.ceph:[global] debug timer = 0/0 2026-02-21T00:24:22.896 INFO:tasks.ceph:[global] debug tp = 0/0 2026-02-21T00:24:22.896 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-21T00:24:22.896 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-21T00:24:22.896 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-21T00:24:22.896 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-21T00:24:22.896 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-21T00:24:22.896 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-21T00:24:22.896 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-21T00:24:22.896 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-21T00:24:22.896 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-21T00:24:22.896 INFO:tasks.ceph:[mon] debug mon = 0/0 2026-02-21T00:24:22.897 INFO:tasks.ceph:[mon] debug ms = 0/0 2026-02-21T00:24:22.897 INFO:tasks.ceph:[mon] debug paxos = 0/0 2026-02-21T00:24:22.897 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-21T00:24:22.897 INFO:tasks.ceph:Setting up mon.a... 2026-02-21T00:24:22.897 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-21T00:24:22.965 INFO:teuthology.orchestra.run.trial120.stdout:creating /etc/ceph/ceph.keyring 2026-02-21T00:24:22.968 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-21T00:24:23.052 DEBUG:teuthology.orchestra.run.trial120:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T00:24:23.116 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.120')] 2026-02-21T00:24:23.117 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'none', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.120', 'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': '0/0', 'debug mon': '0/0', 'debug paxos': '0/0', 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-21T00:24:23.119 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:24:23.119 DEBUG:teuthology.orchestra.run.trial120:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-21T00:24:23.175 DEBUG:teuthology.orchestra.run.trial120:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.120 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:24:23.256 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:23.254+0000 7f830ae39f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:23.256 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:23.254+0000 7f830ae39f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:23.256 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:23.254+0000 7f830ae39f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:23.257 INFO:teuthology.orchestra.run.trial120.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:24:23.258 INFO:teuthology.orchestra.run.trial120.stdout:monmaptool: generated fsid 1a3b4641-234e-4a24-ae80-55b3d108e591 2026-02-21T00:24:23.258 INFO:teuthology.orchestra.run.trial120.stdout:setting min_mon_release = tentacle 2026-02-21T00:24:23.258 INFO:teuthology.orchestra.run.trial120.stdout:epoch 0 2026-02-21T00:24:23.258 INFO:teuthology.orchestra.run.trial120.stdout:fsid 1a3b4641-234e-4a24-ae80-55b3d108e591 2026-02-21T00:24:23.258 INFO:teuthology.orchestra.run.trial120.stdout:last_changed 2026-02-21T00:24:23.255435+0000 2026-02-21T00:24:23.258 INFO:teuthology.orchestra.run.trial120.stdout:created 2026-02-21T00:24:23.255435+0000 2026-02-21T00:24:23.258 INFO:teuthology.orchestra.run.trial120.stdout:min_mon_release 20 (tentacle) 2026-02-21T00:24:23.258 INFO:teuthology.orchestra.run.trial120.stdout:election_strategy: 1 2026-02-21T00:24:23.259 INFO:teuthology.orchestra.run.trial120.stdout:0: [v2:10.20.193.120:3300/0,v1:10.20.193.120:6789/0] mon.a 2026-02-21T00:24:23.259 INFO:teuthology.orchestra.run.trial120.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-21T00:24:23.260 DEBUG:teuthology.orchestra.run.trial120:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-21T00:24:23.316 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 1a3b4641-234e-4a24-ae80-55b3d108e591... 2026-02-21T00:24:23.318 DEBUG:teuthology.orchestra.run.trial120:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-21T00:24:23.396 INFO:teuthology.orchestra.run.trial120.stdout:[global] 2026-02-21T00:24:23.397 INFO:teuthology.orchestra.run.trial120.stdout: chdir = "" 2026-02-21T00:24:23.397 INFO:teuthology.orchestra.run.trial120.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-21T00:24:23.397 INFO:teuthology.orchestra.run.trial120.stdout: auth supported = none 2026-02-21T00:24:23.397 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.397 INFO:teuthology.orchestra.run.trial120.stdout: filestore xattr use omap = true 2026-02-21T00:24:23.397 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.397 INFO:teuthology.orchestra.run.trial120.stdout: mon clock drift allowed = 1.000 2026-02-21T00:24:23.397 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.398 INFO:teuthology.orchestra.run.trial120.stdout: osd crush chooseleaf type = 0 2026-02-21T00:24:23.398 INFO:teuthology.orchestra.run.trial120.stdout: auth debug = true 2026-02-21T00:24:23.398 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.398 INFO:teuthology.orchestra.run.trial120.stdout: ms die on old message = true 2026-02-21T00:24:23.398 INFO:teuthology.orchestra.run.trial120.stdout: ms die on bug = true 2026-02-21T00:24:23.398 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.398 INFO:teuthology.orchestra.run.trial120.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-21T00:24:23.398 INFO:teuthology.orchestra.run.trial120.stdout: mon pg warn max object skew = 0 2026-02-21T00:24:23.398 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.398 INFO:teuthology.orchestra.run.trial120.stdout: # disable pg_autoscaler by default for new pools 2026-02-21T00:24:23.399 INFO:teuthology.orchestra.run.trial120.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-21T00:24:23.399 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.399 INFO:teuthology.orchestra.run.trial120.stdout: osd pool default size = 2 2026-02-21T00:24:23.399 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.399 INFO:teuthology.orchestra.run.trial120.stdout: mon osd allow primary affinity = true 2026-02-21T00:24:23.399 INFO:teuthology.orchestra.run.trial120.stdout: mon osd allow pg remap = true 2026-02-21T00:24:23.399 INFO:teuthology.orchestra.run.trial120.stdout: mon warn on legacy crush tunables = false 2026-02-21T00:24:23.399 INFO:teuthology.orchestra.run.trial120.stdout: mon warn on crush straw calc version zero = false 2026-02-21T00:24:23.399 INFO:teuthology.orchestra.run.trial120.stdout: mon warn on no sortbitwise = false 2026-02-21T00:24:23.399 INFO:teuthology.orchestra.run.trial120.stdout: mon warn on osd down out interval zero = false 2026-02-21T00:24:23.399 INFO:teuthology.orchestra.run.trial120.stdout: mon warn on too few osds = false 2026-02-21T00:24:23.400 INFO:teuthology.orchestra.run.trial120.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-21T00:24:23.400 INFO:teuthology.orchestra.run.trial120.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-21T00:24:23.400 INFO:teuthology.orchestra.run.trial120.stdout: mon_allow_pool_size_one = true 2026-02-21T00:24:23.400 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.400 INFO:teuthology.orchestra.run.trial120.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-21T00:24:23.400 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.400 INFO:teuthology.orchestra.run.trial120.stdout: osd default data pool replay window = 5 2026-02-21T00:24:23.400 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.400 INFO:teuthology.orchestra.run.trial120.stdout: mon allow pool delete = true 2026-02-21T00:24:23.400 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.401 INFO:teuthology.orchestra.run.trial120.stdout: mon cluster log file level = debug 2026-02-21T00:24:23.401 INFO:teuthology.orchestra.run.trial120.stdout: debug asserts on shutdown = true 2026-02-21T00:24:23.401 INFO:teuthology.orchestra.run.trial120.stdout: mon health detail to clog = false 2026-02-21T00:24:23.401 INFO:teuthology.orchestra.run.trial120.stdout: mon host = 10.20.193.120 2026-02-21T00:24:23.401 INFO:teuthology.orchestra.run.trial120.stdout: auth client required = none 2026-02-21T00:24:23.401 INFO:teuthology.orchestra.run.trial120.stdout: auth cluster required = none 2026-02-21T00:24:23.401 INFO:teuthology.orchestra.run.trial120.stdout: auth service required = none 2026-02-21T00:24:23.401 INFO:teuthology.orchestra.run.trial120.stdout: debug asok = 0/0 2026-02-21T00:24:23.401 INFO:teuthology.orchestra.run.trial120.stdout: debug auth = 0/0 2026-02-21T00:24:23.402 INFO:teuthology.orchestra.run.trial120.stdout: debug bdev = 0/0 2026-02-21T00:24:23.402 INFO:teuthology.orchestra.run.trial120.stdout: debug bluefs = 0/0 2026-02-21T00:24:23.402 INFO:teuthology.orchestra.run.trial120.stdout: debug bluestore = 0/0 2026-02-21T00:24:23.402 INFO:teuthology.orchestra.run.trial120.stdout: debug buffer = 0/0 2026-02-21T00:24:23.402 INFO:teuthology.orchestra.run.trial120.stdout: debug civetweb = 0/0 2026-02-21T00:24:23.402 INFO:teuthology.orchestra.run.trial120.stdout: debug client = 0/0 2026-02-21T00:24:23.402 INFO:teuthology.orchestra.run.trial120.stdout: debug compressor = 0/0 2026-02-21T00:24:23.402 INFO:teuthology.orchestra.run.trial120.stdout: debug context = 0/0 2026-02-21T00:24:23.402 INFO:teuthology.orchestra.run.trial120.stdout: debug crush = 0/0 2026-02-21T00:24:23.402 INFO:teuthology.orchestra.run.trial120.stdout: debug crypto = 0/0 2026-02-21T00:24:23.402 INFO:teuthology.orchestra.run.trial120.stdout: debug dpdk = 0/0 2026-02-21T00:24:23.402 INFO:teuthology.orchestra.run.trial120.stdout: debug eventtrace = 0/0 2026-02-21T00:24:23.403 INFO:teuthology.orchestra.run.trial120.stdout: debug filer = 0/0 2026-02-21T00:24:23.403 INFO:teuthology.orchestra.run.trial120.stdout: debug filestore = 0/0 2026-02-21T00:24:23.403 INFO:teuthology.orchestra.run.trial120.stdout: debug finisher = 0/0 2026-02-21T00:24:23.403 INFO:teuthology.orchestra.run.trial120.stdout: debug fuse = 0/0 2026-02-21T00:24:23.403 INFO:teuthology.orchestra.run.trial120.stdout: debug heartbeatmap = 0/0 2026-02-21T00:24:23.403 INFO:teuthology.orchestra.run.trial120.stdout: debug javaclient = 0/0 2026-02-21T00:24:23.403 INFO:teuthology.orchestra.run.trial120.stdout: debug journal = 0/0 2026-02-21T00:24:23.403 INFO:teuthology.orchestra.run.trial120.stdout: debug journaler = 0/0 2026-02-21T00:24:23.403 INFO:teuthology.orchestra.run.trial120.stdout: debug kstore = 0/0 2026-02-21T00:24:23.403 INFO:teuthology.orchestra.run.trial120.stdout: debug leveldb = 0/0 2026-02-21T00:24:23.403 INFO:teuthology.orchestra.run.trial120.stdout: debug lockdep = 0/0 2026-02-21T00:24:23.403 INFO:teuthology.orchestra.run.trial120.stdout: debug mds = 0/0 2026-02-21T00:24:23.403 INFO:teuthology.orchestra.run.trial120.stdout: debug mds balancer = 0/0 2026-02-21T00:24:23.403 INFO:teuthology.orchestra.run.trial120.stdout: debug mds locker = 0/0 2026-02-21T00:24:23.404 INFO:teuthology.orchestra.run.trial120.stdout: debug mds log = 0/0 2026-02-21T00:24:23.404 INFO:teuthology.orchestra.run.trial120.stdout: debug mds log expire = 0/0 2026-02-21T00:24:23.404 INFO:teuthology.orchestra.run.trial120.stdout: debug mds migrator = 0/0 2026-02-21T00:24:23.404 INFO:teuthology.orchestra.run.trial120.stdout: debug memdb = 0/0 2026-02-21T00:24:23.404 INFO:teuthology.orchestra.run.trial120.stdout: debug mgr = 0/0 2026-02-21T00:24:23.404 INFO:teuthology.orchestra.run.trial120.stdout: debug mgrc = 0/0 2026-02-21T00:24:23.404 INFO:teuthology.orchestra.run.trial120.stdout: debug mon = 0/0 2026-02-21T00:24:23.404 INFO:teuthology.orchestra.run.trial120.stdout: debug monc = 0/0 2026-02-21T00:24:23.404 INFO:teuthology.orchestra.run.trial120.stdout: debug ms = 0/0 2026-02-21T00:24:23.404 INFO:teuthology.orchestra.run.trial120.stdout: debug objclass = 0/0 2026-02-21T00:24:23.404 INFO:teuthology.orchestra.run.trial120.stdout: debug objectcacher = 0/0 2026-02-21T00:24:23.404 INFO:teuthology.orchestra.run.trial120.stdout: debug objecter = 0/0 2026-02-21T00:24:23.404 INFO:teuthology.orchestra.run.trial120.stdout: debug optracker = 0/0 2026-02-21T00:24:23.405 INFO:teuthology.orchestra.run.trial120.stdout: debug osd = 0/0 2026-02-21T00:24:23.405 INFO:teuthology.orchestra.run.trial120.stdout: debug paxos = 0/0 2026-02-21T00:24:23.405 INFO:teuthology.orchestra.run.trial120.stdout: debug perfcounter = 0/0 2026-02-21T00:24:23.405 INFO:teuthology.orchestra.run.trial120.stdout: debug rados = 0/0 2026-02-21T00:24:23.405 INFO:teuthology.orchestra.run.trial120.stdout: debug rbd = 0/0 2026-02-21T00:24:23.405 INFO:teuthology.orchestra.run.trial120.stdout: debug rbd mirror = 0/0 2026-02-21T00:24:23.405 INFO:teuthology.orchestra.run.trial120.stdout: debug rbd replay = 0/0 2026-02-21T00:24:23.405 INFO:teuthology.orchestra.run.trial120.stdout: debug refs = 0/0 2026-02-21T00:24:23.405 INFO:teuthology.orchestra.run.trial120.stdout: debug rgw = 0/0 2026-02-21T00:24:23.405 INFO:teuthology.orchestra.run.trial120.stdout: debug rgw lifecycle = 0/0 2026-02-21T00:24:23.405 INFO:teuthology.orchestra.run.trial120.stdout: debug rgw notification = 0/0 2026-02-21T00:24:23.405 INFO:teuthology.orchestra.run.trial120.stdout: debug rgw sync = 0/0 2026-02-21T00:24:23.405 INFO:teuthology.orchestra.run.trial120.stdout: debug rocksdb = 0/0 2026-02-21T00:24:23.406 INFO:teuthology.orchestra.run.trial120.stdout: debug striper = 0/0 2026-02-21T00:24:23.406 INFO:teuthology.orchestra.run.trial120.stdout: debug throttle = 0/0 2026-02-21T00:24:23.406 INFO:teuthology.orchestra.run.trial120.stdout: debug timer = 0/0 2026-02-21T00:24:23.406 INFO:teuthology.orchestra.run.trial120.stdout: debug tp = 0/0 2026-02-21T00:24:23.406 INFO:teuthology.orchestra.run.trial120.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-21T00:24:23.406 INFO:teuthology.orchestra.run.trial120.stdout: ms client mode = crc 2026-02-21T00:24:23.406 INFO:teuthology.orchestra.run.trial120.stdout: ms cluster mode = crc 2026-02-21T00:24:23.406 INFO:teuthology.orchestra.run.trial120.stdout: ms mon client mode = crc 2026-02-21T00:24:23.406 INFO:teuthology.orchestra.run.trial120.stdout: ms mon cluster mode = crc 2026-02-21T00:24:23.406 INFO:teuthology.orchestra.run.trial120.stdout: ms mon service mode = crc 2026-02-21T00:24:23.406 INFO:teuthology.orchestra.run.trial120.stdout: ms service mode = crc 2026-02-21T00:24:23.406 INFO:teuthology.orchestra.run.trial120.stdout: fsid = 1a3b4641-234e-4a24-ae80-55b3d108e591 2026-02-21T00:24:23.407 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.407 INFO:teuthology.orchestra.run.trial120.stdout:[osd] 2026-02-21T00:24:23.407 INFO:teuthology.orchestra.run.trial120.stdout: osd journal size = 100 2026-02-21T00:24:23.407 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.407 INFO:teuthology.orchestra.run.trial120.stdout: osd scrub load threshold = 5.0 2026-02-21T00:24:23.407 INFO:teuthology.orchestra.run.trial120.stdout: osd scrub max interval = 600 2026-02-21T00:24:23.407 INFO:teuthology.orchestra.run.trial120.stdout: osd mclock profile = high_recovery_ops 2026-02-21T00:24:23.407 INFO:teuthology.orchestra.run.trial120.stdout: osd mclock skip benchmark = true 2026-02-21T00:24:23.407 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.407 INFO:teuthology.orchestra.run.trial120.stdout: osd recover clone overlap = true 2026-02-21T00:24:23.407 INFO:teuthology.orchestra.run.trial120.stdout: osd recovery max chunk = 1048576 2026-02-21T00:24:23.408 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.408 INFO:teuthology.orchestra.run.trial120.stdout: osd debug shutdown = true 2026-02-21T00:24:23.408 INFO:teuthology.orchestra.run.trial120.stdout: osd debug op order = true 2026-02-21T00:24:23.408 INFO:teuthology.orchestra.run.trial120.stdout: osd debug verify stray on activate = true 2026-02-21T00:24:23.408 INFO:teuthology.orchestra.run.trial120.stdout: osd debug trim objects = true 2026-02-21T00:24:23.408 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.408 INFO:teuthology.orchestra.run.trial120.stdout: osd open classes on start = true 2026-02-21T00:24:23.408 INFO:teuthology.orchestra.run.trial120.stdout: osd debug pg log writeout = true 2026-02-21T00:24:23.408 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.408 INFO:teuthology.orchestra.run.trial120.stdout: osd deep scrub update digest min age = 30 2026-02-21T00:24:23.408 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.409 INFO:teuthology.orchestra.run.trial120.stdout: osd map max advance = 10 2026-02-21T00:24:23.409 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.409 INFO:teuthology.orchestra.run.trial120.stdout: journal zero on create = true 2026-02-21T00:24:23.409 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.409 INFO:teuthology.orchestra.run.trial120.stdout: filestore ondisk finisher threads = 3 2026-02-21T00:24:23.409 INFO:teuthology.orchestra.run.trial120.stdout: filestore apply finisher threads = 3 2026-02-21T00:24:23.409 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.409 INFO:teuthology.orchestra.run.trial120.stdout: bdev debug aio = true 2026-02-21T00:24:23.409 INFO:teuthology.orchestra.run.trial120.stdout: osd debug misdirected ops = true 2026-02-21T00:24:23.409 INFO:teuthology.orchestra.run.trial120.stdout: debug monc = 20 2026-02-21T00:24:23.409 INFO:teuthology.orchestra.run.trial120.stdout: crimson cpu num = 4 2026-02-21T00:24:23.409 INFO:teuthology.orchestra.run.trial120.stdout: crimson osd obc lru size = 10 2026-02-21T00:24:23.409 INFO:teuthology.orchestra.run.trial120.stdout: debug filestore = 0/0 2026-02-21T00:24:23.410 INFO:teuthology.orchestra.run.trial120.stdout: debug journal = 0/0 2026-02-21T00:24:23.410 INFO:teuthology.orchestra.run.trial120.stdout: debug ms = 0/0 2026-02-21T00:24:23.410 INFO:teuthology.orchestra.run.trial120.stdout: debug osd = 0/0 2026-02-21T00:24:23.410 INFO:teuthology.orchestra.run.trial120.stdout: debug seastore = 20 2026-02-21T00:24:23.410 INFO:teuthology.orchestra.run.trial120.stdout: debug seastore backref = 20 2026-02-21T00:24:23.410 INFO:teuthology.orchestra.run.trial120.stdout: debug seastore cache = 20 2026-02-21T00:24:23.410 INFO:teuthology.orchestra.run.trial120.stdout: debug seastore cleaner = 20 2026-02-21T00:24:23.410 INFO:teuthology.orchestra.run.trial120.stdout: debug seastore device = 20 2026-02-21T00:24:23.410 INFO:teuthology.orchestra.run.trial120.stdout: debug seastore epm = 20 2026-02-21T00:24:23.410 INFO:teuthology.orchestra.run.trial120.stdout: debug seastore fixedkv tree = 20 2026-02-21T00:24:23.410 INFO:teuthology.orchestra.run.trial120.stdout: debug seastore journal = 20 2026-02-21T00:24:23.410 INFO:teuthology.orchestra.run.trial120.stdout: debug seastore lba = 20 2026-02-21T00:24:23.410 INFO:teuthology.orchestra.run.trial120.stdout: debug seastore odata = 20 2026-02-21T00:24:23.411 INFO:teuthology.orchestra.run.trial120.stdout: debug seastore omap = 20 2026-02-21T00:24:23.411 INFO:teuthology.orchestra.run.trial120.stdout: debug seastore onode = 20 2026-02-21T00:24:23.411 INFO:teuthology.orchestra.run.trial120.stdout: debug seastore t = 20 2026-02-21T00:24:23.411 INFO:teuthology.orchestra.run.trial120.stdout: debug seastore tm = 20 2026-02-21T00:24:23.411 INFO:teuthology.orchestra.run.trial120.stdout: osd objectstore = seastore 2026-02-21T00:24:23.411 INFO:teuthology.orchestra.run.trial120.stdout: osd shutdown pgref assert = True 2026-02-21T00:24:23.411 INFO:teuthology.orchestra.run.trial120.stdout: seastore cachepin size pershard = 64M 2026-02-21T00:24:23.411 INFO:teuthology.orchestra.run.trial120.stdout: seastore cachepin type = 2Q 2026-02-21T00:24:23.411 INFO:teuthology.orchestra.run.trial120.stdout: seastore main device type = RANDOM_BLOCK_SSD 2026-02-21T00:24:23.411 INFO:teuthology.orchestra.run.trial120.stdout: seastore max concurrent transactions = 8 2026-02-21T00:24:23.412 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.412 INFO:teuthology.orchestra.run.trial120.stdout:[mgr] 2026-02-21T00:24:23.412 INFO:teuthology.orchestra.run.trial120.stdout: debug ms = 1 2026-02-21T00:24:23.412 INFO:teuthology.orchestra.run.trial120.stdout: debug mgr = 20 2026-02-21T00:24:23.412 INFO:teuthology.orchestra.run.trial120.stdout: debug mon = 20 2026-02-21T00:24:23.412 INFO:teuthology.orchestra.run.trial120.stdout: debug auth = 20 2026-02-21T00:24:23.412 INFO:teuthology.orchestra.run.trial120.stdout: mon reweight min pgs per osd = 4 2026-02-21T00:24:23.412 INFO:teuthology.orchestra.run.trial120.stdout: mon reweight min bytes per osd = 10 2026-02-21T00:24:23.412 INFO:teuthology.orchestra.run.trial120.stdout: mgr/telemetry/nag = false 2026-02-21T00:24:23.412 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.413 INFO:teuthology.orchestra.run.trial120.stdout:[mon] 2026-02-21T00:24:23.413 INFO:teuthology.orchestra.run.trial120.stdout: debug ms = 0/0 2026-02-21T00:24:23.413 INFO:teuthology.orchestra.run.trial120.stdout: debug mon = 0/0 2026-02-21T00:24:23.413 INFO:teuthology.orchestra.run.trial120.stdout: debug paxos = 0/0 2026-02-21T00:24:23.413 INFO:teuthology.orchestra.run.trial120.stdout: debug auth = 20 2026-02-21T00:24:23.413 INFO:teuthology.orchestra.run.trial120.stdout: mon data avail warn = 5 2026-02-21T00:24:23.413 INFO:teuthology.orchestra.run.trial120.stdout: mon mgr mkfs grace = 240 2026-02-21T00:24:23.413 INFO:teuthology.orchestra.run.trial120.stdout: mon reweight min pgs per osd = 4 2026-02-21T00:24:23.413 INFO:teuthology.orchestra.run.trial120.stdout: mon osd reporter subtree level = osd 2026-02-21T00:24:23.413 INFO:teuthology.orchestra.run.trial120.stdout: mon osd prime pg temp = true 2026-02-21T00:24:23.413 INFO:teuthology.orchestra.run.trial120.stdout: mon reweight min bytes per osd = 10 2026-02-21T00:24:23.414 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.414 INFO:teuthology.orchestra.run.trial120.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-21T00:24:23.414 INFO:teuthology.orchestra.run.trial120.stdout: auth mon ticket ttl = 660 # 11m 2026-02-21T00:24:23.414 INFO:teuthology.orchestra.run.trial120.stdout: auth service ticket ttl = 240 # 4m 2026-02-21T00:24:23.414 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.414 INFO:teuthology.orchestra.run.trial120.stdout: # don't complain about insecure global_id in the test suite 2026-02-21T00:24:23.414 INFO:teuthology.orchestra.run.trial120.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-21T00:24:23.414 INFO:teuthology.orchestra.run.trial120.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-21T00:24:23.414 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.414 INFO:teuthology.orchestra.run.trial120.stdout: # 1m isn't quite enough 2026-02-21T00:24:23.414 INFO:teuthology.orchestra.run.trial120.stdout: mon_down_mkfs_grace = 2m 2026-02-21T00:24:23.414 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.415 INFO:teuthology.orchestra.run.trial120.stdout: mon_warn_on_filestore_osds = false 2026-02-21T00:24:23.415 INFO:teuthology.orchestra.run.trial120.stdout: osd pool default crimson = True 2026-02-21T00:24:23.415 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:23.415 INFO:teuthology.orchestra.run.trial120.stdout:[client] 2026-02-21T00:24:23.415 INFO:teuthology.orchestra.run.trial120.stdout: rgw cache enabled = true 2026-02-21T00:24:23.415 INFO:teuthology.orchestra.run.trial120.stdout: rgw enable ops log = true 2026-02-21T00:24:23.415 INFO:teuthology.orchestra.run.trial120.stdout: rgw enable usage log = true 2026-02-21T00:24:23.415 INFO:teuthology.orchestra.run.trial120.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-21T00:24:23.415 INFO:teuthology.orchestra.run.trial120.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-21T00:24:23.415 INFO:teuthology.orchestra.run.trial120.stdout:[mon.a] 2026-02-21T00:24:23.416 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-21T00:24:23.417 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-21T00:24:23.507 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-21T00:24:23.508 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:24:23.508 DEBUG:teuthology.orchestra.run.trial120:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-21T00:24:23.525 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:24:23.525 DEBUG:teuthology.orchestra.run.trial120:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-21T00:24:23.583 INFO:tasks.ceph:Sending monmap to node ubuntu@trial120.front.sepia.ceph.com 2026-02-21T00:24:23.583 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:24:23.583 DEBUG:teuthology.orchestra.run.trial120:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-21T00:24:23.583 DEBUG:teuthology.orchestra.run.trial120:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T00:24:23.655 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:24:23.655 DEBUG:teuthology.orchestra.run.trial120:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:24:23.712 INFO:tasks.ceph:Setting up mon nodes... 2026-02-21T00:24:23.712 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-21T00:24:23.713 DEBUG:teuthology.orchestra.run.trial120:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-21T00:24:23.802 INFO:teuthology.orchestra.run.trial120.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-21T00:24:23.804 INFO:tasks.ceph:Setting up mds nodes... 2026-02-21T00:24:23.805 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-21T00:24:23.805 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-21T00:24:23.846 INFO:teuthology.orchestra.run.trial120.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-21T00:24:23.857 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-21T00:24:23.858 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial120.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-21T00:24:23.858 DEBUG:teuthology.orchestra.run.trial120:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-21T00:24:23.923 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T00:24:23.923 INFO:tasks.ceph:role: osd.0 2026-02-21T00:24:23.923 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial120.front.sepia.ceph.com 2026-02-21T00:24:23.923 DEBUG:teuthology.orchestra.run.trial120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-21T00:24:23.988 INFO:teuthology.orchestra.run.trial120.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T00:24:23.988 INFO:teuthology.orchestra.run.trial120.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T00:24:23.989 INFO:teuthology.orchestra.run.trial120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T00:24:23.989 INFO:teuthology.orchestra.run.trial120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T00:24:23.989 INFO:teuthology.orchestra.run.trial120.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T00:24:23.989 INFO:teuthology.orchestra.run.trial120.stdout: = sunit=0 swidth=0 blks 2026-02-21T00:24:23.989 INFO:teuthology.orchestra.run.trial120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T00:24:23.989 INFO:teuthology.orchestra.run.trial120.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T00:24:23.989 INFO:teuthology.orchestra.run.trial120.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T00:24:23.989 INFO:teuthology.orchestra.run.trial120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T00:24:24.558 INFO:teuthology.orchestra.run.trial120.stdout:Discarding blocks...Done. 2026-02-21T00:24:24.560 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial120.front.sepia.ceph.com -o noatime 2026-02-21T00:24:24.560 DEBUG:teuthology.orchestra.run.trial120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-21T00:24:24.631 DEBUG:teuthology.orchestra.run.trial120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-21T00:24:24.700 DEBUG:teuthology.orchestra.run.trial120:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-21T00:24:24.763 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T00:24:24.763 INFO:tasks.ceph:role: osd.1 2026-02-21T00:24:24.763 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial120.front.sepia.ceph.com 2026-02-21T00:24:24.763 DEBUG:teuthology.orchestra.run.trial120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-21T00:24:24.828 INFO:teuthology.orchestra.run.trial120.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T00:24:24.829 INFO:teuthology.orchestra.run.trial120.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T00:24:24.829 INFO:teuthology.orchestra.run.trial120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T00:24:24.829 INFO:teuthology.orchestra.run.trial120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T00:24:24.829 INFO:teuthology.orchestra.run.trial120.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T00:24:24.829 INFO:teuthology.orchestra.run.trial120.stdout: = sunit=0 swidth=0 blks 2026-02-21T00:24:24.829 INFO:teuthology.orchestra.run.trial120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T00:24:24.829 INFO:teuthology.orchestra.run.trial120.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T00:24:24.829 INFO:teuthology.orchestra.run.trial120.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T00:24:24.829 INFO:teuthology.orchestra.run.trial120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T00:24:25.388 INFO:teuthology.orchestra.run.trial120.stdout:Discarding blocks...Done. 2026-02-21T00:24:25.391 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial120.front.sepia.ceph.com -o noatime 2026-02-21T00:24:25.391 DEBUG:teuthology.orchestra.run.trial120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-21T00:24:25.458 DEBUG:teuthology.orchestra.run.trial120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-21T00:24:25.525 DEBUG:teuthology.orchestra.run.trial120:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-21T00:24:25.589 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T00:24:25.590 INFO:tasks.ceph:role: osd.2 2026-02-21T00:24:25.590 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial120.front.sepia.ceph.com 2026-02-21T00:24:25.590 DEBUG:teuthology.orchestra.run.trial120:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-21T00:24:25.655 INFO:teuthology.orchestra.run.trial120.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T00:24:25.655 INFO:teuthology.orchestra.run.trial120.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T00:24:25.655 INFO:teuthology.orchestra.run.trial120.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T00:24:25.655 INFO:teuthology.orchestra.run.trial120.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T00:24:25.655 INFO:teuthology.orchestra.run.trial120.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T00:24:25.655 INFO:teuthology.orchestra.run.trial120.stdout: = sunit=0 swidth=0 blks 2026-02-21T00:24:25.656 INFO:teuthology.orchestra.run.trial120.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T00:24:25.656 INFO:teuthology.orchestra.run.trial120.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T00:24:25.656 INFO:teuthology.orchestra.run.trial120.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T00:24:25.656 INFO:teuthology.orchestra.run.trial120.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T00:24:26.217 INFO:teuthology.orchestra.run.trial120.stdout:Discarding blocks...Done. 2026-02-21T00:24:26.220 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial120.front.sepia.ceph.com -o noatime 2026-02-21T00:24:26.220 DEBUG:teuthology.orchestra.run.trial120:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-21T00:24:26.289 DEBUG:teuthology.orchestra.run.trial120:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-21T00:24:26.356 DEBUG:teuthology.orchestra.run.trial120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:24:26.510 INFO:teuthology.orchestra.run.trial120.stderr:INFO  osd - OSD::main: parsing early config 2026-02-21T00:24:26.567 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:26,566 seastar - Reactor backend: linux-aio 2026-02-21T00:24:26.580 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:26,579 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:24:26.580 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:26,579 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:24:26.582 INFO:teuthology.orchestra.run.trial120.stderr:==38241==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:24:26.642 INFO:teuthology.orchestra.run.trial120.stderr:DEBUG 2026-02-21 00:24:26,641 [shard 0:main] seastar - reactor::drain 2026-02-21T00:24:28.729 INFO:teuthology.orchestra.run.trial120.stderr:----------------------------------------------------- 2026-02-21T00:24:28.729 INFO:teuthology.orchestra.run.trial120.stderr:Suppressions used: 2026-02-21T00:24:28.729 INFO:teuthology.orchestra.run.trial120.stderr: count bytes template 2026-02-21T00:24:28.730 INFO:teuthology.orchestra.run.trial120.stderr: 1 8 MallocExtension::Register 2026-02-21T00:24:28.730 INFO:teuthology.orchestra.run.trial120.stderr:----------------------------------------------------- 2026-02-21T00:24:28.730 INFO:teuthology.orchestra.run.trial120.stderr: 2026-02-21T00:24:28.748 INFO:teuthology.orchestra.run.trial120.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-21T00:24:28.748 INFO:teuthology.orchestra.run.trial120.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-21T00:24:28.790 INFO:teuthology.orchestra.run.trial120.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-21T00:24:28.790 INFO:teuthology.orchestra.run.trial120.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-21T00:24:28.797 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:28,796 seastar - Reactor backend: linux-aio 2026-02-21T00:24:28.811 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:28,810 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:24:28.811 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:28,810 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:24:28.815 INFO:teuthology.orchestra.run.trial120.stderr:==38239==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:24:28.815 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:28,814 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-21T00:24:28.972 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x9fe4c20 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5f05 0x9fe6403 0x9fe6a33 0x9fe6b9d 0xa0c1f5d 0xa0d4d60 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:28.972 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x7af2192 0x9ec96d6 0xa0a52c9 0xa0a7e41 0xa0a8052 0xa0a80fe 0xa0d163d 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:28.972 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16b36a /lib64/libasan.so.8+0xb0541 /lib64/libasan.so.8+0xb1437 0xba66a7 0x10d1d1e 0xf741d0 0x9fe874e 0x9fe908d 0xa0ce025 0xa0d870b 0x7bc71f4 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:29.004 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x7f51639 0x7ebcec0 0xa0d8726 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:24:29.064 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc3cc /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0xa04034d 0xa04040c 0xa0406b4 0xa0c73db 0xa0c7b9a 0xa0c80e0 0xa0d236e 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:29.064 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0xe55926 0x1168113 0xf741c5 0xa0c28bd 0xa0c2eaa 0xa0c322d 0xa0d3384 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:29.064 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xea791 0xe6c75a 0xe6cb15 0xe6cd7e 0xa0d87e9 0x7bc71f4 0x7effe85 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:29.179 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x10b6c6 0xe6c75a 0xe6cb15 0x9f30d47 0xa0a9a5b 0xa0a9b55 0x7eff032 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:29.179 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0xa0c806b 0xa0d236e 0xa0d67e7 0xa096d59 0xa095d59 0x7efff32 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:29.180 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x3422abc 0x3422c92 0x3422f49 0x342303e 0xa0d4d7d 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:31.360 INFO:teuthology.orchestra.run.trial120.stderr:----------------------------------------------------- 2026-02-21T00:24:31.360 INFO:teuthology.orchestra.run.trial120.stderr:Suppressions used: 2026-02-21T00:24:31.361 INFO:teuthology.orchestra.run.trial120.stderr: count bytes template 2026-02-21T00:24:31.361 INFO:teuthology.orchestra.run.trial120.stderr: 1 8 MallocExtension::Register 2026-02-21T00:24:31.361 INFO:teuthology.orchestra.run.trial120.stderr:----------------------------------------------------- 2026-02-21T00:24:31.361 INFO:teuthology.orchestra.run.trial120.stderr: 2026-02-21T00:24:31.394 DEBUG:teuthology.orchestra.run.trial120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-21T00:24:31.419 DEBUG:teuthology.orchestra.run.trial120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:24:31.571 INFO:teuthology.orchestra.run.trial120.stderr:INFO  osd - OSD::main: parsing early config 2026-02-21T00:24:31.631 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:31,630 seastar - Reactor backend: linux-aio 2026-02-21T00:24:31.646 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:31,645 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:24:31.646 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:31,645 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:24:31.648 INFO:teuthology.orchestra.run.trial120.stderr:==38311==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:24:31.707 INFO:teuthology.orchestra.run.trial120.stderr:DEBUG 2026-02-21 00:24:31,706 [shard 0:main] seastar - reactor::drain 2026-02-21T00:24:33.777 INFO:teuthology.orchestra.run.trial120.stderr:----------------------------------------------------- 2026-02-21T00:24:33.777 INFO:teuthology.orchestra.run.trial120.stderr:Suppressions used: 2026-02-21T00:24:33.777 INFO:teuthology.orchestra.run.trial120.stderr: count bytes template 2026-02-21T00:24:33.777 INFO:teuthology.orchestra.run.trial120.stderr: 1 8 MallocExtension::Register 2026-02-21T00:24:33.777 INFO:teuthology.orchestra.run.trial120.stderr:----------------------------------------------------- 2026-02-21T00:24:33.778 INFO:teuthology.orchestra.run.trial120.stderr: 2026-02-21T00:24:33.797 INFO:teuthology.orchestra.run.trial120.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-21T00:24:33.797 INFO:teuthology.orchestra.run.trial120.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-21T00:24:33.842 INFO:teuthology.orchestra.run.trial120.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-21T00:24:33.842 INFO:teuthology.orchestra.run.trial120.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-21T00:24:33.849 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:33,848 seastar - Reactor backend: linux-aio 2026-02-21T00:24:33.863 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:33,862 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:24:33.863 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:33,862 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:24:33.867 INFO:teuthology.orchestra.run.trial120.stderr:==38309==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:24:33.867 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:33,866 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-21T00:24:34.025 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x9fe8761 0x9fe908d 0xa0ce025 0xa0d870b 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:34.025 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x341e284 0x9fe4f0f 0x9fe51a1 0x9fe6ec7 0xa0c10e2 0xa0c185b 0xa0c1d46 0xa0c20d9 0xa0d4d60 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:34.025 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191ca6 /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x776e6e7 0x776e786 0x755bbfb 0x763dfc6 0x7bc7158 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:34.026 INFO:teuthology.orchestra.run.trial120.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffaa527ef7 0xffffffffaa1f7f16 0xffffffffaa1fb8d0 0xffffffffaa1fcc31 0xffffffffaa224893 0xffffffffaa224a55 0xffffffffaa1cb80e 0xffffffffaa1d3aeb 0xffffffffaa1d3f4b 0xffffffffa9e910d2 0xffffffffaab4d451 0xffffffffaac00ba2 2026-02-21T00:24:34.057 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x165bb 0xa9508e7 0xa0781c8 0xa078489 0xa0d7a03 0xa0d7b50 0xa0d7be3 0xa0d8a28 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:24:34.117 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0xa0a9a44 0xa0a9b55 0x7efed52 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:34.117 INFO:teuthology.orchestra.run.trial120.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffaa01f6b4 0xffffffffaa01fb20 0xffffffffa9e9525e 0xffffffffaa1e61a9 0xffffffffaa1c9715 0xffffffffaa1d3aeb 0xffffffffaa1d3f4b 0xffffffffa9e910d2 0xffffffffaab4d451 0xffffffffaac00ba2 2026-02-21T00:24:34.117 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0xa0306bc 0xa0c80c5 0xa0d236e 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:34.118 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16b377 /lib64/libasan.so.8+0xb0541 /lib64/libasan.so.8+0xb1437 0xba66a7 0x10d1d1e 0xf741d0 0x9fe874e 0x9fe908d 0xa0ce025 0xa0d870b 0x7bc71f4 0x7effe85 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:34.192 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1520f 0x9f30048 0x9f302cc 0xa0d49b5 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x77b6684 0x76972d7 0x787c2e0 0x76e6c34 0x76e6d4d 0xa3a2366 0xa3a2569 0xa3a2607 0xa3a268c 0xa3f5a8e 0xa3f5b47 0xa427489 0xa427830 0xa32dbb7 0xa32dc70 0xa32dcff 0xa3f55f6 0xa3f56b9 0xa3f57c6 0xa3f5981 0x763ea65 0x7697545 0x769777c 0x7538acd 0x726508c 0x781ce68 0x781cfbc 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:24:34.230 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc25 /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x157c7 0xcd2ea4 0xce4ede 0x9fdb1de 0xa0d6a0e 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:34.230 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65d9 /lib64/libasan.so.8+0x157c7 0x10d1d95 0xf741d0 0x9fe4c31 0x9fe4f2e 0x9fe51a1 0x9fe6ec7 0xa0c5b9a 0xa0c7540 0xa0c7b9a 0xa0c80e0 0xa0d236e 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:34.230 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16b377 /lib64/libasan.so.8+0xb0541 /lib64/libasan.so.8+0xb1437 0xba66a7 0x10d1d1e 0xf741d0 0x9fe874e 0x9fe908d 0xa0ce025 0xa0d870b 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:36.457 INFO:teuthology.orchestra.run.trial120.stderr:----------------------------------------------------- 2026-02-21T00:24:36.457 INFO:teuthology.orchestra.run.trial120.stderr:Suppressions used: 2026-02-21T00:24:36.457 INFO:teuthology.orchestra.run.trial120.stderr: count bytes template 2026-02-21T00:24:36.457 INFO:teuthology.orchestra.run.trial120.stderr: 1 8 MallocExtension::Register 2026-02-21T00:24:36.457 INFO:teuthology.orchestra.run.trial120.stderr:----------------------------------------------------- 2026-02-21T00:24:36.457 INFO:teuthology.orchestra.run.trial120.stderr: 2026-02-21T00:24:36.491 DEBUG:teuthology.orchestra.run.trial120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-21T00:24:36.517 DEBUG:teuthology.orchestra.run.trial120:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:24:36.671 INFO:teuthology.orchestra.run.trial120.stderr:INFO  osd - OSD::main: parsing early config 2026-02-21T00:24:36.731 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:36,730 seastar - Reactor backend: linux-aio 2026-02-21T00:24:36.746 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:36,745 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:24:36.746 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:36,745 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:24:36.748 INFO:teuthology.orchestra.run.trial120.stderr:==38377==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:24:36.807 INFO:teuthology.orchestra.run.trial120.stderr:DEBUG 2026-02-21 00:24:36,806 [shard 0:main] seastar - reactor::drain 2026-02-21T00:24:38.875 INFO:teuthology.orchestra.run.trial120.stderr:----------------------------------------------------- 2026-02-21T00:24:38.875 INFO:teuthology.orchestra.run.trial120.stderr:Suppressions used: 2026-02-21T00:24:38.875 INFO:teuthology.orchestra.run.trial120.stderr: count bytes template 2026-02-21T00:24:38.875 INFO:teuthology.orchestra.run.trial120.stderr: 1 8 MallocExtension::Register 2026-02-21T00:24:38.875 INFO:teuthology.orchestra.run.trial120.stderr:----------------------------------------------------- 2026-02-21T00:24:38.875 INFO:teuthology.orchestra.run.trial120.stderr: 2026-02-21T00:24:38.895 INFO:teuthology.orchestra.run.trial120.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-21T00:24:38.895 INFO:teuthology.orchestra.run.trial120.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-21T00:24:38.939 INFO:teuthology.orchestra.run.trial120.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-21T00:24:38.939 INFO:teuthology.orchestra.run.trial120.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-21T00:24:38.946 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:38,945 seastar - Reactor backend: linux-aio 2026-02-21T00:24:38.957 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:38,956 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:24:38.957 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:38,956 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:24:38.961 INFO:teuthology.orchestra.run.trial120.stderr:==38375==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:24:38.961 INFO:teuthology.orchestra.run.trial120.stderr:INFO 2026-02-21 00:24:38,960 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-21T00:24:39.117 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119155 /lib64/libasan.so.8+0xebbf /lib64/libasan.so.8+0xec776 0xcee0e1 0x7242b54 0x7eff20c 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:39.117 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc250 /lib64/libasan.so.8+0xc553 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe6ca27 0x7bc7117 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:39.118 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1006a8 /lib64/libasan.so.8+0xebac /lib64/libasan.so.8+0xec776 0xcee0e1 0x7bc72b5 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:39.149 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191c1b /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9ec99fb 0x9f2f1f6 0x9f2f30c 0x9f2f3e8 0x9f2f664 0x9f2f71d 0x9f2fc0b 0x9f3000f 0x9f302cc 0xa0d49b5 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:24:39.209 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0xcee2be 0x11680e4 0xf741c5 0x9fe4c31 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5a18 0x9fec227 0x9fec907 0xa0d213c 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:39.209 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16b357 /lib64/libasan.so.8+0xb0541 /lib64/libasan.so.8+0xb1437 0xba66a7 0x10d1d1e 0xf741d0 0xa0ce4e9 0xa0d8b34 0x7bc71f4 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:39.209 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1586f 0x10d1d95 0xf741d0 0x9fe4c31 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5a18 0x9fec227 0x9fec907 0xa0d213c 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:39.321 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1521e 0x7ebcf11 0xa0d8726 0x7bc71f4 0x7effe85 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:39.322 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xeb96 /lib64/libasan.so.8+0xef905 0x3422ab8 0x3422c92 0x3422f49 0x342303e 0xa0d4d7d 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:39.322 INFO:teuthology.orchestra.run.trial120.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15eaf 0xa0b1b34 0xa0b3149 0xa0b4464 0xa0b4884 0xa0d1aff 0xa0d67e7 0xa096d59 0xa095d59 0x7efff32 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:24:41.525 INFO:teuthology.orchestra.run.trial120.stderr:----------------------------------------------------- 2026-02-21T00:24:41.525 INFO:teuthology.orchestra.run.trial120.stderr:Suppressions used: 2026-02-21T00:24:41.526 INFO:teuthology.orchestra.run.trial120.stderr: count bytes template 2026-02-21T00:24:41.526 INFO:teuthology.orchestra.run.trial120.stderr: 1 8 MallocExtension::Register 2026-02-21T00:24:41.526 INFO:teuthology.orchestra.run.trial120.stderr:----------------------------------------------------- 2026-02-21T00:24:41.526 INFO:teuthology.orchestra.run.trial120.stderr: 2026-02-21T00:24:41.559 DEBUG:teuthology.orchestra.run.trial120:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-21T00:24:41.584 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-21T00:24:41.585 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:24:41.585 DEBUG:teuthology.orchestra.run.trial120:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-21T00:24:41.649 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:24:41.649 DEBUG:teuthology.orchestra.run.trial120:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-21T00:24:41.713 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:24:41.713 DEBUG:teuthology.orchestra.run.trial120:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-21T00:24:41.777 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:24:41.777 DEBUG:teuthology.orchestra.run.trial120:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-21T00:24:41.841 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:24:41.842 DEBUG:teuthology.orchestra.run.trial120:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-21T00:24:41.897 INFO:tasks.ceph:Adding keys to all mons... 2026-02-21T00:24:41.898 DEBUG:teuthology.orchestra.run.trial120:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-21T00:24:41.960 INFO:teuthology.orchestra.run.trial120.stdout:[mgr.x] 2026-02-21T00:24:41.960 INFO:teuthology.orchestra.run.trial120.stdout: key = AQA3+5hpckrCLxAAfRzelnabOltmm6VmLsbenA== 2026-02-21T00:24:41.960 INFO:teuthology.orchestra.run.trial120.stdout:[osd.0] 2026-02-21T00:24:41.960 INFO:teuthology.orchestra.run.trial120.stdout: key = AQA8+5hp8v9eNBAAeF/dJC8ZgBFGFa4y70WbCg== 2026-02-21T00:24:41.961 INFO:teuthology.orchestra.run.trial120.stdout:[osd.1] 2026-02-21T00:24:41.961 INFO:teuthology.orchestra.run.trial120.stdout: key = AQBB+5hp75KDNxAAc7aTVu8ZqM3yBZul/AsPsQ== 2026-02-21T00:24:41.961 INFO:teuthology.orchestra.run.trial120.stdout:[osd.2] 2026-02-21T00:24:41.961 INFO:teuthology.orchestra.run.trial120.stdout: key = AQBH+5hpmy1rARAAUnebRLRzsKpl+8D3b7vR+g== 2026-02-21T00:24:41.961 INFO:teuthology.orchestra.run.trial120.stdout:[client.0] 2026-02-21T00:24:41.961 INFO:teuthology.orchestra.run.trial120.stdout: key = AQA3+5hpu29jMhAAadg7EtPp7Fj232spRtNDjA== 2026-02-21T00:24:41.962 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-21T00:24:42.049 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T00:24:42.098 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T00:24:42.146 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T00:24:42.194 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-21T00:24:42.244 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-21T00:24:42.245 DEBUG:teuthology.orchestra.run.trial120:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-21T00:24:42.270 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-21T00:24:42.356 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:42.355+0000 7f982f287c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:42.356 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:42.355+0000 7f982f287c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:42.364 DEBUG:teuthology.orchestra.run.trial120:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-21T00:24:42.389 DEBUG:teuthology.orchestra.run.trial120:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:24:42.445 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-21T00:24:42.446 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-21T00:24:42.446 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-21T00:24:42.487 INFO:tasks.ceph.mon.a:Started 2026-02-21T00:24:42.488 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-21T00:24:42.488 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-21T00:24:42.488 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-21T00:24:42.491 INFO:tasks.ceph.mgr.x:Started 2026-02-21T00:24:42.491 DEBUG:tasks.ceph:set 0 configs 2026-02-21T00:24:42.491 DEBUG:teuthology.orchestra.run.trial120:> sudo ceph --cluster ceph config dump 2026-02-21T00:24:42.551 INFO:tasks.ceph.mon.a.trial120.stderr:2026-02-21T00:24:42.549+0000 7f5606038c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:42.551 INFO:tasks.ceph.mon.a.trial120.stderr:2026-02-21T00:24:42.549+0000 7f5606038c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:42.551 INFO:tasks.ceph.mon.a.trial120.stderr:2026-02-21T00:24:42.550+0000 7f5606038c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:42.555 INFO:tasks.ceph.mgr.x.trial120.stderr:2026-02-21T00:24:42.553+0000 7f68a102ffc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:42.567 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:42.563+0000 7fc6b815b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:42.568 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:42.567+0000 7fc6b815b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:42.599 INFO:teuthology.orchestra.run.trial120.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-21T00:24:42.607 INFO:tasks.ceph:Setting crush tunables to default 2026-02-21T00:24:42.608 DEBUG:teuthology.orchestra.run.trial120:> sudo ceph --cluster ceph osd crush tunables default 2026-02-21T00:24:42.681 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:42.676+0000 7f16dbacb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:42.682 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:42.680+0000 7f16dbacb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:42.714 INFO:teuthology.orchestra.run.trial120.stderr:adjusted tunables profile to default 2026-02-21T00:24:42.722 INFO:tasks.ceph:check_enable_crimson: True 2026-02-21T00:24:42.723 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-21T00:24:42.723 DEBUG:teuthology.orchestra.run.trial120:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-21T00:24:42.758 INFO:tasks.ceph.mgr.x.trial120.stderr:2026-02-21T00:24:42.757+0000 7f68a102ffc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:42.759 INFO:tasks.ceph.mgr.x.trial120.stderr:2026-02-21T00:24:42.757+0000 7f68a102ffc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:42.795 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:42.791+0000 7eff27db8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:42.796 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:42.795+0000 7eff27db8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:42.834 INFO:tasks.ceph.mon.a.trial120.stderr:2026-02-21T00:24:42.833+0000 7f55fd3f1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-21T00:24:42.834 INFO:tasks.ceph.mon.a.trial120.stderr:Please be aware that this feature is experimental, untested, 2026-02-21T00:24:42.835 INFO:tasks.ceph.mon.a.trial120.stderr:unsupported, and may result in data corruption, data loss, 2026-02-21T00:24:42.835 INFO:tasks.ceph.mon.a.trial120.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-21T00:24:42.835 INFO:tasks.ceph.mon.a.trial120.stderr:feature with important data. 2026-02-21T00:24:42.835 INFO:tasks.ceph.mon.a.trial120.stderr: 2026-02-21T00:24:43.589 INFO:tasks.ceph.mgr.x.trial120.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-21T00:24:43.589 INFO:tasks.ceph.mgr.x.trial120.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-21T00:24:43.589 INFO:tasks.ceph.mgr.x.trial120.stderr: from numpy import show_config as show_numpy_config 2026-02-21T00:24:43.725 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-21T00:24:43.726 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:24:43.726 DEBUG:teuthology.orchestra.run.trial120:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-21T00:24:43.790 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:24:43.790 DEBUG:teuthology.orchestra.run.trial120:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-21T00:24:43.853 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:24:43.853 DEBUG:teuthology.orchestra.run.trial120:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-21T00:24:43.918 DEBUG:teuthology.orchestra.run.trial120:> sudo ceph --cluster ceph osd new 4e03514f-0283-4759-8a61-904e773594bd 0 2026-02-21T00:24:44.030 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:44.025+0000 7f720f65a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:44.031 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:44.030+0000 7f720f65a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:44.067 INFO:teuthology.orchestra.run.trial120.stdout:0 2026-02-21T00:24:44.076 DEBUG:teuthology.orchestra.run.trial120:> sudo ceph --cluster ceph osd new 4a236bea-4d64-463a-a0f0-88e0282838c2 1 2026-02-21T00:24:44.150 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:44.146+0000 7fef50a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:44.152 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:44.150+0000 7fef50a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:44.187 INFO:teuthology.orchestra.run.trial120.stdout:1 2026-02-21T00:24:44.196 DEBUG:teuthology.orchestra.run.trial120:> sudo ceph --cluster ceph osd new c02c6949-4b7a-46a5-9c6a-9f3c8ddd4af0 2 2026-02-21T00:24:44.271 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:44.267+0000 7f5be5fdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:44.272 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:44.271+0000 7f5be5fdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:44.308 INFO:teuthology.orchestra.run.trial120.stdout:2 2026-02-21T00:24:44.315 DEBUG:teuthology.orchestra.run.trial120:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-21T00:24:44.342 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-21T00:24:44.343 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-21T00:24:44.384 INFO:tasks.ceph.osd.0:Started 2026-02-21T00:24:44.385 DEBUG:teuthology.orchestra.run.trial120:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-21T00:24:44.414 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-21T00:24:44.414 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-21T00:24:44.456 INFO:tasks.ceph.osd.1:Started 2026-02-21T00:24:44.457 DEBUG:teuthology.orchestra.run.trial120:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-21T00:24:44.484 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-21T00:24:44.485 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-21T00:24:44.526 INFO:tasks.ceph.osd.2:Started 2026-02-21T00:24:44.527 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T00:24:44.604 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:44.601+0000 7ff177a11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:44.605 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:44.604+0000 7ff177a11640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:44.637 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:24:44.638 INFO:teuthology.orchestra.run.trial120.stdout:{"epoch":6,"fsid":"1a3b4641-234e-4a24-ae80-55b3d108e591","created":"2026-02-21T00:24:42.560614+0000","modified":"2026-02-21T00:24:44.307258+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T00:24:44.307258+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"4e03514f-0283-4759-8a61-904e773594bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4a236bea-4d64-463a-a0f0-88e0282838c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c02c6949-4b7a-46a5-9c6a-9f3c8ddd4af0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:24:44.647 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-21T00:24:44.648 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-21T00:24:44.950 DEBUG:teuthology.orchestra.run.trial120:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T00:24:45.018 INFO:teuthology.misc.health.trial120.stderr:2026-02-21T00:24:45.014+0000 7fc99841a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:45.019 INFO:teuthology.misc.health.trial120.stderr:2026-02-21T00:24:45.017+0000 7fc99841a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:45.051 INFO:teuthology.misc.health.trial120.stdout: 2026-02-21T00:24:45.051 INFO:teuthology.misc.health.trial120.stdout:{"epoch":6,"fsid":"1a3b4641-234e-4a24-ae80-55b3d108e591","created":"2026-02-21T00:24:42.560614+0000","modified":"2026-02-21T00:24:44.307258+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T00:24:44.307258+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"4e03514f-0283-4759-8a61-904e773594bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4a236bea-4d64-463a-a0f0-88e0282838c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c02c6949-4b7a-46a5-9c6a-9f3c8ddd4af0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:24:45.058 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-21T00:24:45.823 INFO:tasks.ceph.mgr.x.trial120.stderr:2026-02-21T00:24:45.822+0000 7f67bdc93640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-21T00:24:51.360 DEBUG:teuthology.orchestra.run.trial120:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T00:24:51.430 INFO:teuthology.misc.health.trial120.stderr:2026-02-21T00:24:51.425+0000 7fce4990b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:51.431 INFO:teuthology.misc.health.trial120.stderr:2026-02-21T00:24:51.429+0000 7fce4990b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:51.540 INFO:teuthology.misc.health.trial120.stdout: 2026-02-21T00:24:51.540 INFO:teuthology.misc.health.trial120.stdout:{"epoch":7,"fsid":"1a3b4641-234e-4a24-ae80-55b3d108e591","created":"2026-02-21T00:24:42.560614+0000","modified":"2026-02-21T00:24:50.555818+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T00:24:44.307258+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"4e03514f-0283-4759-8a61-904e773594bd","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"4a236bea-4d64-463a-a0f0-88e0282838c2","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c02c6949-4b7a-46a5-9c6a-9f3c8ddd4af0","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:24:51.548 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-21T00:24:57.850 DEBUG:teuthology.orchestra.run.trial120:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T00:24:57.920 INFO:teuthology.misc.health.trial120.stderr:2026-02-21T00:24:57.915+0000 7fc01556f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:57.921 INFO:teuthology.misc.health.trial120.stderr:2026-02-21T00:24:57.919+0000 7fc01556f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:58.030 INFO:teuthology.misc.health.trial120.stdout: 2026-02-21T00:24:58.031 INFO:teuthology.misc.health.trial120.stdout:{"epoch":13,"fsid":"1a3b4641-234e-4a24-ae80-55b3d108e591","created":"2026-02-21T00:24:42.560614+0000","modified":"2026-02-21T00:24:57.570262+0000","last_up_change":"2026-02-21T00:24:53.558752+0000","last_in_change":"2026-02-21T00:24:44.307258+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T00:24:54.832670+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4e03514f-0283-4759-8a61-904e773594bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6804","nonce":343424161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6805","nonce":343424161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6812","nonce":343424161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6811","nonce":343424161}]},"public_addr":"10.20.193.120:6804/343424161","cluster_addr":"10.20.193.120:6805/343424161","heartbeat_back_addr":"10.20.193.120:6812/343424161","heartbeat_front_addr":"10.20.193.120:6811/343424161","state":["exists","up"]},{"osd":1,"uuid":"4a236bea-4d64-463a-a0f0-88e0282838c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6806","nonce":424520277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6807","nonce":424520277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6814","nonce":424520277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6813","nonce":424520277}]},"public_addr":"10.20.193.120:6806/424520277","cluster_addr":"10.20.193.120:6807/424520277","heartbeat_back_addr":"10.20.193.120:6814/424520277","heartbeat_front_addr":"10.20.193.120:6813/424520277","state":["exists","up"]},{"osd":2,"uuid":"c02c6949-4b7a-46a5-9c6a-9f3c8ddd4af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6803","nonce":310946312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6802","nonce":310946312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6809","nonce":310946312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6810","nonce":310946312}]},"public_addr":"10.20.193.120:6803/310946312","cluster_addr":"10.20.193.120:6802/310946312","heartbeat_back_addr":"10.20.193.120:6809/310946312","heartbeat_front_addr":"10.20.193.120:6810/310946312","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:24:58.039 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-21T00:24:58.039 INFO:tasks.ceph:Creating RBD pool 2026-02-21T00:24:58.039 DEBUG:teuthology.orchestra.run.trial120:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-21T00:24:58.113 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:58.107+0000 7fed9965a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:58.114 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:58.111+0000 7fed9965a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:58.575 INFO:teuthology.orchestra.run.trial120.stderr:pool 'rbd' created 2026-02-21T00:24:58.588 DEBUG:teuthology.orchestra.run.trial120:> rbd --cluster ceph pool init rbd 2026-02-21T00:24:58.629 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:58.627+0000 7f3770081240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:58.630 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:58.627+0000 7f3770081240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:24:58.630 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:24:58.628+0000 7f3770081240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:01.614 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-21T00:25:01.615 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-21T00:25:01.615 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-21T00:25:01.730 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:01.725+0000 7f1605e8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:01.731 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:01.729+0000 7f1605e8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:01.841 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:25:01.850 INFO:teuthology.orchestra.run.trial120.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-21T00:25:01.851 INFO:tasks.ceph_manager:config epoch is 1 2026-02-21T00:25:01.851 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-21T00:25:01.851 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-21T00:25:01.852 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-21T00:25:01.926 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:01.922+0000 7f2171d6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:01.928 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:01.926+0000 7f2171d6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.069 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:25:02.077 INFO:teuthology.orchestra.run.trial120.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6800","nonce":3930374016},{"type":"v1","addr":"10.20.193.120:6801","nonce":3930374016}]},"active_addr":"10.20.193.120:6801/3930374016","active_change":"2026-02-21T00:24:44.817458+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.120:0","nonce":2002624790}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.120:0","nonce":1120650501}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.120:0","nonce":3847642576}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.120:0","nonce":673722498}]}]} 2026-02-21T00:25:02.082 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-21T00:25:02.082 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-21T00:25:02.083 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T00:25:02.158 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.154+0000 7f5af805a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.159 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.157+0000 7f5af805a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.268 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:25:02.268 INFO:teuthology.orchestra.run.trial120.stdout:{"epoch":17,"fsid":"1a3b4641-234e-4a24-ae80-55b3d108e591","created":"2026-02-21T00:24:42.560614+0000","modified":"2026-02-21T00:25:01.585826+0000","last_up_change":"2026-02-21T00:24:53.558752+0000","last_in_change":"2026-02-21T00:24:44.307258+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T00:24:54.832670+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-21T00:24:58.225296+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4e03514f-0283-4759-8a61-904e773594bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6804","nonce":343424161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6805","nonce":343424161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6812","nonce":343424161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6811","nonce":343424161}]},"public_addr":"10.20.193.120:6804/343424161","cluster_addr":"10.20.193.120:6805/343424161","heartbeat_back_addr":"10.20.193.120:6812/343424161","heartbeat_front_addr":"10.20.193.120:6811/343424161","state":["exists","up"]},{"osd":1,"uuid":"4a236bea-4d64-463a-a0f0-88e0282838c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6806","nonce":424520277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6807","nonce":424520277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6814","nonce":424520277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6813","nonce":424520277}]},"public_addr":"10.20.193.120:6806/424520277","cluster_addr":"10.20.193.120:6807/424520277","heartbeat_back_addr":"10.20.193.120:6814/424520277","heartbeat_front_addr":"10.20.193.120:6813/424520277","state":["exists","up"]},{"osd":2,"uuid":"c02c6949-4b7a-46a5-9c6a-9f3c8ddd4af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6803","nonce":310946312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6802","nonce":310946312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6809","nonce":310946312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6810","nonce":310946312}]},"public_addr":"10.20.193.120:6803/310946312","cluster_addr":"10.20.193.120:6802/310946312","heartbeat_back_addr":"10.20.193.120:6809/310946312","heartbeat_front_addr":"10.20.193.120:6810/310946312","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:25:02.279 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-21T00:25:02.279 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T00:25:02.355 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.351+0000 7fcca5e1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.356 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.354+0000 7fcca5e1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.465 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:25:02.465 INFO:teuthology.orchestra.run.trial120.stdout:{"epoch":17,"fsid":"1a3b4641-234e-4a24-ae80-55b3d108e591","created":"2026-02-21T00:24:42.560614+0000","modified":"2026-02-21T00:25:01.585826+0000","last_up_change":"2026-02-21T00:24:53.558752+0000","last_in_change":"2026-02-21T00:24:44.307258+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T00:24:54.832670+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-21T00:24:58.225296+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"4e03514f-0283-4759-8a61-904e773594bd","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6804","nonce":343424161}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6805","nonce":343424161}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6812","nonce":343424161}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6811","nonce":343424161}]},"public_addr":"10.20.193.120:6804/343424161","cluster_addr":"10.20.193.120:6805/343424161","heartbeat_back_addr":"10.20.193.120:6812/343424161","heartbeat_front_addr":"10.20.193.120:6811/343424161","state":["exists","up"]},{"osd":1,"uuid":"4a236bea-4d64-463a-a0f0-88e0282838c2","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6806","nonce":424520277}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6807","nonce":424520277}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6814","nonce":424520277}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6813","nonce":424520277}]},"public_addr":"10.20.193.120:6806/424520277","cluster_addr":"10.20.193.120:6807/424520277","heartbeat_back_addr":"10.20.193.120:6814/424520277","heartbeat_front_addr":"10.20.193.120:6813/424520277","state":["exists","up"]},{"osd":2,"uuid":"c02c6949-4b7a-46a5-9c6a-9f3c8ddd4af0","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6803","nonce":310946312}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6802","nonce":310946312}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6809","nonce":310946312}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.120:6810","nonce":310946312}]},"public_addr":"10.20.193.120:6803/310946312","cluster_addr":"10.20.193.120:6802/310946312","heartbeat_back_addr":"10.20.193.120:6809/310946312","heartbeat_front_addr":"10.20.193.120:6810/310946312","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:25:02.475 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-21T00:25:02.475 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-21T00:25:02.475 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-21T00:25:02.556 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.551+0000 7f933a70b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.556 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.551+0000 7f73c8a3d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.557 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.551+0000 7fa7f0ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.557 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.555+0000 7f933a70b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.558 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.556+0000 7f73c8a3d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.558 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.556+0000 7fa7f0ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.572 INFO:teuthology.orchestra.run.trial120.stdout:42949672968 2026-02-21T00:25:02.572 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T00:25:02.573 INFO:teuthology.orchestra.run.trial120.stdout:38654705673 2026-02-21T00:25:02.574 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T00:25:02.574 INFO:teuthology.orchestra.run.trial120.stdout:42949672968 2026-02-21T00:25:02.575 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T00:25:02.694 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.689+0000 7f1f2db74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.695 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.689+0000 7fa912ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.695 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.689+0000 7fe463e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.695 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.693+0000 7f1f2db74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.696 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.693+0000 7fa912ccb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.696 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.694+0000 7fe463e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.808 INFO:teuthology.orchestra.run.trial120.stdout:38654705673 2026-02-21T00:25:02.812 INFO:teuthology.orchestra.run.trial120.stdout:42949672968 2026-02-21T00:25:02.812 INFO:teuthology.orchestra.run.trial120.stdout:42949672968 2026-02-21T00:25:02.819 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705673 got 38654705673 for osd.2 2026-02-21T00:25:02.820 DEBUG:teuthology.parallel:result is None 2026-02-21T00:25:02.823 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672968 got 42949672968 for osd.0 2026-02-21T00:25:02.823 DEBUG:teuthology.parallel:result is None 2026-02-21T00:25:02.824 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672968 got 42949672968 for osd.1 2026-02-21T00:25:02.824 DEBUG:teuthology.parallel:result is None 2026-02-21T00:25:02.824 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-21T00:25:02.824 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T00:25:02.940 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.936+0000 7f687c7dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:02.941 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:02.939+0000 7f687c7dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:03.048 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:25:03.048 INFO:teuthology.orchestra.run.trial120.stderr:dumped all 2026-02-21T00:25:03.057 INFO:teuthology.orchestra.run.trial120.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-21T00:25:02.821140+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62917004,"kb_used_data":62917004,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94369348,"statfs":{"total":161061224448,"available":96634212352,"internally_reserved":0,"allocated":64427012096,"data_stored":64427012096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.255589"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.598265+0000","last_change":"2026-02-21T00:25:01.603859+0000","last_active":"2026-02-21T00:25:01.598265+0000","last_peered":"2026-02-21T00:25:01.598265+0000","last_clean":"2026-02-21T00:25:01.598265+0000","last_became_active":"2026-02-21T00:24:59.617299+0000","last_became_peered":"2026-02-21T00:24:59.617299+0000","last_unstale":"2026-02-21T00:25:01.598265+0000","last_undegraded":"2026-02-21T00:25:01.598265+0000","last_fullsized":"2026-02-21T00:25:01.598265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_clean_scrub_stamp":"2026-02-21T00:24:58.573618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.599787+0000","last_change":"2026-02-21T00:25:01.602903+0000","last_active":"2026-02-21T00:25:01.599787+0000","last_peered":"2026-02-21T00:25:01.599787+0000","last_clean":"2026-02-21T00:25:01.599787+0000","last_became_active":"2026-02-21T00:24:59.616430+0000","last_became_peered":"2026-02-21T00:24:59.616430+0000","last_unstale":"2026-02-21T00:25:01.599787+0000","last_undegraded":"2026-02-21T00:25:01.599787+0000","last_fullsized":"2026-02-21T00:25:01.599787+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_clean_scrub_stamp":"2026-02-21T00:24:58.573618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.599758+0000","last_change":"2026-02-21T00:25:01.604324+0000","last_active":"2026-02-21T00:25:01.599758+0000","last_peered":"2026-02-21T00:25:01.599758+0000","last_clean":"2026-02-21T00:25:01.599758+0000","last_became_active":"2026-02-21T00:24:59.623385+0000","last_became_peered":"2026-02-21T00:24:59.623385+0000","last_unstale":"2026-02-21T00:25:01.599758+0000","last_undegraded":"2026-02-21T00:25:01.599758+0000","last_fullsized":"2026-02-21T00:25:01.599758+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_clean_scrub_stamp":"2026-02-21T00:24:58.573618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.599890+0000","last_change":"2026-02-21T00:25:01.602776+0000","last_active":"2026-02-21T00:25:01.599890+0000","last_peered":"2026-02-21T00:25:01.599890+0000","last_clean":"2026-02-21T00:25:01.599890+0000","last_became_active":"2026-02-21T00:24:59.620858+0000","last_became_peered":"2026-02-21T00:24:59.620858+0000","last_unstale":"2026-02-21T00:25:01.599890+0000","last_undegraded":"2026-02-21T00:25:01.599890+0000","last_fullsized":"2026-02-21T00:25:01.599890+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_clean_scrub_stamp":"2026-02-21T00:24:58.573618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.598860+0000","last_change":"2026-02-21T00:25:01.598860+0000","last_active":"2026-02-21T00:25:01.598860+0000","last_peered":"2026-02-21T00:25:01.598860+0000","last_clean":"2026-02-21T00:25:01.598860+0000","last_became_active":"2026-02-21T00:24:59.617763+0000","last_became_peered":"2026-02-21T00:24:59.617763+0000","last_unstale":"2026-02-21T00:25:01.598860+0000","last_undegraded":"2026-02-21T00:25:01.598860+0000","last_fullsized":"2026-02-21T00:25:01.598860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_clean_scrub_stamp":"2026-02-21T00:24:58.573618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.596706+0000","last_change":"2026-02-21T00:25:01.599109+0000","last_active":"2026-02-21T00:25:01.596706+0000","last_peered":"2026-02-21T00:25:01.596706+0000","last_clean":"2026-02-21T00:25:01.596706+0000","last_became_active":"2026-02-21T00:24:59.624317+0000","last_became_peered":"2026-02-21T00:24:59.624317+0000","last_unstale":"2026-02-21T00:25:01.596706+0000","last_undegraded":"2026-02-21T00:25:01.596706+0000","last_fullsized":"2026-02-21T00:25:01.596706+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_clean_scrub_stamp":"2026-02-21T00:24:58.573618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.594782+0000","last_change":"2026-02-21T00:25:01.597665+0000","last_active":"2026-02-21T00:25:01.594782+0000","last_peered":"2026-02-21T00:25:01.594782+0000","last_clean":"2026-02-21T00:25:01.594782+0000","last_became_active":"2026-02-21T00:24:59.620776+0000","last_became_peered":"2026-02-21T00:24:59.620776+0000","last_unstale":"2026-02-21T00:25:01.594782+0000","last_undegraded":"2026-02-21T00:25:01.594782+0000","last_fullsized":"2026-02-21T00:25:01.594782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_clean_scrub_stamp":"2026-02-21T00:24:58.573618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.599850+0000","last_change":"2026-02-21T00:25:01.604404+0000","last_active":"2026-02-21T00:25:01.599850+0000","last_peered":"2026-02-21T00:25:01.599850+0000","last_clean":"2026-02-21T00:25:01.599850+0000","last_became_active":"2026-02-21T00:24:59.616424+0000","last_became_peered":"2026-02-21T00:24:59.616424+0000","last_unstale":"2026-02-21T00:25:01.599850+0000","last_undegraded":"2026-02-21T00:25:01.599850+0000","last_fullsized":"2026-02-21T00:25:01.599850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_clean_scrub_stamp":"2026-02-21T00:24:58.573618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.597722+0000","last_change":"2026-02-21T00:24:56.600892+0000","last_active":"2026-02-21T00:25:01.597722+0000","last_peered":"2026-02-21T00:25:01.597722+0000","last_clean":"2026-02-21T00:25:01.597722+0000","last_became_active":"2026-02-21T00:24:56.593624+0000","last_became_peered":"2026-02-21T00:24:56.593624+0000","last_unstale":"2026-02-21T00:25:01.597722+0000","last_undegraded":"2026-02-21T00:25:01.597722+0000","last_fullsized":"2026-02-21T00:25:01.597722+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:55.563045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:55.563045+0000","last_clean_scrub_stamp":"2026-02-21T00:24:55.563045+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972620,"kb_used_data":20972620,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456164,"statfs":{"total":53687074816,"available":32211111936,"internally_reserved":0,"allocated":21475962880,"data_stored":21475962880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972572,"kb_used_data":20972572,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456212,"statfs":{"total":53687074816,"available":32211161088,"internally_reserved":0,"allocated":21475913728,"data_stored":21475913728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971812,"kb_used_data":20971812,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456972,"statfs":{"total":53687074816,"available":32211939328,"internally_reserved":0,"allocated":21475135488,"data_stored":21475135488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T00:25:03.058 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T00:25:03.132 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:03.128+0000 7f3128011640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:03.133 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:03.131+0000 7f3128011640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:03.241 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:25:03.242 INFO:teuthology.orchestra.run.trial120.stderr:dumped all 2026-02-21T00:25:03.250 INFO:teuthology.orchestra.run.trial120.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-21T00:25:02.821140+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62917004,"kb_used_data":62917004,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94369348,"statfs":{"total":161061224448,"available":96634212352,"internally_reserved":0,"allocated":64427012096,"data_stored":64427012096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.255589"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.598265+0000","last_change":"2026-02-21T00:25:01.603859+0000","last_active":"2026-02-21T00:25:01.598265+0000","last_peered":"2026-02-21T00:25:01.598265+0000","last_clean":"2026-02-21T00:25:01.598265+0000","last_became_active":"2026-02-21T00:24:59.617299+0000","last_became_peered":"2026-02-21T00:24:59.617299+0000","last_unstale":"2026-02-21T00:25:01.598265+0000","last_undegraded":"2026-02-21T00:25:01.598265+0000","last_fullsized":"2026-02-21T00:25:01.598265+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_clean_scrub_stamp":"2026-02-21T00:24:58.573618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.599787+0000","last_change":"2026-02-21T00:25:01.602903+0000","last_active":"2026-02-21T00:25:01.599787+0000","last_peered":"2026-02-21T00:25:01.599787+0000","last_clean":"2026-02-21T00:25:01.599787+0000","last_became_active":"2026-02-21T00:24:59.616430+0000","last_became_peered":"2026-02-21T00:24:59.616430+0000","last_unstale":"2026-02-21T00:25:01.599787+0000","last_undegraded":"2026-02-21T00:25:01.599787+0000","last_fullsized":"2026-02-21T00:25:01.599787+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_clean_scrub_stamp":"2026-02-21T00:24:58.573618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.599758+0000","last_change":"2026-02-21T00:25:01.604324+0000","last_active":"2026-02-21T00:25:01.599758+0000","last_peered":"2026-02-21T00:25:01.599758+0000","last_clean":"2026-02-21T00:25:01.599758+0000","last_became_active":"2026-02-21T00:24:59.623385+0000","last_became_peered":"2026-02-21T00:24:59.623385+0000","last_unstale":"2026-02-21T00:25:01.599758+0000","last_undegraded":"2026-02-21T00:25:01.599758+0000","last_fullsized":"2026-02-21T00:25:01.599758+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_clean_scrub_stamp":"2026-02-21T00:24:58.573618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.599890+0000","last_change":"2026-02-21T00:25:01.602776+0000","last_active":"2026-02-21T00:25:01.599890+0000","last_peered":"2026-02-21T00:25:01.599890+0000","last_clean":"2026-02-21T00:25:01.599890+0000","last_became_active":"2026-02-21T00:24:59.620858+0000","last_became_peered":"2026-02-21T00:24:59.620858+0000","last_unstale":"2026-02-21T00:25:01.599890+0000","last_undegraded":"2026-02-21T00:25:01.599890+0000","last_fullsized":"2026-02-21T00:25:01.599890+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_clean_scrub_stamp":"2026-02-21T00:24:58.573618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.598860+0000","last_change":"2026-02-21T00:25:01.598860+0000","last_active":"2026-02-21T00:25:01.598860+0000","last_peered":"2026-02-21T00:25:01.598860+0000","last_clean":"2026-02-21T00:25:01.598860+0000","last_became_active":"2026-02-21T00:24:59.617763+0000","last_became_peered":"2026-02-21T00:24:59.617763+0000","last_unstale":"2026-02-21T00:25:01.598860+0000","last_undegraded":"2026-02-21T00:25:01.598860+0000","last_fullsized":"2026-02-21T00:25:01.598860+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_clean_scrub_stamp":"2026-02-21T00:24:58.573618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.596706+0000","last_change":"2026-02-21T00:25:01.599109+0000","last_active":"2026-02-21T00:25:01.596706+0000","last_peered":"2026-02-21T00:25:01.596706+0000","last_clean":"2026-02-21T00:25:01.596706+0000","last_became_active":"2026-02-21T00:24:59.624317+0000","last_became_peered":"2026-02-21T00:24:59.624317+0000","last_unstale":"2026-02-21T00:25:01.596706+0000","last_undegraded":"2026-02-21T00:25:01.596706+0000","last_fullsized":"2026-02-21T00:25:01.596706+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_clean_scrub_stamp":"2026-02-21T00:24:58.573618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.594782+0000","last_change":"2026-02-21T00:25:01.597665+0000","last_active":"2026-02-21T00:25:01.594782+0000","last_peered":"2026-02-21T00:25:01.594782+0000","last_clean":"2026-02-21T00:25:01.594782+0000","last_became_active":"2026-02-21T00:24:59.620776+0000","last_became_peered":"2026-02-21T00:24:59.620776+0000","last_unstale":"2026-02-21T00:25:01.594782+0000","last_undegraded":"2026-02-21T00:25:01.594782+0000","last_fullsized":"2026-02-21T00:25:01.594782+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_clean_scrub_stamp":"2026-02-21T00:24:58.573618+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.599850+0000","last_change":"2026-02-21T00:25:01.604404+0000","last_active":"2026-02-21T00:25:01.599850+0000","last_peered":"2026-02-21T00:25:01.599850+0000","last_clean":"2026-02-21T00:25:01.599850+0000","last_became_active":"2026-02-21T00:24:59.616424+0000","last_became_peered":"2026-02-21T00:24:59.616424+0000","last_unstale":"2026-02-21T00:25:01.599850+0000","last_undegraded":"2026-02-21T00:25:01.599850+0000","last_fullsized":"2026-02-21T00:25:01.599850+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:58.573618+0000","last_clean_scrub_stamp":"2026-02-21T00:24:58.573618+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:25:01.597722+0000","last_change":"2026-02-21T00:24:56.600892+0000","last_active":"2026-02-21T00:25:01.597722+0000","last_peered":"2026-02-21T00:25:01.597722+0000","last_clean":"2026-02-21T00:25:01.597722+0000","last_became_active":"2026-02-21T00:24:56.593624+0000","last_became_peered":"2026-02-21T00:24:56.593624+0000","last_unstale":"2026-02-21T00:25:01.597722+0000","last_undegraded":"2026-02-21T00:25:01.597722+0000","last_fullsized":"2026-02-21T00:25:01.597722+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:24:55.563045+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:24:55.563045+0000","last_clean_scrub_stamp":"2026-02-21T00:24:55.563045+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972620,"kb_used_data":20972620,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456164,"statfs":{"total":53687074816,"available":32211111936,"internally_reserved":0,"allocated":21475962880,"data_stored":21475962880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972572,"kb_used_data":20972572,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456212,"statfs":{"total":53687074816,"available":32211161088,"internally_reserved":0,"allocated":21475913728,"data_stored":21475913728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971812,"kb_used_data":20971812,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456972,"statfs":{"total":53687074816,"available":32211939328,"internally_reserved":0,"allocated":21475135488,"data_stored":21475135488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T00:25:03.251 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-21T00:25:03.251 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-21T00:25:03.252 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-21T00:25:03.252 DEBUG:teuthology.orchestra.run.trial120:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-21T00:25:03.325 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:03.322+0000 7f7e01e1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:03.326 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:25:03.324+0000 7f7e01e1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:25:03.462 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:25:03.462 INFO:teuthology.orchestra.run.trial120.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-21T00:25:03.471 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-21T00:25:03.471 INFO:teuthology.run_tasks:Running task ssh_keys... 2026-02-21T00:25:03.556 DEBUG:teuthology.orchestra.run.trial120:> echo 'StrictHostKeyChecking no 2026-02-21T00:25:03.556 DEBUG:teuthology.orchestra.run.trial120:> ' > /home/ubuntu/.ssh/config && echo 'UserKnownHostsFile ' /dev/null >> /home/ubuntu/.ssh/config && chmod 600 /home/ubuntu/.ssh/config 2026-02-21T00:25:03.572 INFO:teuthology.task.ssh_keys:generated public key AAAAB3NzaC1yc2EAAAADAQABAAABAQCR8WRW541B38y8zEa8IP2OGJJltjN0I0h89WP1NxsbeSEDEIFhp0PLr7LtOG1vhgPuWeGSd8Jf71L1cf+s3sHfwT0FfN/72cJ9+I3qEu+oJHYnUaM41RGQVmFYjM0mQIux+x4JRCPeESiZR4dzuKqu8wvCqF8vxsIvm80CxWO4BtTS2bi3bVjuzej4WOQxezR6CLR5K5WXwfoRBNbvb4avQdJGQbzXKVphemBDtFoyF3XoUHSlP16QLGLQgFPlplpxzsQe+FOiPbvhbMOTJM/AZ6a7i9GNrniFzFdJj3Jrs7Ke1x6Oo8hW8dY/YR3RNZZfT80oG+Jxf5CPub1if4kh 2026-02-21T00:25:03.572 INFO:teuthology.task.ssh_keys:pushing keys to trial120.front.sepia.ceph.com for ubuntu 2026-02-21T00:25:03.572 DEBUG:teuthology.orchestra.run.trial120:> rm -f -- /home/ubuntu/.ssh/id_rsa 2026-02-21T00:25:03.628 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:25:03.628 DEBUG:teuthology.orchestra.run.trial120:> dd of=/home/ubuntu/.ssh/id_rsa 2026-02-21T00:25:03.628 DEBUG:teuthology.orchestra.run.trial120:> chmod 0500 /home/ubuntu/.ssh/id_rsa 2026-02-21T00:25:03.685 DEBUG:teuthology.orchestra.run.trial120:> rm -f -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-21T00:25:03.741 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:25:03.741 DEBUG:teuthology.orchestra.run.trial120:> dd of=/home/ubuntu/.ssh/id_rsa.pub 2026-02-21T00:25:03.797 DEBUG:teuthology.orchestra.run.trial120:> cp -v -a /home/ubuntu/.ssh/authorized_keys /home/ubuntu/.ssh/authorized_keys_2026-02-21_00:25:03:797475 2026-02-21T00:25:03.854 INFO:teuthology.orchestra.run.trial120.stdout:'/home/ubuntu/.ssh/authorized_keys' -> '/home/ubuntu/.ssh/authorized_keys_2026-02-21_00:25:03:797475' 2026-02-21T00:25:03.854 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:25:03.854 DEBUG:teuthology.orchestra.run.trial120:> dd of=/home/ubuntu/.ssh/authorized_keys conv=notrunc oflag=append 2026-02-21T00:25:03.911 INFO:teuthology.run_tasks:Running task cbt... 2026-02-21T00:25:03.936 INFO:tasks.cbt:cbt configuration is {'cluster': {'user': 'ubuntu', 'head': 'ubuntu@trial120.front.sepia.ceph.com', 'osds': ['ubuntu@trial120.front.sepia.ceph.com'], 'mons': ['ubuntu@trial120.front.sepia.ceph.com'], 'clients': ['ubuntu@trial120.front.sepia.ceph.com'], 'rgws': {'ubuntu@trial120.front.sepia.ceph.com': None}, 'osds_per_node': 3, 'rebuild_every_test': False, 'use_existing': True, 'is_teuthology': True, 'iterations': 1, 'tmp_dir': '/tmp/cbt', 'pool_profiles': {'replicated': {'pg_size': 256, 'pgp_size': 256, 'replication': 'replicated'}}, 'pid_dir': '/var/run/ceph'}, 'benchmarks': {'radosbench': {'concurrent_ops': 4, 'concurrent_procs': 2, 'op_size': [4194304], 'pool_monitoring_list': ['collectl'], 'pool_profile': 'replicated', 'run_monitoring_list': ['collectl'], 'time': 60, 'write_only': False}}, 'client_endpoints': None, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}} 2026-02-21T00:25:03.937 DEBUG:teuthology.orchestra.run.trial120:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/cbt 2026-02-21T00:25:03.972 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:25:03.972 DEBUG:teuthology.orchestra.run.trial120:> dd of=/home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2026-02-21T00:25:04.024 DEBUG:teuthology.orchestra.run.trial120:> git clone --depth 1 -b master https://github.com/ceph/cbt.git /home/ubuntu/cephtest/cbt 2026-02-21T00:25:04.086 INFO:teuthology.orchestra.run.trial120.stderr:Cloning into '/home/ubuntu/cephtest/cbt'... 2026-02-21T00:25:04.318 INFO:tasks.cbt:Installing collectl 2026-02-21T00:25:04.318 DEBUG:teuthology.orchestra.run.trial120:> sudo mkdir collectl && cd collectl && sudo wget https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download -O collectl.tar.gz && sudo tar -xvf collectl.tar.gz && cd collectl-4.3.1 && sudo ./INSTALL 2026-02-21T00:25:04.392 INFO:teuthology.orchestra.run.trial120.stderr:--2026-02-21 00:25:04-- https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download 2026-02-21T00:25:04.532 INFO:teuthology.orchestra.run.trial120.stderr:Resolving sourceforge.net (sourceforge.net)... 104.18.13.149, 104.18.12.149, 2606:4700::6812:d95, ... 2026-02-21T00:25:04.535 INFO:teuthology.orchestra.run.trial120.stderr:Connecting to sourceforge.net (sourceforge.net)|104.18.13.149|:443... connected. 2026-02-21T00:25:04.726 INFO:teuthology.orchestra.run.trial120.stderr:HTTP request sent, awaiting response... 302 Found 2026-02-21T00:25:04.727 INFO:teuthology.orchestra.run.trial120.stderr:Location: https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABpmPtg8cfE-wJ-gwDYESarOfiEClxUie4cbNN7jbT2ersFT7woYlNXeWtBqjDgjdQVGK7ij0HJnSdkHW7pNM_whSBpWg%3D%3D&use_mirror=phoenixnap&r= [following] 2026-02-21T00:25:04.727 INFO:teuthology.orchestra.run.trial120.stderr:--2026-02-21 00:25:04-- https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABpmPtg8cfE-wJ-gwDYESarOfiEClxUie4cbNN7jbT2ersFT7woYlNXeWtBqjDgjdQVGK7ij0HJnSdkHW7pNM_whSBpWg%3D%3D&use_mirror=phoenixnap&r= 2026-02-21T00:25:04.794 INFO:teuthology.orchestra.run.trial120.stderr:Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 104.18.13.149, 104.18.12.149, 2606:4700::6812:c95, ... 2026-02-21T00:25:04.797 INFO:teuthology.orchestra.run.trial120.stderr:Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|104.18.13.149|:443... connected. 2026-02-21T00:25:05.009 INFO:teuthology.orchestra.run.trial120.stderr:HTTP request sent, awaiting response... 302 Found 2026-02-21T00:25:05.009 INFO:teuthology.orchestra.run.trial120.stderr:Location: https://phoenixnap.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 [following] 2026-02-21T00:25:05.009 INFO:teuthology.orchestra.run.trial120.stderr:--2026-02-21 00:25:05-- https://phoenixnap.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 2026-02-21T00:25:05.022 INFO:teuthology.orchestra.run.trial120.stderr:Resolving phoenixnap.dl.sourceforge.net (phoenixnap.dl.sourceforge.net)... 184.164.141.26 2026-02-21T00:25:05.084 INFO:teuthology.orchestra.run.trial120.stderr:Connecting to phoenixnap.dl.sourceforge.net (phoenixnap.dl.sourceforge.net)|184.164.141.26|:443... connected. 2026-02-21T00:25:05.377 INFO:teuthology.orchestra.run.trial120.stderr:HTTP request sent, awaiting response... 200 OK 2026-02-21T00:25:05.378 INFO:teuthology.orchestra.run.trial120.stderr:Length: 620693 (606K) [application/x-gzip] 2026-02-21T00:25:05.378 INFO:teuthology.orchestra.run.trial120.stderr:Saving to: ‘collectl.tar.gz’ 2026-02-21T00:25:05.378 INFO:teuthology.orchestra.run.trial120.stderr: 2026-02-21T00:25:05.490 INFO:teuthology.orchestra.run.trial120.stderr: 0K .......... .......... .......... .......... .......... 8% 445K 1s 2026-02-21T00:25:05.500 INFO:teuthology.orchestra.run.trial120.stderr: 50K .......... .......... .......... .......... .......... 16% 4.58M 1s 2026-02-21T00:25:05.552 INFO:teuthology.orchestra.run.trial120.stderr: 100K .......... .......... .......... .......... .......... 24% 979K 1s 2026-02-21T00:25:05.561 INFO:teuthology.orchestra.run.trial120.stderr: 150K .......... .......... .......... .......... .......... 32% 4.97M 0s 2026-02-21T00:25:05.615 INFO:teuthology.orchestra.run.trial120.stderr: 200K .......... .......... .......... .......... .......... 41% 977K 0s 2026-02-21T00:25:05.615 INFO:teuthology.orchestra.run.trial120.stderr: 250K .......... .......... .......... .......... .......... 49% 77.4M 0s 2026-02-21T00:25:05.615 INFO:teuthology.orchestra.run.trial120.stderr: 300K .......... .......... .......... .......... .......... 57% 373M 0s 2026-02-21T00:25:05.615 INFO:teuthology.orchestra.run.trial120.stderr: 350K .......... .......... .......... .......... .......... 65% 136M 0s 2026-02-21T00:25:05.624 INFO:teuthology.orchestra.run.trial120.stderr: 400K .......... .......... .......... .......... .......... 74% 5.14M 0s 2026-02-21T00:25:05.624 INFO:teuthology.orchestra.run.trial120.stderr: 450K .......... .......... .......... .......... .......... 82% 83.1M 0s 2026-02-21T00:25:05.676 INFO:teuthology.orchestra.run.trial120.stderr: 500K .......... .......... .......... .......... .......... 90% 979K 0s 2026-02-21T00:25:05.676 INFO:teuthology.orchestra.run.trial120.stderr: 550K .......... .......... .......... .......... .......... 98% 174M 0s 2026-02-21T00:25:05.676 INFO:teuthology.orchestra.run.trial120.stderr: 600K ...... 100% 11.4T=0.3s 2026-02-21T00:25:05.677 INFO:teuthology.orchestra.run.trial120.stderr: 2026-02-21T00:25:05.677 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21 00:25:05 (1.99 MB/s) - ‘collectl.tar.gz’ saved [620693/620693] 2026-02-21T00:25:05.677 INFO:teuthology.orchestra.run.trial120.stderr: 2026-02-21T00:25:05.697 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/ 2026-02-21T00:25:05.697 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/collectl.conf 2026-02-21T00:25:05.697 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/ 2026-02-21T00:25:05.697 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/SlowProc.html 2026-02-21T00:25:05.697 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/BuddyInfo.html 2026-02-21T00:25:05.697 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/WhySummary.html 2026-02-21T00:25:05.698 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Socket.html 2026-02-21T00:25:05.698 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/colmux.html 2026-02-21T00:25:05.698 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Tutorial.html 2026-02-21T00:25:05.698 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Gexpr.html 2026-02-21T00:25:05.698 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Import.html 2026-02-21T00:25:05.698 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/style.css 2026-02-21T00:25:05.698 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Ganglia3.jpg 2026-02-21T00:25:05.698 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Ganglia2.jpg 2026-02-21T00:25:05.698 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/FileNaming.html 2026-02-21T00:25:05.698 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/TheMath.html 2026-02-21T00:25:05.698 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Environmental.html 2026-02-21T00:25:05.698 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Process.html 2026-02-21T00:25:05.698 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/OutputFiles.html 2026-02-21T00:25:05.699 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/HiResTime.html 2026-02-21T00:25:05.699 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/OutputFormats.html 2026-02-21T00:25:05.699 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/slub.jpg 2026-02-21T00:25:05.699 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Lustre.html 2026-02-21T00:25:05.699 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Documentation.html 2026-02-21T00:25:05.699 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Inodes.html 2026-02-21T00:25:05.699 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Graphite.html 2026-02-21T00:25:05.699 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/InputFiles.html 2026-02-21T00:25:05.699 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Export.html 2026-02-21T00:25:05.699 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Interrupts.html 2026-02-21T00:25:05.699 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Startup.html 2026-02-21T00:25:05.699 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Performance.html 2026-02-21T00:25:05.699 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Messages.html 2026-02-21T00:25:05.700 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Logging.html 2026-02-21T00:25:05.700 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Data-detail.html 2026-02-21T00:25:05.700 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/CPUs.html 2026-02-21T00:25:05.700 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Matrix.html 2026-02-21T00:25:05.700 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Data-brief.html 2026-02-21T00:25:05.700 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/SlabInfo.html 2026-02-21T00:25:05.700 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Misc.html 2026-02-21T00:25:05.700 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Examples.html 2026-02-21T00:25:05.700 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/OpenStack.html 2026-02-21T00:25:05.700 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Plotfiles.html 2026-02-21T00:25:05.700 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/ColmuxCPU.jpg 2026-02-21T00:25:05.700 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Tutorial-Lustre.html 2026-02-21T00:25:05.700 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Data-verbose.html 2026-02-21T00:25:05.701 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Data.html 2026-02-21T00:25:05.701 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Exceptions.html 2026-02-21T00:25:05.701 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/RunningAsAService.html 2026-02-21T00:25:05.701 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/FAQ-collectl.html 2026-02-21T00:25:05.701 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Ganglia.jpg 2026-02-21T00:25:05.701 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/OperationalModes.html 2026-02-21T00:25:05.701 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Playback.html 2026-02-21T00:25:05.701 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Hello.html 2026-02-21T00:25:05.701 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Sockets.html 2026-02-21T00:25:05.701 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Network.html 2026-02-21T00:25:05.701 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Infiniband.html 2026-02-21T00:25:05.701 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Memory.html 2026-02-21T00:25:05.701 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/ColmuxLustre.jpg 2026-02-21T00:25:05.702 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/Disks.html 2026-02-21T00:25:05.702 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/docs/NfsInfo.html 2026-02-21T00:25:05.702 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/service/ 2026-02-21T00:25:05.702 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/service/collectl.service 2026-02-21T00:25:05.702 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/initd/ 2026-02-21T00:25:05.702 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/initd/collectl-suse 2026-02-21T00:25:05.702 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/initd/collectl-debian 2026-02-21T00:25:05.702 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/initd/collectl-generic 2026-02-21T00:25:05.702 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/initd/collectl 2026-02-21T00:25:05.702 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/misc.ph 2026-02-21T00:25:05.702 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/gexpr.ph 2026-02-21T00:25:05.702 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/statsd.ph 2026-02-21T00:25:05.702 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/envrules.std 2026-02-21T00:25:05.702 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/hello.ph 2026-02-21T00:25:05.703 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/vmsum.ph 2026-02-21T00:25:05.703 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/proctree.ph 2026-02-21T00:25:05.703 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/README-WINDOWS 2026-02-21T00:25:05.703 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/formatit.ph 2026-02-21T00:25:05.703 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/colmux 2026-02-21T00:25:05.703 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/INSTALL 2026-02-21T00:25:05.703 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/graphite.ph 2026-02-21T00:25:05.703 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/RELEASE-collectl 2026-02-21T00:25:05.703 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/vmstat.ph 2026-02-21T00:25:05.703 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/README 2026-02-21T00:25:05.703 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/man1/ 2026-02-21T00:25:05.703 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/man1/collectl.1 2026-02-21T00:25:05.703 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/man1/colmux.1 2026-02-21T00:25:05.704 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/lexpr.ph 2026-02-21T00:25:05.704 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/client.pl 2026-02-21T00:25:05.704 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/collectl 2026-02-21T00:25:05.704 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/COPYING 2026-02-21T00:25:05.704 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/vnet.ph 2026-02-21T00:25:05.704 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/GPL 2026-02-21T00:25:05.704 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/UNINSTALL 2026-02-21T00:25:05.704 INFO:teuthology.orchestra.run.trial120.stdout:collectl-4.3.1/ARTISTIC 2026-02-21T00:25:05.739 INFO:teuthology.orchestra.run.trial120.stderr:./INSTALL: line 90: chkconfig: command not found 2026-02-21T00:25:05.741 DEBUG:teuthology.orchestra.run.trial120:> sudo yum -y install librbd-devel pdsh pdsh-rcmd-ssh perf 2026-02-21T00:25:06.091 INFO:teuthology.orchestra.run.trial120.stdout:Last metadata expiration check: 0:00:57 ago on Sat 21 Feb 2026 12:24:09 AM UTC. 2026-02-21T00:25:06.177 INFO:teuthology.orchestra.run.trial120.stdout:Dependencies resolved. 2026-02-21T00:25:06.178 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:25:06.178 INFO:teuthology.orchestra.run.trial120.stdout: Package Arch Version Repository Size 2026-02-21T00:25:06.178 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:25:06.178 INFO:teuthology.orchestra.run.trial120.stdout:Installing: 2026-02-21T00:25:06.178 INFO:teuthology.orchestra.run.trial120.stdout: librbd-devel x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 24 k 2026-02-21T00:25:06.178 INFO:teuthology.orchestra.run.trial120.stdout: pdsh x86_64 2.34-7.el9 epel 118 k 2026-02-21T00:25:06.178 INFO:teuthology.orchestra.run.trial120.stdout: pdsh-rcmd-ssh x86_64 2.34-7.el9 epel 13 k 2026-02-21T00:25:06.178 INFO:teuthology.orchestra.run.trial120.stdout: perf x86_64 5.14.0-681.el9 appstream 3.2 M 2026-02-21T00:25:06.178 INFO:teuthology.orchestra.run.trial120.stdout:Installing dependencies: 2026-02-21T00:25:06.179 INFO:teuthology.orchestra.run.trial120.stdout: libradospp-devel x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 31 k 2026-02-21T00:25:06.179 INFO:teuthology.orchestra.run.trial120.stdout: libtraceevent x86_64 1.8.4-2.el9 baseos 290 k 2026-02-21T00:25:06.179 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:25:06.179 INFO:teuthology.orchestra.run.trial120.stdout:Transaction Summary 2026-02-21T00:25:06.179 INFO:teuthology.orchestra.run.trial120.stdout:================================================================================ 2026-02-21T00:25:06.179 INFO:teuthology.orchestra.run.trial120.stdout:Install 6 Packages 2026-02-21T00:25:06.179 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:25:06.179 INFO:teuthology.orchestra.run.trial120.stdout:Total download size: 3.7 M 2026-02-21T00:25:06.179 INFO:teuthology.orchestra.run.trial120.stdout:Installed size: 14 M 2026-02-21T00:25:06.179 INFO:teuthology.orchestra.run.trial120.stdout:Downloading Packages: 2026-02-21T00:25:06.467 INFO:teuthology.orchestra.run.trial120.stdout:(1/6): libtraceevent-1.8.4-2.el9.x86_64.rpm 5.2 MB/s | 290 kB 00:00 2026-02-21T00:25:06.473 INFO:teuthology.orchestra.run.trial120.stdout:(2/6): librbd-devel-20.3.0-5343.gbcdd429b.el9.x 391 kB/s | 24 kB 00:00 2026-02-21T00:25:06.484 INFO:teuthology.orchestra.run.trial120.stdout:(3/6): libradospp-devel-20.3.0-5343.gbcdd429b.e 431 kB/s | 31 kB 00:00 2026-02-21T00:25:06.506 INFO:teuthology.orchestra.run.trial120.stdout:(4/6): pdsh-rcmd-ssh-2.34-7.el9.x86_64.rpm 619 kB/s | 13 kB 00:00 2026-02-21T00:25:06.525 INFO:teuthology.orchestra.run.trial120.stdout:(5/6): pdsh-2.34-7.el9.x86_64.rpm 2.2 MB/s | 118 kB 00:00 2026-02-21T00:25:07.252 INFO:teuthology.orchestra.run.trial120.stdout:(6/6): perf-5.14.0-681.el9.x86_64.rpm 4.1 MB/s | 3.2 MB 00:00 2026-02-21T00:25:07.253 INFO:teuthology.orchestra.run.trial120.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:25:07.253 INFO:teuthology.orchestra.run.trial120.stdout:Total 3.4 MB/s | 3.7 MB 00:01 2026-02-21T00:25:07.277 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction check 2026-02-21T00:25:07.288 INFO:teuthology.orchestra.run.trial120.stdout:Transaction check succeeded. 2026-02-21T00:25:07.288 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction test 2026-02-21T00:25:07.343 INFO:teuthology.orchestra.run.trial120.stdout:Transaction test succeeded. 2026-02-21T00:25:07.343 INFO:teuthology.orchestra.run.trial120.stdout:Running transaction 2026-02-21T00:25:07.414 INFO:teuthology.orchestra.run.trial120.stdout: Preparing : 1/1 2026-02-21T00:25:07.420 INFO:teuthology.orchestra.run.trial120.stdout: Installing : pdsh-rcmd-ssh-2.34-7.el9.x86_64 1/6 2026-02-21T00:25:07.459 INFO:teuthology.orchestra.run.trial120.stdout: Installing : pdsh-2.34-7.el9.x86_64 2/6 2026-02-21T00:25:07.463 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libtraceevent-1.8.4-2.el9.x86_64 3/6 2026-02-21T00:25:07.466 INFO:teuthology.orchestra.run.trial120.stdout: Installing : libradospp-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 4/6 2026-02-21T00:25:07.628 INFO:teuthology.orchestra.run.trial120.stdout: Installing : librbd-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 5/6 2026-02-21T00:25:07.649 INFO:teuthology.orchestra.run.trial120.stdout: Installing : perf-5.14.0-681.el9.x86_64 6/6 2026-02-21T00:25:07.757 INFO:teuthology.orchestra.run.trial120.stdout: Running scriptlet: perf-5.14.0-681.el9.x86_64 6/6 2026-02-21T00:25:07.757 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libradospp-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/6 2026-02-21T00:25:07.757 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : librbd-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/6 2026-02-21T00:25:07.758 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : libtraceevent-1.8.4-2.el9.x86_64 3/6 2026-02-21T00:25:07.758 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : perf-5.14.0-681.el9.x86_64 4/6 2026-02-21T00:25:07.758 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : pdsh-2.34-7.el9.x86_64 5/6 2026-02-21T00:25:07.815 INFO:teuthology.orchestra.run.trial120.stdout: Verifying : pdsh-rcmd-ssh-2.34-7.el9.x86_64 6/6 2026-02-21T00:25:07.815 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:25:07.815 INFO:teuthology.orchestra.run.trial120.stdout:Installed: 2026-02-21T00:25:07.815 INFO:teuthology.orchestra.run.trial120.stdout: libradospp-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:25:07.815 INFO:teuthology.orchestra.run.trial120.stdout: librbd-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:25:07.816 INFO:teuthology.orchestra.run.trial120.stdout: libtraceevent-1.8.4-2.el9.x86_64 2026-02-21T00:25:07.816 INFO:teuthology.orchestra.run.trial120.stdout: pdsh-2.34-7.el9.x86_64 2026-02-21T00:25:07.816 INFO:teuthology.orchestra.run.trial120.stdout: pdsh-rcmd-ssh-2.34-7.el9.x86_64 2026-02-21T00:25:07.816 INFO:teuthology.orchestra.run.trial120.stdout: perf-5.14.0-681.el9.x86_64 2026-02-21T00:25:07.816 INFO:teuthology.orchestra.run.trial120.stdout: 2026-02-21T00:25:07.816 INFO:teuthology.orchestra.run.trial120.stdout:Complete! 2026-02-21T00:25:07.847 DEBUG:teuthology.orchestra.run.trial120:> python3 -m pip install -r /home/ubuntu/cephtest/cbt/requirements.txt 2026-02-21T00:25:07.986 INFO:teuthology.orchestra.run.trial120.stdout:Defaulting to user installation because normal site-packages is not writeable 2026-02-21T00:25:07.997 INFO:teuthology.orchestra.run.trial120.stdout:Requirement already satisfied: pyyaml in /usr/lib64/python3.9/site-packages (from -r /home/ubuntu/cephtest/cbt/requirements.txt (line 1)) (5.4.1) 2026-02-21T00:25:08.457 INFO:teuthology.orchestra.run.trial120.stdout:Collecting lxml 2026-02-21T00:25:08.490 INFO:teuthology.orchestra.run.trial120.stdout: Downloading lxml-6.0.2-cp39-cp39-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl (5.3 MB) 2026-02-21T00:25:08.869 INFO:teuthology.orchestra.run.trial120.stdout:Collecting matplotlib 2026-02-21T00:25:08.877 INFO:teuthology.orchestra.run.trial120.stdout: Downloading matplotlib-3.9.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.3 MB) 2026-02-21T00:25:08.954 INFO:teuthology.orchestra.run.trial120.stdout:Requirement already satisfied: packaging>=20.0 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (20.9) 2026-02-21T00:25:08.955 INFO:teuthology.orchestra.run.trial120.stdout:Requirement already satisfied: pyparsing>=2.3.1 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (2.4.7) 2026-02-21T00:25:08.990 INFO:teuthology.orchestra.run.trial120.stdout:Collecting importlib-resources>=3.2.0 2026-02-21T00:25:08.994 INFO:teuthology.orchestra.run.trial120.stdout: Downloading importlib_resources-6.5.2-py3-none-any.whl (37 kB) 2026-02-21T00:25:09.318 INFO:teuthology.orchestra.run.trial120.stdout:Collecting fonttools>=4.22.0 2026-02-21T00:25:09.325 INFO:teuthology.orchestra.run.trial120.stdout: Downloading fonttools-4.60.2-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.8 MB) 2026-02-21T00:25:09.759 INFO:teuthology.orchestra.run.trial120.stdout:Collecting pillow>=8 2026-02-21T00:25:09.769 INFO:teuthology.orchestra.run.trial120.stdout: Downloading pillow-11.3.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (6.6 MB) 2026-02-21T00:25:09.978 INFO:teuthology.orchestra.run.trial120.stdout:Collecting contourpy>=1.0.1 2026-02-21T00:25:09.984 INFO:teuthology.orchestra.run.trial120.stdout: Downloading contourpy-1.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (321 kB) 2026-02-21T00:25:10.122 INFO:teuthology.orchestra.run.trial120.stdout:Collecting kiwisolver>=1.3.1 2026-02-21T00:25:10.130 INFO:teuthology.orchestra.run.trial120.stdout: Downloading kiwisolver-1.4.7-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB) 2026-02-21T00:25:10.142 INFO:teuthology.orchestra.run.trial120.stdout:Requirement already satisfied: numpy>=1.23 in /usr/lib64/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (1.23.5) 2026-02-21T00:25:10.143 INFO:teuthology.orchestra.run.trial120.stdout:Requirement already satisfied: python-dateutil>=2.7 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (2.9.0.post0) 2026-02-21T00:25:10.156 INFO:teuthology.orchestra.run.trial120.stdout:Collecting cycler>=0.10 2026-02-21T00:25:10.161 INFO:teuthology.orchestra.run.trial120.stdout: Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB) 2026-02-21T00:25:10.239 INFO:teuthology.orchestra.run.trial120.stdout:Collecting zipp>=3.1.0 2026-02-21T00:25:10.244 INFO:teuthology.orchestra.run.trial120.stdout: Downloading zipp-3.23.0-py3-none-any.whl (10 kB) 2026-02-21T00:25:10.255 INFO:teuthology.orchestra.run.trial120.stdout:Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (1.15.0) 2026-02-21T00:25:10.369 INFO:teuthology.orchestra.run.trial120.stdout:Installing collected packages: zipp, pillow, kiwisolver, importlib-resources, fonttools, cycler, contourpy, matplotlib, lxml 2026-02-21T00:25:11.504 INFO:teuthology.orchestra.run.trial120.stdout:Successfully installed contourpy-1.3.0 cycler-0.12.1 fonttools-4.60.2 importlib-resources-6.5.2 kiwisolver-1.4.7 lxml-6.0.2 matplotlib-3.9.4 pillow-11.3.0 zipp-3.23.0 2026-02-21T00:25:11.643 INFO:tasks.cbt:benchmark: radosbench 2026-02-21T00:25:11.643 DEBUG:teuthology.orchestra.run.trial120:> sudo /sbin/sysctl -q -w kernel.perf_event_paranoid=0 2026-02-21T00:25:11.669 DEBUG:teuthology.orchestra.run.trial120:> /home/ubuntu/cephtest/cbt/cbt.py -a /home/ubuntu/cephtest/archive/cbt /home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2026-02-21T00:25:11.780 INFO:teuthology.orchestra.run.trial120.stderr:00:25:11 - DEBUG - cbt - Settings.cluster: 2026-02-21T00:25:11.780 INFO:teuthology.orchestra.run.trial120.stderr: {'archive_dir': '/home/ubuntu/cephtest/archive/cbt', 2026-02-21T00:25:11.780 INFO:teuthology.orchestra.run.trial120.stderr: 'clients': ['ubuntu@trial120.front.sepia.ceph.com'], 2026-02-21T00:25:11.780 INFO:teuthology.orchestra.run.trial120.stderr: 'conf_file': '/etc/ceph/ceph.conf', 2026-02-21T00:25:11.781 INFO:teuthology.orchestra.run.trial120.stderr: 'head': 'ubuntu@trial120.front.sepia.ceph.com', 2026-02-21T00:25:11.781 INFO:teuthology.orchestra.run.trial120.stderr: 'is_teuthology': True, 2026-02-21T00:25:11.781 INFO:teuthology.orchestra.run.trial120.stderr: 'iterations': 1, 2026-02-21T00:25:11.781 INFO:teuthology.orchestra.run.trial120.stderr: 'mons': ['ubuntu@trial120.front.sepia.ceph.com'], 2026-02-21T00:25:11.781 INFO:teuthology.orchestra.run.trial120.stderr: 'osds': ['ubuntu@trial120.front.sepia.ceph.com'], 2026-02-21T00:25:11.781 INFO:teuthology.orchestra.run.trial120.stderr: 'osds_per_node': 3, 2026-02-21T00:25:11.781 INFO:teuthology.orchestra.run.trial120.stderr: 'pid_dir': '/var/run/ceph', 2026-02-21T00:25:11.781 INFO:teuthology.orchestra.run.trial120.stderr: 'pool_profiles': {'replicated': {'pg_size': 256, 2026-02-21T00:25:11.781 INFO:teuthology.orchestra.run.trial120.stderr: 'pgp_size': 256, 2026-02-21T00:25:11.781 INFO:teuthology.orchestra.run.trial120.stderr: 'replication': 'replicated'}}, 2026-02-21T00:25:11.781 INFO:teuthology.orchestra.run.trial120.stderr: 'rebuild_every_test': False, 2026-02-21T00:25:11.781 INFO:teuthology.orchestra.run.trial120.stderr: 'rgws': {'ubuntu@trial120.front.sepia.ceph.com': None}, 2026-02-21T00:25:11.782 INFO:teuthology.orchestra.run.trial120.stderr: 'tmp_dir': '/tmp/cbt', 2026-02-21T00:25:11.782 INFO:teuthology.orchestra.run.trial120.stderr: 'use_existing': True, 2026-02-21T00:25:11.782 INFO:teuthology.orchestra.run.trial120.stderr: 'user': 'ubuntu'} 2026-02-21T00:25:11.782 INFO:teuthology.orchestra.run.trial120.stderr:00:25:11 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d 2026-02-21T00:25:11.782 INFO:teuthology.orchestra.run.trial120.stderr:00:25:11 - INFO - cbt - Pausing for 60s for idle monitoring. 2026-02-21T00:25:11.782 INFO:teuthology.orchestra.run.trial120.stderr:00:25:11 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/collectl 2026-02-21T00:25:11.952 INFO:teuthology.orchestra.run.trial120.stderr:00:25:11 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-21T00:25:11.953 INFO:teuthology.orchestra.run.trial120.stderr:00:25:11 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/perf 2026-02-21T00:25:12.120 INFO:teuthology.orchestra.run.trial120.stderr:00:25:12 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-21T00:25:12.120 INFO:teuthology.orchestra.run.trial120.stderr:00:25:12 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/perf/perf_stat.${pid} & done 2026-02-21T00:26:12.164 INFO:teuthology.orchestra.run.trial120.stderr:00:26:12 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-21T00:26:12.344 INFO:teuthology.orchestra.run.trial120.stderr:00:26:12 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-21T00:26:12.532 INFO:teuthology.orchestra.run.trial120.stderr:00:26:12 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/perf/perf.data 2026-02-21T00:26:12.533 INFO:teuthology.orchestra.run.trial120.stderr:00:26:12 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/perf/perf_stat.* 2026-02-21T00:26:12.535 INFO:teuthology.orchestra.run.trial120.stderr:00:26:12 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/* 2026-02-21T00:26:12.718 INFO:teuthology.orchestra.run.trial120.stderr:00:26:12 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial120.front.sepia.ceph.com -r /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d 2026-02-21T00:26:12.888 INFO:teuthology.orchestra.run.trial120.stderr:00:26:12 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo killall -9 rados 2026-02-21T00:26:13.068 INFO:teuthology.orchestra.run.trial120.stderr:00:26:13 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d 2026-02-21T00:26:13.068 INFO:teuthology.orchestra.run.trial120.stderr:00:26:13 - INFO - cbt - Skipping existing test in /home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d. 2026-02-21T00:26:13.068 INFO:teuthology.orchestra.run.trial120.stderr:00:26:13 - INFO - cbt - Running benchmark /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004 2026-02-21T00:26:13.068 INFO:teuthology.orchestra.run.trial120.stderr:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d 2026-02-21T00:26:13.068 INFO:teuthology.orchestra.run.trial120.stderr:{'concurrent_ops': 4, 'concurrent_procs': 2, 'pool_profile': 'replicated', 'time': 60, 'write_only': False, 'op_size': 4194304, 'pool_monitoring_list': 'collectl', 'run_monitoring_list': 'collectl', 'benchmark': 'radosbench', 'iteration': 0} == iteration 0 == 2026-02-21T00:26:13.068 INFO:teuthology.orchestra.run.trial120.stderr:00:26:13 - INFO - cbt - Setting OSD Read Ahead to: 0 2026-02-21T00:26:13.068 INFO:teuthology.orchestra.run.trial120.stderr:00:26:13 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com find /dev/disk/by-partlabel/osd-device-*data -exec readlink {} \; | cut -d"/" -f 3 | sed "s/[0-9]$//" | xargs -I{} sudo sh -c "echo 0 > /sys/block/'{}'/queue/read_ahead_kb" 2026-02-21T00:26:13.069 INFO:teuthology.orchestra.run.trial120.stderr:00:26:13 - DEBUG - cbt - Cleaning existing temporary run directory: /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004 2026-02-21T00:26:13.069 INFO:teuthology.orchestra.run.trial120.stderr:00:26:13 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo rm -rf /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004 2026-02-21T00:26:13.244 INFO:teuthology.orchestra.run.trial120.stderr:00:26:13 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/collectl 2026-02-21T00:26:13.407 INFO:teuthology.orchestra.run.trial120.stderr:00:26:13 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-21T00:26:13.409 INFO:teuthology.orchestra.run.trial120.stderr:00:26:13 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/perf 2026-02-21T00:26:13.574 INFO:teuthology.orchestra.run.trial120.stderr:00:26:13 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-21T00:26:13.574 INFO:teuthology.orchestra.run.trial120.stderr:00:26:13 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/perf/perf_stat.${pid} & done 2026-02-21T00:26:13.576 INFO:teuthology.orchestra.run.trial120.stderr:00:26:13 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool delete rados-bench-cbt rados-bench-cbt --yes-i-really-really-mean-it 2026-02-21T00:26:13.920 INFO:teuthology.orchestra.run.trial120.stderr:00:26:13 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool create rados-bench-cbt 256 256 2026-02-21T00:26:14.649 INFO:teuthology.orchestra.run.trial120.stderr:00:26:14 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool application enable rados-bench-cbt radosbench 2026-02-21T00:26:15.652 INFO:teuthology.orchestra.run.trial120.stderr:00:26:15 - INFO - cbt - Checking Health after pool creation. 2026-02-21T00:26:20.658 INFO:teuthology.orchestra.run.trial120.stderr:00:26:20 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com /usr/bin/ceph -c /etc/ceph/ceph.conf health 2026-02-21T00:26:21.018 INFO:teuthology.orchestra.run.trial120.stderr:00:26:21 - INFO - cbt - Final Pool Health Check. 2026-02-21T00:26:26.023 INFO:teuthology.orchestra.run.trial120.stderr:00:26:26 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com /usr/bin/ceph -c /etc/ceph/ceph.conf health 2026-02-21T00:26:26.381 INFO:teuthology.orchestra.run.trial120.stderr:00:26:26 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-21T00:26:26.553 INFO:teuthology.orchestra.run.trial120.stderr:00:26:26 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-21T00:26:26.734 INFO:teuthology.orchestra.run.trial120.stderr:00:26:26 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/perf/perf.data 2026-02-21T00:26:26.735 INFO:teuthology.orchestra.run.trial120.stderr:00:26:26 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/perf/perf_stat.* 2026-02-21T00:26:26.737 INFO:teuthology.orchestra.run.trial120.stderr:00:26:26 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sync 2026-02-21T00:26:26.910 INFO:teuthology.orchestra.run.trial120.stderr:00:26:26 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com echo 3 | sudo tee /proc/sys/vm/drop_caches 2026-02-21T00:26:27.878 INFO:teuthology.orchestra.run.trial120.stderr:00:26:27 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -v 2026-02-21T00:26:28.102 INFO:teuthology.orchestra.run.trial120.stderr:00:26:28 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write 2026-02-21T00:26:28.268 INFO:teuthology.orchestra.run.trial120.stderr:00:26:28 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf daemon osd.0 config show > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/ceph_settings.out 2026-02-21T00:26:28.563 INFO:teuthology.orchestra.run.trial120.stderr:00:26:28 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/collectl 2026-02-21T00:26:28.726 INFO:teuthology.orchestra.run.trial120.stderr:00:26:28 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-21T00:26:28.728 INFO:teuthology.orchestra.run.trial120.stderr:00:26:28 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/perf 2026-02-21T00:26:28.895 INFO:teuthology.orchestra.run.trial120.stderr:00:26:28 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-21T00:26:28.896 INFO:teuthology.orchestra.run.trial120.stderr:00:26:28 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/perf/perf_stat.${pid} & done 2026-02-21T00:26:28.897 INFO:teuthology.orchestra.run.trial120.stderr:00:26:28 - INFO - cbt - Running radosbench write test. 2026-02-21T00:26:28.897 INFO:teuthology.orchestra.run.trial120.stderr:00:26:28 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench -b 4194304 60 write --concurrent-ios 4 --run-name `hostname -f`-0 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/objecter.0.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/output.0 2026-02-21T00:26:28.898 INFO:teuthology.orchestra.run.trial120.stderr:00:26:28 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench -b 4194304 60 write --concurrent-ios 4 --run-name `hostname -f`-1 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/objecter.1.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/output.1 2026-02-21T00:27:30.136 INFO:teuthology.orchestra.run.trial120.stderr:00:27:30 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-21T00:27:30.316 INFO:teuthology.orchestra.run.trial120.stderr:00:27:30 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-21T00:27:30.502 INFO:teuthology.orchestra.run.trial120.stderr:00:27:30 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/perf/perf.data 2026-02-21T00:27:30.504 INFO:teuthology.orchestra.run.trial120.stderr:00:27:30 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/perf/perf_stat.* 2026-02-21T00:27:30.505 INFO:teuthology.orchestra.run.trial120.stderr:00:27:30 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com find /var/run/ceph/ceph-osd*.asok -maxdepth 1 -exec sudo /usr/bin/ceph --admin-daemon {} dump_historic_ops \; > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/historic_ops.out 2026-02-21T00:27:30.868 INFO:teuthology.orchestra.run.trial120.stderr:00:27:30 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/* 2026-02-21T00:27:31.048 INFO:teuthology.orchestra.run.trial120.stderr:00:27:31 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial120.front.sepia.ceph.com -r /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d/write 2026-02-21T00:27:31.228 INFO:teuthology.orchestra.run.trial120.stderr:00:27:31 - INFO - cbt - Convert results to json format. 2026-02-21T00:27:31.228 INFO:teuthology.orchestra.run.trial120.stderr:00:27:31 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sync 2026-02-21T00:27:31.446 INFO:teuthology.orchestra.run.trial120.stderr:00:27:31 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com echo 3 | sudo tee /proc/sys/vm/drop_caches 2026-02-21T00:27:31.769 INFO:teuthology.orchestra.run.trial120.stderr:00:27:31 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -v 2026-02-21T00:27:31.982 INFO:teuthology.orchestra.run.trial120.stderr:00:27:31 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/seq 2026-02-21T00:27:32.162 INFO:teuthology.orchestra.run.trial120.stderr:00:27:32 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf daemon osd.0 config show > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/seq/ceph_settings.out 2026-02-21T00:27:32.451 INFO:teuthology.orchestra.run.trial120.stderr:00:27:32 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/seq/collectl 2026-02-21T00:27:32.617 INFO:teuthology.orchestra.run.trial120.stderr:00:27:32 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/seq/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-21T00:27:32.619 INFO:teuthology.orchestra.run.trial120.stderr:00:27:32 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/seq/perf 2026-02-21T00:27:32.788 INFO:teuthology.orchestra.run.trial120.stderr:00:27:32 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-21T00:27:32.789 INFO:teuthology.orchestra.run.trial120.stderr:00:27:32 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/seq/perf/perf_stat.${pid} & done 2026-02-21T00:27:32.790 INFO:teuthology.orchestra.run.trial120.stderr:00:27:32 - INFO - cbt - Running radosbench seq test. 2026-02-21T00:27:32.790 INFO:teuthology.orchestra.run.trial120.stderr:00:27:32 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench 60 seq --concurrent-ios 4 --run-name `hostname -f`-0 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/seq/objecter.0.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/seq/output.0 2026-02-21T00:27:32.791 INFO:teuthology.orchestra.run.trial120.stderr:00:27:32 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench 60 seq --concurrent-ios 4 --run-name `hostname -f`-1 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/seq/objecter.1.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/seq/output.1 2026-02-21T00:27:48.030 INFO:teuthology.orchestra.run.trial120.stderr:00:27:48 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-21T00:27:48.206 INFO:teuthology.orchestra.run.trial120.stderr:00:27:48 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-21T00:27:48.391 INFO:teuthology.orchestra.run.trial120.stderr:00:27:48 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/seq/perf/perf.data 2026-02-21T00:27:48.393 INFO:teuthology.orchestra.run.trial120.stderr:00:27:48 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/seq/perf/perf_stat.* 2026-02-21T00:27:48.395 INFO:teuthology.orchestra.run.trial120.stderr:00:27:48 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com find /var/run/ceph/ceph-osd*.asok -maxdepth 1 -exec sudo /usr/bin/ceph --admin-daemon {} dump_historic_ops \; > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/seq/historic_ops.out 2026-02-21T00:27:48.753 INFO:teuthology.orchestra.run.trial120.stderr:00:27:48 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial120.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/seq/* 2026-02-21T00:27:48.931 INFO:teuthology.orchestra.run.trial120.stderr:00:27:48 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial120.front.sepia.ceph.com -r /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/seq/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d/seq 2026-02-21T00:27:49.097 INFO:teuthology.orchestra.run.trial120.stderr:00:27:49 - INFO - cbt - Convert results to json format. 2026-02-21T00:27:49.108 DEBUG:teuthology.run_tasks:Unwinding manager cbt 2026-02-21T00:27:49.123 DEBUG:teuthology.orchestra.run.trial120:> rm --one-file-system -rf -- /home/ubuntu/cephtest/cbt 2026-02-21T00:27:49.152 INFO:tasks.cbt_performance:Collecting CBT performance data config 2026-02-21T00:27:49.152 INFO:tasks.cbt_performance:reading cbt preferences 2026-02-21T00:27:49.152 DEBUG:teuthology.orchestra.run.trial120:> find /home/ubuntu/cephtest -name 'json_output.*' 2026-02-21T00:27:49.209 INFO:teuthology.orchestra.run.trial120.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d/write/json_output.1.trial120.front.sepia.ceph.com 2026-02-21T00:27:49.209 INFO:teuthology.orchestra.run.trial120.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d/write/json_output.0.trial120.front.sepia.ceph.com 2026-02-21T00:27:49.209 INFO:teuthology.orchestra.run.trial120.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d/seq/json_output.1.trial120.front.sepia.ceph.com 2026-02-21T00:27:49.209 INFO:teuthology.orchestra.run.trial120.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d/seq/json_output.0.trial120.front.sepia.ceph.com 2026-02-21T00:27:49.210 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:27:49.210 DEBUG:teuthology.orchestra.run.trial120:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d/write/json_output.1.trial120.front.sepia.ceph.com of=/dev/stdout 2026-02-21T00:27:49.266 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-21T00:27:49.267 DEBUG:teuthology.orchestra.run.trial120:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d/write -name 'perf_stat.*' 2026-02-21T00:27:49.322 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-21T00:27:49.322 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-21T00:27:49.322 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:27:49.322 DEBUG:teuthology.orchestra.run.trial120:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d/write/json_output.0.trial120.front.sepia.ceph.com of=/dev/stdout 2026-02-21T00:27:49.377 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-21T00:27:49.377 DEBUG:teuthology.orchestra.run.trial120:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d/write -name 'perf_stat.*' 2026-02-21T00:27:49.433 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-21T00:27:49.433 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-21T00:27:49.433 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:27:49.433 DEBUG:teuthology.orchestra.run.trial120:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d/seq/json_output.1.trial120.front.sepia.ceph.com of=/dev/stdout 2026-02-21T00:27:49.489 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-21T00:27:49.490 DEBUG:teuthology.orchestra.run.trial120:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d/seq -name 'perf_stat.*' 2026-02-21T00:27:49.546 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-21T00:27:49.546 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-21T00:27:49.546 DEBUG:teuthology.orchestra.run.trial120:> set -ex 2026-02-21T00:27:49.546 DEBUG:teuthology.orchestra.run.trial120:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d/seq/json_output.0.trial120.front.sepia.ceph.com of=/dev/stdout 2026-02-21T00:27:49.602 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-21T00:27:49.602 DEBUG:teuthology.orchestra.run.trial120:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d/seq -name 'perf_stat.*' 2026-02-21T00:27:49.659 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-21T00:27:49.659 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-21T00:27:49.665 WARNING:tasks.cbt_performance:Could not send performance data to http://mira118.front.sepia.ceph.com:4000/cbt_performance: HTTPConnectionPool(host='mira118.front.sepia.ceph.com', port=4000): Max retries exceeded with url: /cbt_performance (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known')) 2026-02-21T00:27:49.665 WARNING:tasks.cbt_performance:Continuing without performance data collection 2026-02-21T00:27:49.665 DEBUG:teuthology.run_tasks:Unwinding manager ssh_keys 2026-02-21T00:27:49.678 INFO:teuthology.task.ssh_keys:Cleaning up SSH keys 2026-02-21T00:27:49.678 INFO:teuthology.task.ssh_keys:cleaning up keys added for testing 2026-02-21T00:27:49.678 INFO:teuthology.task.ssh_keys: cleaning up keys for user ubuntu on trial120.front.sepia.ceph.com 2026-02-21T00:27:49.679 DEBUG:teuthology.orchestra.run.trial120:> rm -- /home/ubuntu/.ssh/id_rsa 2026-02-21T00:27:49.715 DEBUG:teuthology.orchestra.run.trial120:> rm -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-21T00:27:49.772 DEBUG:teuthology.orchestra.run.trial120:> stat -c '"%a"' /home/ubuntu/.ssh/authorized_keys 2026-02-21T00:27:49.829 INFO:teuthology.orchestra.run.trial120.stdout:"600" 2026-02-21T00:27:49.830 DEBUG:teuthology.orchestra.run.trial120:> mv -- /home/ubuntu/.ssh/authorized_keys_2026-02-21_00:25:03:797475 /home/ubuntu/.ssh/authorized_keys 2026-02-21T00:27:49.887 DEBUG:teuthology.orchestra.run.trial120:> chmod 600 /home/ubuntu/.ssh/authorized_keys 2026-02-21T00:27:49.943 DEBUG:teuthology.orchestra.run.trial120:> rm /home/ubuntu/.ssh/config 2026-02-21T00:27:50.001 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-21T00:27:50.012 DEBUG:teuthology.orchestra.run.trial120:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-21T00:27:50.115 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:27:50.110+0000 7f85097d3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:27:50.116 INFO:teuthology.orchestra.run.trial120.stderr:2026-02-21T00:27:50.115+0000 7f85097d3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:27:50.245 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-21T00:27:50.246 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-21T00:27:50.246 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-21T00:27:50.246 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:27:50.317 INFO:tasks.ceph.osd.0:Stopped 2026-02-21T00:27:50.317 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-21T00:27:50.317 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:27:50.391 INFO:tasks.ceph.osd.1:Stopped 2026-02-21T00:27:50.392 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-21T00:27:50.392 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:27:50.470 INFO:tasks.ceph.osd.2:Stopped 2026-02-21T00:27:50.470 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-21T00:27:50.470 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-21T00:27:50.470 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:27:50.506 INFO:tasks.ceph.mgr.x:Stopped 2026-02-21T00:27:50.506 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-21T00:27:50.506 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-21T00:27:50.506 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:27:50.555 INFO:tasks.ceph.mon.a:Stopped 2026-02-21T00:27:50.555 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-21T00:27:50.556 DEBUG:teuthology.orchestra.run.trial120:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v '\(PG_' | grep -E -v '\(OSD_' | grep -E -v '\(OBJECT_' | grep -E -v 'overall HEALTH' | head -n 1 2026-02-21T00:27:50.621 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial120.front.sepia.ceph.com 2026-02-21T00:27:50.621 DEBUG:teuthology.orchestra.run.trial120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-21T00:27:50.770 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial120.front.sepia.ceph.com 2026-02-21T00:27:50.770 DEBUG:teuthology.orchestra.run.trial120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-21T00:27:50.799 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial120.front.sepia.ceph.com 2026-02-21T00:27:50.799 DEBUG:teuthology.orchestra.run.trial120:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-21T00:27:50.866 INFO:tasks.ceph:Archiving mon data... 2026-02-21T00:27:50.867 DEBUG:teuthology.misc:Transferring archived files from trial120:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61964/data/mon.a.tgz 2026-02-21T00:27:50.867 DEBUG:teuthology.orchestra.run.trial120:> mktemp 2026-02-21T00:27:50.922 INFO:teuthology.orchestra.run.trial120.stdout:/tmp/tmp.bSN1qOHeDr 2026-02-21T00:27:50.923 DEBUG:teuthology.orchestra.run.trial120:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.bSN1qOHeDr 2026-02-21T00:27:51.011 DEBUG:teuthology.orchestra.run.trial120:> sudo chmod 0666 /tmp/tmp.bSN1qOHeDr 2026-02-21T00:27:51.090 DEBUG:teuthology.orchestra.remote:trial120:/tmp/tmp.bSN1qOHeDr is 199KB 2026-02-21T00:27:51.109 DEBUG:teuthology.orchestra.run.trial120:> rm -fr /tmp/tmp.bSN1qOHeDr 2026-02-21T00:27:51.124 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-21T00:27:51.124 DEBUG:teuthology.orchestra.run.trial120:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-21T00:27:51.246 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-21T00:27:51.246 INFO:tasks.ceph:Archiving crash dumps... 2026-02-21T00:27:51.248 DEBUG:teuthology.misc:Transferring archived files from trial120:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61964/remote/trial120/crash 2026-02-21T00:27:51.250 DEBUG:teuthology.orchestra.run.trial120:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-21T00:27:51.308 INFO:tasks.ceph:Compressing logs... 2026-02-21T00:27:51.308 DEBUG:teuthology.orchestra.run.trial120:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T00:27:51.331 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-21T00:27:51.331 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43183.log 2026-02-21T00:27:51.331 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39995.log 2026-02-21T00:27:51.332 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.43183.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-21T00:27:51.342 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.39995.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43183.log.gz 2026-02-21T00:27:51.342 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44570.log 2026-02-21T00:27:51.342 INFO:teuthology.orchestra.run.trial120.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39995.log.gz 2026-02-21T00:27:51.342 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-21T00:27:51.342 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40147.log 2026-02-21T00:27:51.342 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.44570.log: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.44570.log.gz 2026-02-21T00:27:51.342 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.37606.log 2026-02-21T00:27:51.342 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38994.log 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.40147.log: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph.tmp-client.admin.37606.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40147.log.gz 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39736.log 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr: 68.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.37606.log.gz 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43336.log 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.38994.log: /var/log/ceph/ceph-client.admin.39736.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38994.log.gz 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40485.log 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39736.log.gz 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.43336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39225.log 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.40485.log: 50.8% -- replaced with /var/log/ceph/ceph-client.admin.43336.log.gz 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40387.log 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40485.log.gz 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.39225.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43893.log 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.40387.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39225.log.gz 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39831.log 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40387.log.gz 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40293.log/var/log/ceph/ceph-client.admin.43893.log: 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.39831.log: 66.9%gzip -- replaced with /var/log/ceph/ceph-client.admin.43893.log.gz -5 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr: --verbose -- /var/log/ceph/ceph-client.admin.39946.log 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr: 66.9% -- replaced with /var/log/ceph/ceph-client.admin.39831.log.gz 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.40293.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45249.log 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40293.log.gz 2026-02-21T00:27:51.343 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39897.log 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.39946.log: /var/log/ceph/ceph-client.admin.45249.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39946.log.gz 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38898.log 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.45249.log.gz 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.39897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43241.log 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39897.log.gz 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40294.log 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.38898.log: /var/log/ceph/ceph-client.admin.43241.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40436.log 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38898.log.gz 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43241.log.gz 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.40294.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39491.log 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40294.log.gz 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.40436.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43127.log 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40436.log.gz 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.39491.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40145.log 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.43127.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr: 50.8% -- replaced with /var/log/ceph/ceph-client.admin.39491.log.gz 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.43127.log.gz 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.40145.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40292.log 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40145.log.gz 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43882.log 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.40292.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40044.log 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr: 87.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40292.log.gz 2026-02-21T00:27:51.344 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.43882.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr: 66.9% -- replaced with /var/log/ceph/ceph-client.admin.43882.log.gz 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.40044.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.43281.log 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40044.log.gz 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39806.log 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.43281.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39600.log 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.43281.log.gz 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.39806.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39272.log 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39806.log.gz 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.39600.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39600.log.gz 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose/var/log/ceph/ceph-client.admin.39272.log: -- /var/log/ceph/ceph-client.admin.40146.log 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-mgr.x.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39272.log.gz 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44572.log 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.40146.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39041.log 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.44572.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40146.log.gz 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr: 66.4% -- replaced with /var/log/ceph/ceph-client.admin.44572.log.gz 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.39041.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39668.log 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39178.log 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39041.log.gz 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr:/var/log/ceph/ceph-client.admin.39668.log: /var/log/ceph/ceph-client.admin.39178.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39668.log.gz 2026-02-21T00:27:51.345 INFO:teuthology.orchestra.run.trial120.stderr: 85.0% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-21T00:27:51.346 INFO:teuthology.orchestra.run.trial120.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39178.log.gz 2026-02-21T00:27:51.348 INFO:teuthology.orchestra.run.trial120.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-21T00:27:53.171 INFO:teuthology.orchestra.run.trial120.stderr: 92.1% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-21T00:27:53.250 INFO:teuthology.orchestra.run.trial120.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-21T00:27:53.365 INFO:teuthology.orchestra.run.trial120.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-21T00:27:53.366 INFO:teuthology.orchestra.run.trial120.stderr: 2026-02-21T00:27:53.366 INFO:teuthology.orchestra.run.trial120.stderr:real 0m2.044s 2026-02-21T00:27:53.367 INFO:teuthology.orchestra.run.trial120.stderr:user 0m5.399s 2026-02-21T00:27:53.367 INFO:teuthology.orchestra.run.trial120.stderr:sys 0m0.413s 2026-02-21T00:27:53.367 INFO:tasks.ceph:Archiving logs... 2026-02-21T00:27:53.367 DEBUG:teuthology.misc:Transferring archived files from trial120:/var/log/ceph to /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61964/remote/trial120/log 2026-02-21T00:27:53.369 DEBUG:teuthology.orchestra.run.trial120:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-21T00:27:53.982 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-21T00:27:54.053 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-21T00:27:54.064 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-21T00:27:54.064 DEBUG:teuthology.orchestra.run.trial120:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-21T00:27:54.092 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-21T00:27:54.103 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-21T00:27:54.103 DEBUG:teuthology.orchestra.run.trial120:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T00:27:54.150 INFO:teuthology.orchestra.run.trial120.stderr:bash: line 1: ntpq: command not found 2026-02-21T00:27:54.185 INFO:teuthology.orchestra.run.trial120.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-21T00:27:54.185 INFO:teuthology.orchestra.run.trial120.stdout:=============================================================================== 2026-02-21T00:27:54.185 INFO:teuthology.orchestra.run.trial120.stdout:^- acheron.bitsrc.net 3 6 377 5 +388us[ +402us] +/- 75ms 2026-02-21T00:27:54.185 INFO:teuthology.orchestra.run.trial120.stdout:^+ ntp.netlinkify.com 2 6 377 3 -190us[ -190us] +/- 7301us 2026-02-21T00:27:54.185 INFO:teuthology.orchestra.run.trial120.stdout:^* 159.203.158.197 2 6 377 5 +255us[ +270us] +/- 8184us 2026-02-21T00:27:54.185 INFO:teuthology.orchestra.run.trial120.stdout:^- triton.ellipse.net 2 6 377 3 -600us[ -600us] +/- 31ms 2026-02-21T00:27:54.187 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-21T00:27:54.203 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-21T00:27:54.203 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-21T00:27:54.217 DEBUG:teuthology.orchestra.run.trial120:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-21T00:27:54.288 DEBUG:teuthology.orchestra.run.trial120:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-21T00:27:54.314 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T00:27:54.314 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-21T00:27:54.327 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-21T00:27:54.343 INFO:teuthology.task.internal:Duration was 946.668628 seconds 2026-02-21T00:27:54.344 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-21T00:27:54.362 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-21T00:27:54.362 DEBUG:teuthology.orchestra.run.trial120:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-21T00:27:54.403 INFO:teuthology.orchestra.run.trial120.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-21T00:27:54.667 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-21T00:27:54.667 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial120.front.sepia.ceph.com 2026-02-21T00:27:54.668 DEBUG:teuthology.orchestra.run.trial120:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-21T00:27:54.726 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-21T00:27:54.726 DEBUG:teuthology.orchestra.run.trial120:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T00:27:54.834 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-21T00:27:54.834 DEBUG:teuthology.orchestra.run.trial120:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T00:27:54.857 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T00:27:54.857 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T00:27:54.857 INFO:teuthology.orchestra.run.trial120.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-21T00:27:54.857 INFO:teuthology.orchestra.run.trial120.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T00:27:54.858 INFO:teuthology.orchestra.run.trial120.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-21T00:27:54.868 INFO:teuthology.orchestra.run.trial120.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.8% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-21T00:27:54.870 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-21T00:27:54.880 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-21T00:27:54.881 DEBUG:teuthology.orchestra.run.trial120:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-21T00:27:54.935 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-21T00:27:54.950 DEBUG:teuthology.orchestra.run.trial120:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-21T00:27:55.002 INFO:teuthology.orchestra.run.trial120.stdout:kernel.core_pattern = core 2026-02-21T00:27:55.019 DEBUG:teuthology.orchestra.run.trial120:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-21T00:27:55.074 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T00:27:55.074 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-21T00:27:55.091 INFO:teuthology.task.internal:Transferring archived files... 2026-02-21T00:27:55.092 DEBUG:teuthology.misc:Transferring archived files from trial120:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61964/remote/trial120 2026-02-21T00:27:55.092 DEBUG:teuthology.orchestra.run.trial120:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-21T00:27:55.185 INFO:teuthology.task.internal:Removing archive directory... 2026-02-21T00:27:55.185 DEBUG:teuthology.orchestra.run.trial120:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-21T00:27:55.212 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-21T00:27:55.227 INFO:teuthology.task.internal:Not uploading archives. 2026-02-21T00:27:55.227 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-21T00:27:55.243 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-21T00:27:55.244 DEBUG:teuthology.orchestra.run.trial120:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-21T00:27:55.270 INFO:teuthology.orchestra.run.trial120.stdout: 917576 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 21 00:27 /home/ubuntu/cephtest 2026-02-21T00:27:55.270 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-21T00:27:55.282 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-21T00:27:55.311 INFO:teuthology.run:Summary data: description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} perf_ignorelist settings/optimized workloads/radosbench_4M_seq_read} duration: 946.6686282157898 flavor: debug owner: scheduled_liucm@soko04.front.sepia.ceph.com success: true 2026-02-21T00:27:55.312 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T00:27:55.406 INFO:teuthology.run:pass