2026-02-20T23:40:15.007 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-20T23:40:15.023 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-20T23:40:15.104 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61901 branch: wip-liucm-osd-shard-redesign-crimson-only description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} perf_ignorelist settings/optimized workloads/fio_4M_rand_write} email: null first_in_suite: false flavor: debug job_id: '61901' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-liucm-osd-shard-redesign-crimson-only ceph: conf: global: auth client required: none auth cluster required: none auth service required: none auth supported: none debug asok: 0/0 debug auth: 0/0 debug bdev: 0/0 debug bluefs: 0/0 debug bluestore: 0/0 debug buffer: 0/0 debug civetweb: 0/0 debug client: 0/0 debug compressor: 0/0 debug context: 0/0 debug crush: 0/0 debug crypto: 0/0 debug dpdk: 0/0 debug eventtrace: 0/0 debug filer: 0/0 debug filestore: 0/0 debug finisher: 0/0 debug fuse: 0/0 debug heartbeatmap: 0/0 debug javaclient: 0/0 debug journal: 0/0 debug journaler: 0/0 debug kstore: 0/0 debug leveldb: 0/0 debug lockdep: 0/0 debug mds: 0/0 debug mds balancer: 0/0 debug mds locker: 0/0 debug mds log: 0/0 debug mds log expire: 0/0 debug mds migrator: 0/0 debug memdb: 0/0 debug mgr: 0/0 debug mgrc: 0/0 debug mon: 0/0 debug monc: 0/0 debug ms: 0/0 debug objclass: 0/0 debug objectcacher: 0/0 debug objecter: 0/0 debug optracker: 0/0 debug osd: 0/0 debug paxos: 0/0 debug perfcounter: 0/0 debug rados: 0/0 debug rbd: 0/0 debug rbd mirror: 0/0 debug rbd replay: 0/0 debug refs: 0/0 debug rgw: 0/0 debug rgw lifecycle: 0/0 debug rgw notification: 0/0 debug rgw sync: 0/0 debug rocksdb: 0/0 debug striper: 0/0 debug throttle: 0/0 debug timer: 0/0 debug tp: 0/0 enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 0/0 debug ms: 0/0 debug paxos: 0/0 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug filestore: 0/0 debug journal: 0/0 debug ms: 0/0 debug osd: 0/0 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - \(PG_ - \(OSD_ - \(OBJECT_ - overall HEALTH sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: flavor: debug sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f workunit: branch: wip-liucm-osd-shard-redesign-crimson-only env: CRIMSON_COMPAT: '1' sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f owner: scheduled_liucm@soko04.front.sepia.ceph.com priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 6283 sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f sleep_before_teardown: 0 suite: crimson-rados suite_branch: wip-liucm-osd-shard-redesign-crimson-only suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f targets: trial085.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3p1vOMUy5kD/Fxh6GyJJgm54KsYm8wNNX282hwUf+3YqpBmyRhF00BiijjZT4XZl+6ZAd+Ztb5IkZDIlMpjjQ= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: osd: debug monc: 20 wait-for-scrub: false - ssh_keys: null - cbt: benchmarks: librbdfio: iodepth: - 32 log_avg_msec: 100 mode: - randwrite norandommap: true op_size: - 4194304 osd_ra: - 4096 pool_profile: rbd procs_per_volume: - 1 time: 60 vol_size: 4096 volumes_per_client: - 2 cluster: iterations: 1 osds_per_node: 3 pool_profiles: rbd: pg_size: 128 pgp_size: 128 replication: 3 user: ubuntu monitoring_profiles: perf: args: stat -p {pid} -o {perf_dir}/perf_stat.{pid} nodes: - osds perf_cmd: perf pid_glob: crimson-osd.*.pid teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-20_22:16:48 tube: trial user: liucm verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-20T23:40:15.104 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/qa; will attempt to use it 2026-02-20T23:40:15.105 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/qa/tasks 2026-02-20T23:40:15.105 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-20T23:40:15.107 INFO:teuthology.task.internal:Checking packages... 2026-02-20T23:40:15.107 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f' 2026-02-20T23:40:15.107 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-20T23:40:15.107 INFO:teuthology.packaging:ref: None 2026-02-20T23:40:15.107 INFO:teuthology.packaging:tag: None 2026-02-20T23:40:15.107 INFO:teuthology.packaging:branch: wip-liucm-osd-shard-redesign-crimson-only 2026-02-20T23:40:15.107 INFO:teuthology.packaging:sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-20T23:40:15.107 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=wip-liucm-osd-shard-redesign-crimson-only 2026-02-20T23:40:15.237 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5343.gbcdd429b 2026-02-20T23:40:15.238 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-20T23:40:15.251 INFO:teuthology.task.internal:no buildpackages task found 2026-02-20T23:40:15.251 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-20T23:40:15.266 INFO:teuthology.task.internal:Saving configuration 2026-02-20T23:40:15.280 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-20T23:40:15.292 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-20T23:40:15.367 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial085.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61901', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-20 23:36:52.232690', 'locked_by': 'scheduled_liucm@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3p1vOMUy5kD/Fxh6GyJJgm54KsYm8wNNX282hwUf+3YqpBmyRhF00BiijjZT4XZl+6ZAd+Ztb5IkZDIlMpjjQ='} 2026-02-20T23:40:15.367 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-20T23:40:15.378 INFO:teuthology.task.internal:roles: ubuntu@trial085.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-20T23:40:15.378 INFO:teuthology.run_tasks:Running task console_log... 2026-02-20T23:40:15.461 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f4e49e332e0>, signals=[15]) 2026-02-20T23:40:15.461 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-20T23:40:15.474 INFO:teuthology.task.internal:Opening connections... 2026-02-20T23:40:15.475 DEBUG:teuthology.task.internal:connecting to ubuntu@trial085.front.sepia.ceph.com 2026-02-20T23:40:15.475 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T23:40:15.538 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-20T23:40:15.552 DEBUG:teuthology.orchestra.run.trial085:> uname -m 2026-02-20T23:40:15.610 INFO:teuthology.orchestra.run.trial085.stdout:x86_64 2026-02-20T23:40:15.611 DEBUG:teuthology.orchestra.run.trial085:> cat /etc/os-release 2026-02-20T23:40:15.667 INFO:teuthology.orchestra.run.trial085.stdout:NAME="CentOS Stream" 2026-02-20T23:40:15.667 INFO:teuthology.orchestra.run.trial085.stdout:VERSION="9" 2026-02-20T23:40:15.667 INFO:teuthology.orchestra.run.trial085.stdout:ID="centos" 2026-02-20T23:40:15.667 INFO:teuthology.orchestra.run.trial085.stdout:ID_LIKE="rhel fedora" 2026-02-20T23:40:15.667 INFO:teuthology.orchestra.run.trial085.stdout:VERSION_ID="9" 2026-02-20T23:40:15.667 INFO:teuthology.orchestra.run.trial085.stdout:PLATFORM_ID="platform:el9" 2026-02-20T23:40:15.668 INFO:teuthology.orchestra.run.trial085.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-20T23:40:15.668 INFO:teuthology.orchestra.run.trial085.stdout:ANSI_COLOR="0;31" 2026-02-20T23:40:15.668 INFO:teuthology.orchestra.run.trial085.stdout:LOGO="fedora-logo-icon" 2026-02-20T23:40:15.668 INFO:teuthology.orchestra.run.trial085.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-20T23:40:15.668 INFO:teuthology.orchestra.run.trial085.stdout:HOME_URL="https://centos.org/" 2026-02-20T23:40:15.668 INFO:teuthology.orchestra.run.trial085.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-20T23:40:15.668 INFO:teuthology.orchestra.run.trial085.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-20T23:40:15.668 INFO:teuthology.orchestra.run.trial085.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-20T23:40:15.669 INFO:teuthology.lock.ops:Updating trial085.front.sepia.ceph.com on lock server 2026-02-20T23:40:15.836 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-20T23:40:15.849 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-20T23:40:15.860 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-20T23:40:15.861 DEBUG:teuthology.orchestra.run.trial085:> test '!' -e /home/ubuntu/cephtest 2026-02-20T23:40:15.876 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-20T23:40:15.890 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-20T23:40:15.890 DEBUG:teuthology.orchestra.run.trial085:> test -z $(ls -A /var/lib/ceph) 2026-02-20T23:40:15.932 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-20T23:40:16.019 INFO:teuthology.run_tasks:Running task kernel... 2026-02-20T23:40:16.039 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-20T23:40:16.039 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-20T23:40:16.039 DEBUG:teuthology.orchestra.run.trial085:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-20T23:40:16.054 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T23:40:16.054 DEBUG:teuthology.orchestra.run.trial085:> uname -r 2026-02-20T23:40:16.112 INFO:teuthology.orchestra.run.trial085.stdout:5.14.0-681.el9.x86_64 2026-02-20T23:40:16.112 INFO:teuthology.task.kernel:Running kernel on trial085: 5.14.0-681.el9.x86_64 2026-02-20T23:40:16.112 DEBUG:teuthology.orchestra.run.trial085:> sudo yum install -y kernel 2026-02-20T23:40:17.673 INFO:teuthology.orchestra.run.trial085.stdout:CentOS Stream 9 - BaseOS 7.6 MB/s | 8.9 MB 00:01 2026-02-20T23:40:20.353 INFO:teuthology.orchestra.run.trial085.stdout:CentOS Stream 9 - AppStream 13 MB/s | 27 MB 00:02 2026-02-20T23:40:31.142 INFO:teuthology.orchestra.run.trial085.stdout:CentOS Stream 9 - CRB 944 kB/s | 7.8 MB 00:08 2026-02-20T23:40:33.367 INFO:teuthology.orchestra.run.trial085.stdout:CentOS Stream 9 - Extras packages 13 kB/s | 20 kB 00:01 2026-02-20T23:40:34.132 INFO:teuthology.orchestra.run.trial085.stdout:Extra Packages for Enterprise Linux 29 MB/s | 20 MB 00:00 2026-02-20T23:40:37.247 INFO:teuthology.orchestra.run.trial085.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-20T23:40:38.256 INFO:teuthology.orchestra.run.trial085.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-20T23:40:38.256 INFO:teuthology.orchestra.run.trial085.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-20T23:40:38.276 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:40:38.276 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-20T23:40:38.277 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:40:38.299 DEBUG:teuthology.orchestra.run.trial085:> echo no | sudo yum reinstall kernel || true 2026-02-20T23:40:38.599 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:01 ago on Fri 20 Feb 2026 11:40:37 PM UTC. 2026-02-20T23:40:38.678 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:40:38.679 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:40:38.679 INFO:teuthology.orchestra.run.trial085.stdout: Package Architecture Version Repository Size 2026-02-20T23:40:38.679 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:40:38.679 INFO:teuthology.orchestra.run.trial085.stdout:Reinstalling: 2026-02-20T23:40:38.679 INFO:teuthology.orchestra.run.trial085.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T23:40:38.679 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:40:38.679 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:40:38.679 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:40:38.679 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:40:38.679 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 803 k 2026-02-20T23:40:38.679 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 0 2026-02-20T23:40:38.679 INFO:teuthology.orchestra.run.trial085.stderr:Operation aborted. 2026-02-20T23:40:38.699 INFO:teuthology.orchestra.run.trial085.stdout:Is this ok [y/N]: 2026-02-20T23:40:38.699 DEBUG:teuthology.orchestra.run.trial085:> sudo yum reinstall -y kernel || true 2026-02-20T23:40:38.997 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:01 ago on Fri 20 Feb 2026 11:40:37 PM UTC. 2026-02-20T23:40:39.077 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:40:39.077 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:40:39.077 INFO:teuthology.orchestra.run.trial085.stdout: Package Architecture Version Repository Size 2026-02-20T23:40:39.077 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:40:39.077 INFO:teuthology.orchestra.run.trial085.stdout:Reinstalling: 2026-02-20T23:40:39.077 INFO:teuthology.orchestra.run.trial085.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T23:40:39.077 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:40:39.077 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:40:39.077 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:40:39.077 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:40:39.077 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 803 k 2026-02-20T23:40:39.077 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 0 2026-02-20T23:40:39.077 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:40:39.765 INFO:teuthology.orchestra.run.trial085.stdout:kernel-5.14.0-681.el9.x86_64.rpm 1.3 MB/s | 803 kB 00:00 2026-02-20T23:40:39.765 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:40:39.765 INFO:teuthology.orchestra.run.trial085.stdout:Total 1.1 MB/s | 803 kB 00:00 2026-02-20T23:40:39.776 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:40:39.809 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:40:39.809 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:40:39.813 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:40:39.813 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:40:39.837 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:40:39.841 INFO:teuthology.orchestra.run.trial085.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T23:40:39.842 INFO:teuthology.orchestra.run.trial085.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T23:40:39.950 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T23:40:39.950 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T23:40:40.004 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T23:40:40.004 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:40:40.004 INFO:teuthology.orchestra.run.trial085.stdout:Reinstalled: 2026-02-20T23:40:40.004 INFO:teuthology.orchestra.run.trial085.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-20T23:40:40.004 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:40:40.005 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:40:40.030 DEBUG:teuthology.orchestra.run.trial085:> rpm -q kernel | sort -rV | head -n 1 2026-02-20T23:40:40.095 INFO:teuthology.orchestra.run.trial085.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-20T23:40:40.095 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-20T23:40:40.095 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-20T23:40:40.096 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-20T23:40:40.096 DEBUG:teuthology.orchestra.run.trial085:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-20T23:40:40.159 INFO:teuthology.orchestra.run.trial085.stdout:ttyS1 2026-02-20T23:40:40.181 DEBUG:teuthology.parallel:result is None 2026-02-20T23:40:40.181 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-20T23:40:40.197 INFO:teuthology.task.internal:Creating test directory... 2026-02-20T23:40:40.197 DEBUG:teuthology.orchestra.run.trial085:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-20T23:40:40.238 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-20T23:40:40.253 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-20T23:40:40.270 INFO:teuthology.task.internal:Creating archive directory... 2026-02-20T23:40:40.270 DEBUG:teuthology.orchestra.run.trial085:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-20T23:40:40.300 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-20T23:40:40.312 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-20T23:40:40.313 DEBUG:teuthology.orchestra.run.trial085:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-20T23:40:40.359 INFO:teuthology.orchestra.run.trial085.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T23:40:40.367 INFO:teuthology.orchestra.run.trial085.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T23:40:40.369 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-20T23:40:40.382 INFO:teuthology.task.internal:Configuring sudo... 2026-02-20T23:40:40.382 DEBUG:teuthology.orchestra.run.trial085:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-20T23:40:40.435 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-20T23:40:40.458 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-20T23:40:40.458 DEBUG:teuthology.orchestra.run.trial085:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-20T23:40:40.491 DEBUG:teuthology.orchestra.run.trial085:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T23:40:40.636 DEBUG:teuthology.orchestra.run.trial085:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T23:40:40.659 DEBUG:teuthology.orchestra.run.trial085:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T23:40:40.778 DEBUG:teuthology.orchestra.run.trial085:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T23:40:40.801 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-20T23:40:40.801 DEBUG:teuthology.orchestra.run.trial085:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-20T23:40:40.864 DEBUG:teuthology.orchestra.run.trial085:> sudo service rsyslog restart 2026-02-20T23:40:40.932 INFO:teuthology.orchestra.run.trial085.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-20T23:40:41.368 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-20T23:40:41.379 INFO:teuthology.task.internal:Starting timer... 2026-02-20T23:40:41.379 INFO:teuthology.run_tasks:Running task pcp... 2026-02-20T23:40:41.401 INFO:teuthology.run_tasks:Running task selinux... 2026-02-20T23:40:41.419 DEBUG:teuthology.orchestra.run.trial085:> sudo service auditd rotate 2026-02-20T23:40:41.448 INFO:teuthology.orchestra.run.trial085.stdout:Rotating logs: 2026-02-20T23:40:41.450 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-20T23:40:41.450 DEBUG:teuthology.orchestra.run.trial085:> /usr/sbin/getenforce 2026-02-20T23:40:41.506 INFO:teuthology.orchestra.run.trial085.stdout:Permissive 2026-02-20T23:40:41.507 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial085.front.sepia.ceph.com': 'permissive'} 2026-02-20T23:40:41.507 DEBUG:teuthology.orchestra.run.trial085:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-20T23:40:41.571 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T23:40:41.571 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-20T23:40:41.571 DEBUG:teuthology.orchestra.run.trial085:> sudo /usr/sbin/setenforce permissive 2026-02-20T23:40:41.634 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-20T23:40:41.650 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-20T23:40:41.663 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-20T23:40:41.665 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial085.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-20T23:43:19.643 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial085.front.sepia.ceph.com')] 2026-02-20T23:43:19.644 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial085.front.sepia.ceph.com' 2026-02-20T23:43:19.647 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial085.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T23:43:19.753 DEBUG:teuthology.orchestra.run.trial085:> true 2026-02-20T23:43:19.830 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial085.front.sepia.ceph.com' 2026-02-20T23:43:19.830 INFO:teuthology.run_tasks:Running task clock... 2026-02-20T23:43:19.846 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-20T23:43:19.846 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-20T23:43:19.847 DEBUG:teuthology.orchestra.run.trial085:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-20T23:43:19.899 INFO:teuthology.orchestra.run.trial085.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-20T23:43:19.911 INFO:teuthology.orchestra.run.trial085.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-20T23:43:19.932 INFO:teuthology.orchestra.run.trial085.stderr:sudo: ntpd: command not found 2026-02-20T23:43:19.942 INFO:teuthology.orchestra.run.trial085.stdout:506 Cannot talk to daemon 2026-02-20T23:43:19.953 INFO:teuthology.orchestra.run.trial085.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-20T23:43:19.964 INFO:teuthology.orchestra.run.trial085.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-20T23:43:20.011 INFO:teuthology.orchestra.run.trial085.stderr:bash: line 1: ntpq: command not found 2026-02-20T23:43:20.012 INFO:teuthology.orchestra.run.trial085.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-20T23:43:20.012 INFO:teuthology.orchestra.run.trial085.stdout:=============================================================================== 2026-02-20T23:43:20.013 INFO:teuthology.run_tasks:Running task install... 2026-02-20T23:43:20.024 DEBUG:teuthology.task.install:project ceph 2026-02-20T23:43:20.024 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}} 2026-02-20T23:43:20.025 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'} 2026-02-20T23:43:20.025 INFO:teuthology.task.install:Using flavor: debug 2026-02-20T23:43:20.030 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-20T23:43:20.030 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-20T23:43:20.031 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'tag': None, 'wait_for_package': False} 2026-02-20T23:43:20.031 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-20T23:43:20.190 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/ 2026-02-20T23:43:20.190 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5343.gbcdd429b 2026-02-20T23:43:20.582 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-20T23:43:20.583 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-20T23:43:20.583 DEBUG:teuthology.orchestra.run.trial085:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-20T23:43:20.611 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-20T23:43:20.612 DEBUG:teuthology.orchestra.run.trial085:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-20T23:43:20.677 DEBUG:teuthology.orchestra.run.trial085:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-20T23:43:20.752 DEBUG:teuthology.orchestra.run.trial085:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-20T23:43:20.814 INFO:teuthology.orchestra.run.trial085.stdout:check_obsoletes = 1 2026-02-20T23:43:20.816 DEBUG:teuthology.orchestra.run.trial085:> sudo yum clean all 2026-02-20T23:43:20.973 INFO:teuthology.orchestra.run.trial085.stdout:41 files removed 2026-02-20T23:43:20.987 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-radosgw 2026-02-20T23:43:21.847 INFO:teuthology.orchestra.run.trial085.stdout:ceph packages for x86_64 128 kB/s | 92 kB 00:00 2026-02-20T23:43:22.580 INFO:teuthology.orchestra.run.trial085.stdout:ceph noarch packages 33 kB/s | 24 kB 00:00 2026-02-20T23:43:23.165 INFO:teuthology.orchestra.run.trial085.stdout:ceph source packages 3.7 kB/s | 2.1 kB 00:00 2026-02-20T23:43:24.031 INFO:teuthology.orchestra.run.trial085.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-20T23:43:51.138 INFO:teuthology.orchestra.run.trial085.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 27 MB 00:26 2026-02-20T23:43:54.385 INFO:teuthology.orchestra.run.trial085.stdout:CentOS Stream 9 - CRB 8.7 MB/s | 7.8 MB 00:00 2026-02-20T23:43:56.099 INFO:teuthology.orchestra.run.trial085.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 20 kB 00:00 2026-02-20T23:43:57.530 INFO:teuthology.orchestra.run.trial085.stdout:Extra Packages for Enterprise Linux 15 MB/s | 20 MB 00:01 2026-02-20T23:44:00.675 INFO:teuthology.orchestra.run.trial085.stdout:lab-extras 2.0 MB/s | 50 kB 00:00 2026-02-20T23:44:01.861 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:44:01.862 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:44:01.862 INFO:teuthology.orchestra.run.trial085.stdout: Package Arch Version Repository Size 2026-02-20T23:44:01.862 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: ceph-radosgw x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 11 M 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout:Upgrading: 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: librados2 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 3.3 M 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: librbd1 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 2.7 M 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout:Installing dependencies: 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: ceph-base x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 5.1 M 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: ceph-common x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 21 M 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: ceph-selinux x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 25 k 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: libcephfs2 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 860 k 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: libradosstriper1 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 263 k 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: librgw2 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 5.6 M 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-20T23:44:01.863 INFO:teuthology.orchestra.run.trial085.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout: python3-ceph-argparse x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 45 k 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout: python3-ceph-common x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 185 k 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout: python3-cephfs x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 203 k 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout: python3-rados x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 324 k 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout: python3-rbd x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 305 k 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout: python3-rgw x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 100 k 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout:Installing weak dependencies: 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout:Install 32 Packages 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout:Upgrade 2 Packages 2026-02-20T23:44:01.864 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:44:01.865 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 60 M 2026-02-20T23:44:01.865 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:44:13.785 INFO:teuthology.orchestra.run.trial085.stdout:(1/34): ceph-base-20.3.0-5343.gbcdd429b.el9.x86 448 kB/s | 5.1 MB 00:11 2026-02-20T23:44:13.854 INFO:teuthology.orchestra.run.trial085.stdout:(2/34): ceph-selinux-20.3.0-5343.gbcdd429b.el9. 365 kB/s | 25 kB 00:00 2026-02-20T23:44:15.620 INFO:teuthology.orchestra.run.trial085.stdout:(3/34): libcephfs2-20.3.0-5343.gbcdd429b.el9.x8 487 kB/s | 860 kB 00:01 2026-02-20T23:44:16.236 INFO:teuthology.orchestra.run.trial085.stdout:(4/34): libradosstriper1-20.3.0-5343.gbcdd429b. 426 kB/s | 263 kB 00:00 2026-02-20T23:44:16.711 INFO:teuthology.orchestra.run.trial085.stdout:(5/34): ceph-radosgw-20.3.0-5343.gbcdd429b.el9. 783 kB/s | 11 MB 00:14 2026-02-20T23:44:16.783 INFO:teuthology.orchestra.run.trial085.stdout:(6/34): python3-ceph-argparse-20.3.0-5343.gbcdd 630 kB/s | 45 kB 00:00 2026-02-20T23:44:17.057 INFO:teuthology.orchestra.run.trial085.stdout:(7/34): python3-ceph-common-20.3.0-5343.gbcdd42 675 kB/s | 185 kB 00:00 2026-02-20T23:44:17.405 INFO:teuthology.orchestra.run.trial085.stdout:(8/34): python3-cephfs-20.3.0-5343.gbcdd429b.el 585 kB/s | 203 kB 00:00 2026-02-20T23:44:17.888 INFO:teuthology.orchestra.run.trial085.stdout:(9/34): python3-rados-20.3.0-5343.gbcdd429b.el9 672 kB/s | 324 kB 00:00 2026-02-20T23:44:18.303 INFO:teuthology.orchestra.run.trial085.stdout:(10/34): python3-rbd-20.3.0-5343.gbcdd429b.el9. 734 kB/s | 305 kB 00:00 2026-02-20T23:44:18.442 INFO:teuthology.orchestra.run.trial085.stdout:(11/34): python3-rgw-20.3.0-5343.gbcdd429b.el9. 726 kB/s | 100 kB 00:00 2026-02-20T23:44:18.925 INFO:teuthology.orchestra.run.trial085.stdout:[MIRROR] mailcap-2.1.49-5.el9.noarch.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/mailcap-2.1.49-5.el9.noarch.rpm (IP: 140.211.166.134) 2026-02-20T23:44:19.561 INFO:teuthology.orchestra.run.trial085.stdout:[MIRROR] mailcap-2.1.49-5.el9.noarch.rpm: Status code: 503 for https://ftp2.osuosl.org/pub/centos-stream/9-stream/BaseOS/x86_64/os/Packages/mailcap-2.1.49-5.el9.noarch.rpm (IP: 140.211.166.134) 2026-02-20T23:44:20.135 INFO:teuthology.orchestra.run.trial085.stdout:(12/34): mailcap-2.1.49-5.el9.noarch.rpm 20 kB/s | 33 kB 00:01 2026-02-20T23:44:20.771 INFO:teuthology.orchestra.run.trial085.stdout:(13/34): pciutils-3.7.0-7.el9.x86_64.rpm 146 kB/s | 93 kB 00:00 2026-02-20T23:44:20.826 INFO:teuthology.orchestra.run.trial085.stdout:(14/34): boost-program-options-1.75.0-13.el9.x8 1.9 MB/s | 104 kB 00:00 2026-02-20T23:44:20.844 INFO:teuthology.orchestra.run.trial085.stdout:(15/34): libnbd-1.20.3-4.el9.x86_64.rpm 8.7 MB/s | 164 kB 00:00 2026-02-20T23:44:20.856 INFO:teuthology.orchestra.run.trial085.stdout:(16/34): libpmemobj-1.12.1-1.el9.x86_64.rpm 13 MB/s | 160 kB 00:00 2026-02-20T23:44:20.866 INFO:teuthology.orchestra.run.trial085.stdout:(17/34): librabbitmq-0.11.0-7.el9.x86_64.rpm 4.7 MB/s | 45 kB 00:00 2026-02-20T23:44:20.890 INFO:teuthology.orchestra.run.trial085.stdout:(18/34): librdkafka-1.6.1-102.el9.x86_64.rpm 27 MB/s | 662 kB 00:00 2026-02-20T23:44:20.901 INFO:teuthology.orchestra.run.trial085.stdout:(19/34): lttng-ust-2.12.0-6.el9.x86_64.rpm 25 MB/s | 292 kB 00:00 2026-02-20T23:44:20.912 INFO:teuthology.orchestra.run.trial085.stdout:(20/34): qatlib-25.08.0-2.el9.x86_64.rpm 21 MB/s | 240 kB 00:00 2026-02-20T23:44:20.922 INFO:teuthology.orchestra.run.trial085.stdout:(21/34): qatlib-service-25.08.0-2.el9.x86_64.rp 3.9 MB/s | 37 kB 00:00 2026-02-20T23:44:20.931 INFO:teuthology.orchestra.run.trial085.stdout:(22/34): qatzip-libs-1.3.1-1.el9.x86_64.rpm 6.7 MB/s | 66 kB 00:00 2026-02-20T23:44:21.181 INFO:teuthology.orchestra.run.trial085.stdout:(23/34): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.2 MB/s | 308 kB 00:00 2026-02-20T23:44:21.350 INFO:teuthology.orchestra.run.trial085.stdout:(24/34): libarrow-9.0.0-15.el9.x86_64.rpm 26 MB/s | 4.4 MB 00:00 2026-02-20T23:44:21.392 INFO:teuthology.orchestra.run.trial085.stdout:(25/34): libarrow-doc-9.0.0-15.el9.noarch.rpm 583 kB/s | 25 kB 00:00 2026-02-20T23:44:21.433 INFO:teuthology.orchestra.run.trial085.stdout:(26/34): liboath-2.6.12-1.el9.x86_64.rpm 1.2 MB/s | 49 kB 00:00 2026-02-20T23:44:21.474 INFO:teuthology.orchestra.run.trial085.stdout:(27/34): libunwind-1.6.2-1.el9.x86_64.rpm 1.6 MB/s | 67 kB 00:00 2026-02-20T23:44:21.519 INFO:teuthology.orchestra.run.trial085.stdout:(28/34): parquet-libs-9.0.0-15.el9.x86_64.rpm 18 MB/s | 838 kB 00:00 2026-02-20T23:44:21.562 INFO:teuthology.orchestra.run.trial085.stdout:(29/34): re2-20211101-20.el9.x86_64.rpm 4.4 MB/s | 191 kB 00:00 2026-02-20T23:44:21.608 INFO:teuthology.orchestra.run.trial085.stdout:(30/34): thrift-0.15.0-4.el9.x86_64.rpm 34 MB/s | 1.6 MB 00:00 2026-02-20T23:44:24.970 INFO:teuthology.orchestra.run.trial085.stdout:(31/34): librgw2-20.3.0-5343.gbcdd429b.el9.x86_ 660 kB/s | 5.6 MB 00:08 2026-02-20T23:44:25.689 INFO:teuthology.orchestra.run.trial085.stdout:(32/34): librados2-20.3.0-5343.gbcdd429b.el9.x8 824 kB/s | 3.3 MB 00:04 2026-02-20T23:44:28.832 INFO:teuthology.orchestra.run.trial085.stdout:(33/34): librbd1-20.3.0-5343.gbcdd429b.el9.x86_ 713 kB/s | 2.7 MB 00:03 2026-02-20T23:44:29.275 INFO:teuthology.orchestra.run.trial085.stdout:(34/34): ceph-common-20.3.0-5343.gbcdd429b.el9. 784 kB/s | 21 MB 00:27 2026-02-20T23:44:29.278 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:44:29.278 INFO:teuthology.orchestra.run.trial085.stdout:Total 2.2 MB/s | 60 MB 00:27 2026-02-20T23:44:29.368 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:44:29.385 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:44:29.386 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:44:29.630 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:44:29.631 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:44:29.914 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:44:30.057 INFO:teuthology.orchestra.run.trial085.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/36 2026-02-20T23:44:30.061 INFO:teuthology.orchestra.run.trial085.stdout: Upgrading : librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/36 2026-02-20T23:44:30.090 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/36 2026-02-20T23:44:30.128 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 3/36 2026-02-20T23:44:30.129 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 4/36 2026-02-20T23:44:30.141 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 4/36 2026-02-20T23:44:30.148 INFO:teuthology.orchestra.run.trial085.stdout: Installing : liboath-2.6.12-1.el9.x86_64 5/36 2026-02-20T23:44:30.151 INFO:teuthology.orchestra.run.trial085.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/36 2026-02-20T23:44:30.154 INFO:teuthology.orchestra.run.trial085.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/36 2026-02-20T23:44:30.159 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 8/36 2026-02-20T23:44:30.170 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-ceph-argparse-2:20.3.0-5343.gbcdd429b.el9. 9/36 2026-02-20T23:44:30.183 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 10/36 2026-02-20T23:44:30.184 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libradosstriper1-2:20.3.0-5343.gbcdd429b.el9.x86_6 11/36 2026-02-20T23:44:30.206 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5343.gbcdd429b.el9.x86_6 11/36 2026-02-20T23:44:30.210 INFO:teuthology.orchestra.run.trial085.stdout: Installing : thrift-0.15.0-4.el9.x86_64 12/36 2026-02-20T23:44:30.240 INFO:teuthology.orchestra.run.trial085.stdout: Installing : re2-1:20211101-20.el9.x86_64 13/36 2026-02-20T23:44:30.243 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 14/36 2026-02-20T23:44:30.251 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 15/36 2026-02-20T23:44:30.253 INFO:teuthology.orchestra.run.trial085.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 16/36 2026-02-20T23:44:30.258 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 17/36 2026-02-20T23:44:30.375 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/36 2026-02-20T23:44:30.376 INFO:teuthology.orchestra.run.trial085.stdout: Upgrading : librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 19/36 2026-02-20T23:44:30.403 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 19/36 2026-02-20T23:44:30.406 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 20/36 2026-02-20T23:44:30.412 INFO:teuthology.orchestra.run.trial085.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/36 2026-02-20T23:44:30.665 INFO:teuthology.orchestra.run.trial085.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/36 2026-02-20T23:44:30.666 INFO:teuthology.orchestra.run.trial085.stdout: Installing : librgw2-2:20.3.0-5343.gbcdd429b.el9.x86_64 23/36 2026-02-20T23:44:30.682 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: librgw2-2:20.3.0-5343.gbcdd429b.el9.x86_64 23/36 2026-02-20T23:44:30.686 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 24/36 2026-02-20T23:44:30.691 INFO:teuthology.orchestra.run.trial085.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/36 2026-02-20T23:44:30.693 INFO:teuthology.orchestra.run.trial085.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/36 2026-02-20T23:44:30.694 INFO:teuthology.orchestra.run.trial085.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-20T23:44:30.712 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-20T23:44:30.715 INFO:teuthology.orchestra.run.trial085.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/36 2026-02-20T23:44:30.750 INFO:teuthology.orchestra.run.trial085.stdout: Installing : mailcap-2.1.49-5.el9.noarch 29/36 2026-02-20T23:44:30.752 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-ceph-common-2:20.3.0-5343.gbcdd429b.el9.x8 30/36 2026-02-20T23:44:31.707 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 31/36 2026-02-20T23:44:31.710 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 31/36 2026-02-20T23:44:31.975 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 31/36 2026-02-20T23:44:31.978 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 32/36 2026-02-20T23:44:32.015 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 32/36 2026-02-20T23:44:32.015 INFO:teuthology.orchestra.run.trial085.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-20T23:44:32.015 INFO:teuthology.orchestra.run.trial085.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-20T23:44:32.015 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:44:32.018 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-selinux-2:20.3.0-5343.gbcdd429b.el9.x86_64 33/36 2026-02-20T23:44:36.494 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5343.gbcdd429b.el9.x86_64 33/36 2026-02-20T23:44:36.494 INFO:teuthology.orchestra.run.trial085.stdout:skipping the directory /sys 2026-02-20T23:44:36.495 INFO:teuthology.orchestra.run.trial085.stdout:skipping the directory /proc 2026-02-20T23:44:36.495 INFO:teuthology.orchestra.run.trial085.stdout:skipping the directory /mnt 2026-02-20T23:44:36.495 INFO:teuthology.orchestra.run.trial085.stdout:skipping the directory /var/tmp 2026-02-20T23:44:36.495 INFO:teuthology.orchestra.run.trial085.stdout:skipping the directory /home 2026-02-20T23:44:36.495 INFO:teuthology.orchestra.run.trial085.stdout:skipping the directory /root 2026-02-20T23:44:36.495 INFO:teuthology.orchestra.run.trial085.stdout:skipping the directory /tmp 2026-02-20T23:44:36.495 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:44:36.996 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 34/36 2026-02-20T23:44:37.019 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 34/36 2026-02-20T23:44:37.019 INFO:teuthology.orchestra.run.trial085.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:44:37.019 INFO:teuthology.orchestra.run.trial085.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-20T23:44:37.019 INFO:teuthology.orchestra.run.trial085.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T23:44:37.019 INFO:teuthology.orchestra.run.trial085.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T23:44:37.019 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:44:37.021 INFO:teuthology.orchestra.run.trial085.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-20T23:44:37.033 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-20T23:44:37.033 INFO:teuthology.orchestra.run.trial085.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-20T23:44:37.086 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-20T23:44:37.086 INFO:teuthology.orchestra.run.trial085.stdout:Creating group 'qat' with GID 992. 2026-02-20T23:44:37.086 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:44:37.369 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/36 2026-02-20T23:44:37.369 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/36 2026-02-20T23:44:37.369 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 3/36 2026-02-20T23:44:37.370 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-selinux-2:20.3.0-5343.gbcdd429b.el9.x86_64 4/36 2026-02-20T23:44:37.370 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 5/36 2026-02-20T23:44:37.370 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libradosstriper1-2:20.3.0-5343.gbcdd429b.el9.x86_6 6/36 2026-02-20T23:44:37.370 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : librgw2-2:20.3.0-5343.gbcdd429b.el9.x86_64 7/36 2026-02-20T23:44:37.370 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5343.gbcdd429b.el9. 8/36 2026-02-20T23:44:37.370 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-ceph-common-2:20.3.0-5343.gbcdd429b.el9.x8 9/36 2026-02-20T23:44:37.370 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 10/36 2026-02-20T23:44:37.370 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 11/36 2026-02-20T23:44:37.370 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 12/36 2026-02-20T23:44:37.370 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 13/36 2026-02-20T23:44:37.370 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/36 2026-02-20T23:44:37.370 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 15/36 2026-02-20T23:44:37.371 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 16/36 2026-02-20T23:44:37.371 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 17/36 2026-02-20T23:44:37.371 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 18/36 2026-02-20T23:44:37.371 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/36 2026-02-20T23:44:37.371 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 20/36 2026-02-20T23:44:37.371 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 21/36 2026-02-20T23:44:37.371 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 22/36 2026-02-20T23:44:37.371 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 23/36 2026-02-20T23:44:37.371 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 24/36 2026-02-20T23:44:37.371 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 25/36 2026-02-20T23:44:37.371 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 26/36 2026-02-20T23:44:37.371 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 27/36 2026-02-20T23:44:37.371 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 28/36 2026-02-20T23:44:37.372 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 29/36 2026-02-20T23:44:37.372 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 30/36 2026-02-20T23:44:37.372 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : re2-1:20211101-20.el9.x86_64 31/36 2026-02-20T23:44:37.372 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 32/36 2026-02-20T23:44:37.372 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 33/36 2026-02-20T23:44:37.372 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 34/36 2026-02-20T23:44:37.372 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 35/36 2026-02-20T23:44:37.431 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 36/36 2026-02-20T23:44:37.431 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:44:37.431 INFO:teuthology.orchestra.run.trial085.stdout:Upgraded: 2026-02-20T23:44:37.431 INFO:teuthology.orchestra.run.trial085.stdout: librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:44:37.431 INFO:teuthology.orchestra.run.trial085.stdout: librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:44:37.431 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-20T23:44:37.431 INFO:teuthology.orchestra.run.trial085.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-20T23:44:37.432 INFO:teuthology.orchestra.run.trial085.stdout: ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:44:37.432 INFO:teuthology.orchestra.run.trial085.stdout: ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:44:37.432 INFO:teuthology.orchestra.run.trial085.stdout: ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:44:37.432 INFO:teuthology.orchestra.run.trial085.stdout: ceph-selinux-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:44:37.432 INFO:teuthology.orchestra.run.trial085.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-20T23:44:37.432 INFO:teuthology.orchestra.run.trial085.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-20T23:44:37.432 INFO:teuthology.orchestra.run.trial085.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-20T23:44:37.432 INFO:teuthology.orchestra.run.trial085.stdout: libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:44:37.432 INFO:teuthology.orchestra.run.trial085.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-20T23:44:37.432 INFO:teuthology.orchestra.run.trial085.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-20T23:44:37.432 INFO:teuthology.orchestra.run.trial085.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-20T23:44:37.432 INFO:teuthology.orchestra.run.trial085.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-20T23:44:37.433 INFO:teuthology.orchestra.run.trial085.stdout: libradosstriper1-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:44:37.433 INFO:teuthology.orchestra.run.trial085.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-20T23:44:37.433 INFO:teuthology.orchestra.run.trial085.stdout: librgw2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:44:37.433 INFO:teuthology.orchestra.run.trial085.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-20T23:44:37.433 INFO:teuthology.orchestra.run.trial085.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-20T23:44:37.433 INFO:teuthology.orchestra.run.trial085.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-20T23:44:37.433 INFO:teuthology.orchestra.run.trial085.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-20T23:44:37.433 INFO:teuthology.orchestra.run.trial085.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-20T23:44:37.433 INFO:teuthology.orchestra.run.trial085.stdout: python3-ceph-argparse-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:44:37.433 INFO:teuthology.orchestra.run.trial085.stdout: python3-ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:44:37.433 INFO:teuthology.orchestra.run.trial085.stdout: python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:44:37.433 INFO:teuthology.orchestra.run.trial085.stdout: python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:44:37.434 INFO:teuthology.orchestra.run.trial085.stdout: python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:44:37.434 INFO:teuthology.orchestra.run.trial085.stdout: python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:44:37.434 INFO:teuthology.orchestra.run.trial085.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-20T23:44:37.434 INFO:teuthology.orchestra.run.trial085.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-20T23:44:37.434 INFO:teuthology.orchestra.run.trial085.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-20T23:44:37.434 INFO:teuthology.orchestra.run.trial085.stdout: re2-1:20211101-20.el9.x86_64 2026-02-20T23:44:37.434 INFO:teuthology.orchestra.run.trial085.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-20T23:44:37.434 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:44:37.434 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:44:37.471 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-test 2026-02-20T23:44:37.780 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:37 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:44:37.866 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:44:37.867 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:44:37.868 INFO:teuthology.orchestra.run.trial085.stdout: Package Arch Version Repository Size 2026-02-20T23:44:37.868 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:44:37.868 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:44:37.868 INFO:teuthology.orchestra.run.trial085.stdout: ceph-test x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 126 M 2026-02-20T23:44:37.868 INFO:teuthology.orchestra.run.trial085.stdout:Installing dependencies: 2026-02-20T23:44:37.868 INFO:teuthology.orchestra.run.trial085.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-20T23:44:37.868 INFO:teuthology.orchestra.run.trial085.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-20T23:44:37.868 INFO:teuthology.orchestra.run.trial085.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-20T23:44:37.868 INFO:teuthology.orchestra.run.trial085.stdout: libcephsqlite x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 153 k 2026-02-20T23:44:37.868 INFO:teuthology.orchestra.run.trial085.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-20T23:44:37.868 INFO:teuthology.orchestra.run.trial085.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-20T23:44:37.869 INFO:teuthology.orchestra.run.trial085.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-20T23:44:37.869 INFO:teuthology.orchestra.run.trial085.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-20T23:44:37.869 INFO:teuthology.orchestra.run.trial085.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-20T23:44:37.869 INFO:teuthology.orchestra.run.trial085.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-20T23:44:37.869 INFO:teuthology.orchestra.run.trial085.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-20T23:44:37.869 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:44:37.869 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:44:37.869 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:44:37.869 INFO:teuthology.orchestra.run.trial085.stdout:Install 12 Packages 2026-02-20T23:44:37.869 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:44:37.869 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 131 M 2026-02-20T23:44:37.869 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 981 M 2026-02-20T23:44:37.870 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:44:38.282 INFO:teuthology.orchestra.run.trial085.stdout:(1/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 14 MB/s | 2.1 MB 00:00 2026-02-20T23:44:38.304 INFO:teuthology.orchestra.run.trial085.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 1.0 MB/s | 23 kB 00:00 2026-02-20T23:44:38.327 INFO:teuthology.orchestra.run.trial085.stdout:(3/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 4.1 MB/s | 94 kB 00:00 2026-02-20T23:44:38.787 INFO:teuthology.orchestra.run.trial085.stdout:(4/12): libcephsqlite-20.3.0-5343.gbcdd429b.el9 234 kB/s | 153 kB 00:00 2026-02-20T23:44:39.320 INFO:teuthology.orchestra.run.trial085.stdout:(5/12): libasan8-15.2.1-7.el9.x86_64.rpm 528 kB/s | 525 kB 00:00 2026-02-20T23:44:39.441 INFO:teuthology.orchestra.run.trial085.stdout:(6/12): libubsan-11.5.0-14.el9.x86_64.rpm 269 kB/s | 176 kB 00:00 2026-02-20T23:44:39.656 INFO:teuthology.orchestra.run.trial085.stdout:(7/12): libxslt-1.1.34-12.el9.x86_64.rpm 694 kB/s | 233 kB 00:00 2026-02-20T23:44:40.070 INFO:teuthology.orchestra.run.trial085.stdout:(8/12): socat-1.7.4.1-8.el9.x86_64.rpm 732 kB/s | 303 kB 00:00 2026-02-20T23:44:40.170 INFO:teuthology.orchestra.run.trial085.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 641 kB/s | 64 kB 00:00 2026-02-20T23:44:40.304 INFO:teuthology.orchestra.run.trial085.stdout:(10/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 909 kB/s | 122 kB 00:00 2026-02-20T23:44:40.637 INFO:teuthology.orchestra.run.trial085.stdout:(11/12): protobuf-3.14.0-17.el9.x86_64.rpm 860 kB/s | 1.0 MB 00:01 2026-02-20T23:47:14.514 INFO:teuthology.orchestra.run.trial085.stdout:(12/12): ceph-test-20.3.0-5343.gbcdd429b.el9.x8 827 kB/s | 126 MB 02:36 2026-02-20T23:47:14.517 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:47:14.517 INFO:teuthology.orchestra.run.trial085.stdout:Total 857 kB/s | 131 MB 02:36 2026-02-20T23:47:14.589 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:47:14.600 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:47:14.600 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:47:15.027 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:47:15.028 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:47:15.490 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:47:15.499 INFO:teuthology.orchestra.run.trial085.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-20T23:47:15.507 INFO:teuthology.orchestra.run.trial085.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-20T23:47:15.512 INFO:teuthology.orchestra.run.trial085.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-20T23:47:15.516 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-20T23:47:15.519 INFO:teuthology.orchestra.run.trial085.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-20T23:47:15.523 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-20T23:47:15.524 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-20T23:47:15.539 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-20T23:47:15.541 INFO:teuthology.orchestra.run.trial085.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-20T23:47:15.550 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-20T23:47:15.559 INFO:teuthology.orchestra.run.trial085.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-20T23:47:15.560 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libcephsqlite-2:20.3.0-5343.gbcdd429b.el9.x86_64 11/12 2026-02-20T23:47:22.477 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5343.gbcdd429b.el9.x86_64 11/12 2026-02-20T23:47:22.494 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 12/12 2026-02-20T23:47:23.320 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 12/12 2026-02-20T23:47:23.320 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/12 2026-02-20T23:47:23.320 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libcephsqlite-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/12 2026-02-20T23:47:23.321 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-20T23:47:23.321 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-20T23:47:23.321 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-20T23:47:23.321 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-20T23:47:23.321 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-20T23:47:23.321 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-20T23:47:23.321 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-20T23:47:23.321 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-20T23:47:23.321 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-20T23:47:23.375 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-20T23:47:23.375 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:47:23.375 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-20T23:47:23.375 INFO:teuthology.orchestra.run.trial085.stdout: ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:23.375 INFO:teuthology.orchestra.run.trial085.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-20T23:47:23.375 INFO:teuthology.orchestra.run.trial085.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-20T23:47:23.375 INFO:teuthology.orchestra.run.trial085.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-20T23:47:23.375 INFO:teuthology.orchestra.run.trial085.stdout: libcephsqlite-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:23.376 INFO:teuthology.orchestra.run.trial085.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-20T23:47:23.376 INFO:teuthology.orchestra.run.trial085.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-20T23:47:23.376 INFO:teuthology.orchestra.run.trial085.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-20T23:47:23.376 INFO:teuthology.orchestra.run.trial085.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-20T23:47:23.376 INFO:teuthology.orchestra.run.trial085.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-20T23:47:23.376 INFO:teuthology.orchestra.run.trial085.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-20T23:47:23.376 INFO:teuthology.orchestra.run.trial085.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-20T23:47:23.376 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:47:23.376 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:47:23.414 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph 2026-02-20T23:47:23.718 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:03:23 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:47:23.802 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:47:23.803 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:47:23.803 INFO:teuthology.orchestra.run.trial085.stdout: Package Arch Version Repository Size 2026-02-20T23:47:23.804 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:47:23.804 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:47:23.804 INFO:teuthology.orchestra.run.trial085.stdout: ceph x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 6.5 k 2026-02-20T23:47:23.804 INFO:teuthology.orchestra.run.trial085.stdout:Installing dependencies: 2026-02-20T23:47:23.804 INFO:teuthology.orchestra.run.trial085.stdout: ceph-mds x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 2.0 M 2026-02-20T23:47:23.804 INFO:teuthology.orchestra.run.trial085.stdout: ceph-mgr x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 937 k 2026-02-20T23:47:23.804 INFO:teuthology.orchestra.run.trial085.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 297 k 2026-02-20T23:47:23.804 INFO:teuthology.orchestra.run.trial085.stdout: ceph-mon x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 4.9 M 2026-02-20T23:47:23.804 INFO:teuthology.orchestra.run.trial085.stdout: ceph-osd x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 163 k 2026-02-20T23:47:23.805 INFO:teuthology.orchestra.run.trial085.stdout: ceph-osd-classic x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 15 M 2026-02-20T23:47:23.805 INFO:teuthology.orchestra.run.trial085.stdout: ceph-osd-crimson x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 88 M 2026-02-20T23:47:23.805 INFO:teuthology.orchestra.run.trial085.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-20T23:47:23.805 INFO:teuthology.orchestra.run.trial085.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-20T23:47:23.805 INFO:teuthology.orchestra.run.trial085.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-20T23:47:23.805 INFO:teuthology.orchestra.run.trial085.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-20T23:47:23.805 INFO:teuthology.orchestra.run.trial085.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-20T23:47:23.805 INFO:teuthology.orchestra.run.trial085.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-20T23:47:23.805 INFO:teuthology.orchestra.run.trial085.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-20T23:47:23.805 INFO:teuthology.orchestra.run.trial085.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-20T23:47:23.805 INFO:teuthology.orchestra.run.trial085.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-20T23:47:23.805 INFO:teuthology.orchestra.run.trial085.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-20T23:47:23.805 INFO:teuthology.orchestra.run.trial085.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-20T23:47:23.806 INFO:teuthology.orchestra.run.trial085.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-20T23:47:23.806 INFO:teuthology.orchestra.run.trial085.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-20T23:47:23.806 INFO:teuthology.orchestra.run.trial085.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-20T23:47:23.806 INFO:teuthology.orchestra.run.trial085.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-20T23:47:23.806 INFO:teuthology.orchestra.run.trial085.stdout: python3-jaraco-collections 2026-02-20T23:47:23.806 INFO:teuthology.orchestra.run.trial085.stdout: noarch 3.0.0-8.el9 epel 23 k 2026-02-20T23:47:23.806 INFO:teuthology.orchestra.run.trial085.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-20T23:47:23.806 INFO:teuthology.orchestra.run.trial085.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-20T23:47:23.806 INFO:teuthology.orchestra.run.trial085.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-20T23:47:23.806 INFO:teuthology.orchestra.run.trial085.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-20T23:47:23.806 INFO:teuthology.orchestra.run.trial085.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-20T23:47:23.806 INFO:teuthology.orchestra.run.trial085.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-20T23:47:23.807 INFO:teuthology.orchestra.run.trial085.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-20T23:47:23.807 INFO:teuthology.orchestra.run.trial085.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-20T23:47:23.807 INFO:teuthology.orchestra.run.trial085.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-20T23:47:23.807 INFO:teuthology.orchestra.run.trial085.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-20T23:47:23.807 INFO:teuthology.orchestra.run.trial085.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-20T23:47:23.807 INFO:teuthology.orchestra.run.trial085.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-20T23:47:23.807 INFO:teuthology.orchestra.run.trial085.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-20T23:47:23.807 INFO:teuthology.orchestra.run.trial085.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-20T23:47:23.807 INFO:teuthology.orchestra.run.trial085.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-20T23:47:23.807 INFO:teuthology.orchestra.run.trial085.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-20T23:47:23.807 INFO:teuthology.orchestra.run.trial085.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-20T23:47:23.807 INFO:teuthology.orchestra.run.trial085.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-20T23:47:23.807 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:47:23.808 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:47:23.808 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:47:23.808 INFO:teuthology.orchestra.run.trial085.stdout:Install 42 Packages 2026-02-20T23:47:23.808 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:47:23.809 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 115 M 2026-02-20T23:47:23.809 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 974 M 2026-02-20T23:47:23.809 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:47:24.426 INFO:teuthology.orchestra.run.trial085.stdout:(1/42): ceph-20.3.0-5343.gbcdd429b.el9.x86_64.r 22 kB/s | 6.5 kB 00:00 2026-02-20T23:47:27.023 INFO:teuthology.orchestra.run.trial085.stdout:(2/42): ceph-mgr-20.3.0-5343.gbcdd429b.el9.x86_ 324 kB/s | 937 kB 00:02 2026-02-20T23:47:27.437 INFO:teuthology.orchestra.run.trial085.stdout:(3/42): ceph-osd-20.3.0-5343.gbcdd429b.el9.x86_ 394 kB/s | 163 kB 00:00 2026-02-20T23:47:29.382 INFO:teuthology.orchestra.run.trial085.stdout:(4/42): ceph-mds-20.3.0-5343.gbcdd429b.el9.x86_ 395 kB/s | 2.0 MB 00:05 2026-02-20T23:47:33.824 INFO:teuthology.orchestra.run.trial085.stdout:(5/42): ceph-mon-20.3.0-5343.gbcdd429b.el9.x86_ 530 kB/s | 4.9 MB 00:09 2026-02-20T23:47:34.241 INFO:teuthology.orchestra.run.trial085.stdout:(6/42): ceph-mgr-modules-core-20.3.0-5343.gbcdd 712 kB/s | 297 kB 00:00 2026-02-20T23:47:34.798 INFO:teuthology.orchestra.run.trial085.stdout:(7/42): ledmon-libs-1.1.0-3.el9.x86_64.rpm 72 kB/s | 40 kB 00:00 2026-02-20T23:47:35.035 INFO:teuthology.orchestra.run.trial085.stdout:(8/42): libconfig-1.7.2-9.el9.x86_64.rpm 305 kB/s | 72 kB 00:00 2026-02-20T23:47:35.272 INFO:teuthology.orchestra.run.trial085.stdout:(9/42): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.0 MB/s | 253 kB 00:00 2026-02-20T23:47:35.589 INFO:teuthology.orchestra.run.trial085.stdout:(10/42): python3-cryptography-36.0.1-5.el9.x86_ 3.9 MB/s | 1.2 MB 00:00 2026-02-20T23:47:35.748 INFO:teuthology.orchestra.run.trial085.stdout:(11/42): python3-ply-3.11-14.el9.noarch.rpm 670 kB/s | 106 kB 00:00 2026-02-20T23:47:35.906 INFO:teuthology.orchestra.run.trial085.stdout:(12/42): python3-pycparser-2.20-6.el9.noarch.rp 851 kB/s | 135 kB 00:00 2026-02-20T23:47:36.066 INFO:teuthology.orchestra.run.trial085.stdout:(13/42): python3-pyparsing-2.4.7-9.el9.noarch.r 942 kB/s | 150 kB 00:00 2026-02-20T23:47:36.225 INFO:teuthology.orchestra.run.trial085.stdout:(14/42): python3-requests-2.25.1-10.el9.noarch. 796 kB/s | 126 kB 00:00 2026-02-20T23:47:36.385 INFO:teuthology.orchestra.run.trial085.stdout:(15/42): python3-urllib3-1.26.5-7.el9.noarch.rp 1.3 MB/s | 218 kB 00:00 2026-02-20T23:47:36.543 INFO:teuthology.orchestra.run.trial085.stdout:(16/42): unzip-6.0-59.el9.x86_64.rpm 1.1 MB/s | 182 kB 00:00 2026-02-20T23:47:36.702 INFO:teuthology.orchestra.run.trial085.stdout:(17/42): zip-3.0-35.el9.x86_64.rpm 1.6 MB/s | 266 kB 00:00 2026-02-20T23:47:36.724 INFO:teuthology.orchestra.run.trial085.stdout:(18/42): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 11 MB/s | 246 kB 00:00 2026-02-20T23:47:36.731 INFO:teuthology.orchestra.run.trial085.stdout:(19/42): lua-5.4.4-4.el9.x86_64.rpm 26 MB/s | 188 kB 00:00 2026-02-20T23:47:36.738 INFO:teuthology.orchestra.run.trial085.stdout:(20/42): python3-libstoragemgmt-1.10.1-1.el9.x8 26 MB/s | 177 kB 00:00 2026-02-20T23:47:36.743 INFO:teuthology.orchestra.run.trial085.stdout:(21/42): python3-packaging-20.9-5.el9.noarch.rp 13 MB/s | 77 kB 00:00 2026-02-20T23:47:36.748 INFO:teuthology.orchestra.run.trial085.stdout:(22/42): python3-toml-0.10.2-6.el9.noarch.rpm 8.4 MB/s | 42 kB 00:00 2026-02-20T23:47:36.779 INFO:teuthology.orchestra.run.trial085.stdout:(23/42): lua-devel-5.4.4-4.el9.x86_64.rpm 721 kB/s | 22 kB 00:00 2026-02-20T23:47:36.832 INFO:teuthology.orchestra.run.trial085.stdout:(24/42): luarocks-3.9.2-5.el9.noarch.rpm 2.8 MB/s | 151 kB 00:00 2026-02-20T23:47:36.848 INFO:teuthology.orchestra.run.trial085.stdout:(25/42): python3-autocommand-2.2.2-8.el9.noarch 1.8 MB/s | 29 kB 00:00 2026-02-20T23:47:36.865 INFO:teuthology.orchestra.run.trial085.stdout:(26/42): python3-backports-tarfile-1.2.0-1.el9. 3.3 MB/s | 60 kB 00:00 2026-02-20T23:47:36.882 INFO:teuthology.orchestra.run.trial085.stdout:(27/42): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 2.6 MB/s | 43 kB 00:00 2026-02-20T23:47:36.906 INFO:teuthology.orchestra.run.trial085.stdout:(28/42): python3-cheroot-10.0.1-4.el9.noarch.rp 7.1 MB/s | 173 kB 00:00 2026-02-20T23:47:36.932 INFO:teuthology.orchestra.run.trial085.stdout:(29/42): python3-cherrypy-18.6.1-2.el9.noarch.r 13 MB/s | 358 kB 00:00 2026-02-20T23:47:36.940 INFO:teuthology.orchestra.run.trial085.stdout:(30/42): python3-jaraco-8.2.1-3.el9.noarch.rpm 1.3 MB/s | 11 kB 00:00 2026-02-20T23:47:36.948 INFO:teuthology.orchestra.run.trial085.stdout:(31/42): python3-jaraco-classes-3.2.1-5.el9.noa 2.1 MB/s | 18 kB 00:00 2026-02-20T23:47:36.956 INFO:teuthology.orchestra.run.trial085.stdout:(32/42): python3-jaraco-collections-3.0.0-8.el9 2.8 MB/s | 23 kB 00:00 2026-02-20T23:47:36.964 INFO:teuthology.orchestra.run.trial085.stdout:(33/42): python3-jaraco-context-6.0.1-3.el9.noa 2.4 MB/s | 20 kB 00:00 2026-02-20T23:47:36.972 INFO:teuthology.orchestra.run.trial085.stdout:(34/42): python3-jaraco-functools-3.5.0-2.el9.n 2.4 MB/s | 19 kB 00:00 2026-02-20T23:47:36.981 INFO:teuthology.orchestra.run.trial085.stdout:(35/42): python3-jaraco-text-4.0.0-2.el9.noarch 3.2 MB/s | 26 kB 00:00 2026-02-20T23:47:36.990 INFO:teuthology.orchestra.run.trial085.stdout:(36/42): python3-more-itertools-8.12.0-2.el9.no 8.7 MB/s | 79 kB 00:00 2026-02-20T23:47:36.998 INFO:teuthology.orchestra.run.trial085.stdout:(37/42): python3-portend-3.1.0-2.el9.noarch.rpm 2.0 MB/s | 16 kB 00:00 2026-02-20T23:47:37.007 INFO:teuthology.orchestra.run.trial085.stdout:(38/42): python3-pyOpenSSL-21.0.0-1.el9.noarch. 9.7 MB/s | 90 kB 00:00 2026-02-20T23:47:37.015 INFO:teuthology.orchestra.run.trial085.stdout:(39/42): python3-tempora-5.0.0-2.el9.noarch.rpm 4.2 MB/s | 36 kB 00:00 2026-02-20T23:47:37.023 INFO:teuthology.orchestra.run.trial085.stdout:(40/42): python3-zc-lockfile-2.0-10.el9.noarch. 2.4 MB/s | 20 kB 00:00 2026-02-20T23:47:59.492 INFO:teuthology.orchestra.run.trial085.stdout:(41/42): ceph-osd-classic-20.3.0-5343.gbcdd429b 472 kB/s | 15 MB 00:32 2026-02-20T23:50:45.653 INFO:teuthology.orchestra.run.trial085.stdout:(42/42): ceph-osd-crimson-20.3.0-5343.gbcdd429b 458 kB/s | 88 MB 03:16 2026-02-20T23:50:45.659 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:50:45.659 INFO:teuthology.orchestra.run.trial085.stdout:Total 586 kB/s | 115 MB 03:21 2026-02-20T23:50:45.777 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:50:45.796 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:50:45.796 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:50:46.191 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:50:46.191 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:50:46.630 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:50:46.632 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/42 2026-02-20T23:50:46.636 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/42 2026-02-20T23:50:46.641 INFO:teuthology.orchestra.run.trial085.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/42 2026-02-20T23:50:46.643 INFO:teuthology.orchestra.run.trial085.stdout: Installing : unzip-6.0-59.el9.x86_64 4/42 2026-02-20T23:50:46.648 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 5/42 2026-02-20T23:50:46.672 INFO:teuthology.orchestra.run.trial085.stdout: Installing : zip-3.0-35.el9.x86_64 6/42 2026-02-20T23:50:46.675 INFO:teuthology.orchestra.run.trial085.stdout: Installing : luarocks-3.9.2-5.el9.noarch 7/42 2026-02-20T23:50:46.680 INFO:teuthology.orchestra.run.trial085.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 8/42 2026-02-20T23:50:46.685 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 9/42 2026-02-20T23:50:46.691 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 10/42 2026-02-20T23:50:46.694 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 11/42 2026-02-20T23:50:46.700 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 12/42 2026-02-20T23:50:46.705 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 13/42 2026-02-20T23:50:46.709 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 14/42 2026-02-20T23:50:46.714 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 15/42 2026-02-20T23:50:46.718 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 16/42 2026-02-20T23:50:46.724 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 17/42 2026-02-20T23:50:46.728 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 18/42 2026-02-20T23:50:46.750 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 19/42 2026-02-20T23:50:46.762 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 20/42 2026-02-20T23:50:46.768 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 21/42 2026-02-20T23:50:46.777 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 22/42 2026-02-20T23:50:46.783 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-20T23:50:46.796 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/42 2026-02-20T23:50:46.812 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/42 2026-02-20T23:50:46.886 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/42 2026-02-20T23:50:46.894 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 27/42 2026-02-20T23:50:46.917 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/42 2026-02-20T23:50:46.945 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 29/42 2026-02-20T23:50:46.951 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/42 2026-02-20T23:50:47.002 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/42 2026-02-20T23:50:47.042 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5343.gbcdd429b.el9. 32/42 2026-02-20T23:50:47.043 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 33/42 2026-02-20T23:50:47.066 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 33/42 2026-02-20T23:50:47.066 INFO:teuthology.orchestra.run.trial085.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:50:47.066 INFO:teuthology.orchestra.run.trial085.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-20T23:50:47.066 INFO:teuthology.orchestra.run.trial085.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T23:50:47.067 INFO:teuthology.orchestra.run.trial085.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T23:50:47.067 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:50:47.071 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 34/42 2026-02-20T23:50:47.087 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-20T23:50:47.087 INFO:teuthology.orchestra.run.trial085.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-20T23:50:47.087 INFO:teuthology.orchestra.run.trial085.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-20T23:50:47.087 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:50:47.096 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-20T23:50:47.119 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-20T23:50:47.119 INFO:teuthology.orchestra.run.trial085.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-20T23:50:47.119 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:50:47.764 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 36/42 2026-02-20T23:50:53.414 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_6 37/42 2026-02-20T23:50:53.424 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86_6 38/42 2026-02-20T23:50:53.425 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 39/42 2026-02-20T23:50:53.449 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 39/42 2026-02-20T23:50:53.449 INFO:teuthology.orchestra.run.trial085.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:50:53.449 INFO:teuthology.orchestra.run.trial085.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-20T23:50:53.449 INFO:teuthology.orchestra.run.trial085.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T23:50:53.449 INFO:teuthology.orchestra.run.trial085.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T23:50:53.449 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:50:53.668 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-mon-2:20.3.0-5343.gbcdd429b.el9.x86_64 40/42 2026-02-20T23:50:53.689 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-mon-2:20.3.0-5343.gbcdd429b.el9.x86_64 40/42 2026-02-20T23:50:53.689 INFO:teuthology.orchestra.run.trial085.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:50:53.689 INFO:teuthology.orchestra.run.trial085.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-20T23:50:53.689 INFO:teuthology.orchestra.run.trial085.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T23:50:53.690 INFO:teuthology.orchestra.run.trial085.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T23:50:53.690 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:50:53.778 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-mds-2:20.3.0-5343.gbcdd429b.el9.x86_64 41/42 2026-02-20T23:50:53.798 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-mds-2:20.3.0-5343.gbcdd429b.el9.x86_64 41/42 2026-02-20T23:50:53.798 INFO:teuthology.orchestra.run.trial085.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:50:53.798 INFO:teuthology.orchestra.run.trial085.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-20T23:50:53.799 INFO:teuthology.orchestra.run.trial085.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T23:50:53.799 INFO:teuthology.orchestra.run.trial085.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T23:50:53.799 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:50:53.799 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 42/42 2026-02-20T23:50:53.805 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_6 42/42 2026-02-20T23:50:53.827 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86_6 42/42 2026-02-20T23:50:54.764 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 42/42 2026-02-20T23:50:54.764 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/42 2026-02-20T23:50:54.764 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-mds-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/42 2026-02-20T23:50:54.765 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 3/42 2026-02-20T23:50:54.765 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-mon-2:20.3.0-5343.gbcdd429b.el9.x86_64 4/42 2026-02-20T23:50:54.765 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 5/42 2026-02-20T23:50:54.765 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_6 6/42 2026-02-20T23:50:54.765 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86_6 7/42 2026-02-20T23:50:54.765 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5343.gbcdd429b.el9. 8/42 2026-02-20T23:50:54.765 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 9/42 2026-02-20T23:50:54.765 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 10/42 2026-02-20T23:50:54.765 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/42 2026-02-20T23:50:54.765 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 12/42 2026-02-20T23:50:54.766 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/42 2026-02-20T23:50:54.766 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/42 2026-02-20T23:50:54.766 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 15/42 2026-02-20T23:50:54.766 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 16/42 2026-02-20T23:50:54.766 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 17/42 2026-02-20T23:50:54.766 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : unzip-6.0-59.el9.x86_64 18/42 2026-02-20T23:50:54.766 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : zip-3.0-35.el9.x86_64 19/42 2026-02-20T23:50:54.766 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 20/42 2026-02-20T23:50:54.766 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : lua-5.4.4-4.el9.x86_64 21/42 2026-02-20T23:50:54.766 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 22/42 2026-02-20T23:50:54.766 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-20T23:50:54.767 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 24/42 2026-02-20T23:50:54.767 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 25/42 2026-02-20T23:50:54.767 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 26/42 2026-02-20T23:50:54.767 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 27/42 2026-02-20T23:50:54.767 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 28/42 2026-02-20T23:50:54.767 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 29/42 2026-02-20T23:50:54.767 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 30/42 2026-02-20T23:50:54.767 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 31/42 2026-02-20T23:50:54.767 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 32/42 2026-02-20T23:50:54.767 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 33/42 2026-02-20T23:50:54.767 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 34/42 2026-02-20T23:50:54.768 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 35/42 2026-02-20T23:50:54.768 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 36/42 2026-02-20T23:50:54.768 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 37/42 2026-02-20T23:50:54.768 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 38/42 2026-02-20T23:50:54.768 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 39/42 2026-02-20T23:50:54.768 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 40/42 2026-02-20T23:50:54.768 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 41/42 2026-02-20T23:50:54.821 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 42/42 2026-02-20T23:50:54.822 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:50:54.822 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-20T23:50:54.822 INFO:teuthology.orchestra.run.trial085.stdout: ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:50:54.822 INFO:teuthology.orchestra.run.trial085.stdout: ceph-mds-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:50:54.822 INFO:teuthology.orchestra.run.trial085.stdout: ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:50:54.822 INFO:teuthology.orchestra.run.trial085.stdout: ceph-mgr-modules-core-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:50:54.822 INFO:teuthology.orchestra.run.trial085.stdout: ceph-mon-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:50:54.822 INFO:teuthology.orchestra.run.trial085.stdout: ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:50:54.822 INFO:teuthology.orchestra.run.trial085.stdout: ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:50:54.822 INFO:teuthology.orchestra.run.trial085.stdout: ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:50:54.823 INFO:teuthology.orchestra.run.trial085.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-20T23:50:54.823 INFO:teuthology.orchestra.run.trial085.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-20T23:50:54.823 INFO:teuthology.orchestra.run.trial085.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T23:50:54.823 INFO:teuthology.orchestra.run.trial085.stdout: lua-5.4.4-4.el9.x86_64 2026-02-20T23:50:54.823 INFO:teuthology.orchestra.run.trial085.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-20T23:50:54.823 INFO:teuthology.orchestra.run.trial085.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-20T23:50:54.823 INFO:teuthology.orchestra.run.trial085.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-20T23:50:54.823 INFO:teuthology.orchestra.run.trial085.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-20T23:50:54.823 INFO:teuthology.orchestra.run.trial085.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-20T23:50:54.823 INFO:teuthology.orchestra.run.trial085.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-20T23:50:54.823 INFO:teuthology.orchestra.run.trial085.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-20T23:50:54.823 INFO:teuthology.orchestra.run.trial085.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-20T23:50:54.823 INFO:teuthology.orchestra.run.trial085.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-20T23:50:54.824 INFO:teuthology.orchestra.run.trial085.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-20T23:50:54.824 INFO:teuthology.orchestra.run.trial085.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-20T23:50:54.824 INFO:teuthology.orchestra.run.trial085.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-20T23:50:54.824 INFO:teuthology.orchestra.run.trial085.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-20T23:50:54.824 INFO:teuthology.orchestra.run.trial085.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-20T23:50:54.824 INFO:teuthology.orchestra.run.trial085.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-20T23:50:54.824 INFO:teuthology.orchestra.run.trial085.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T23:50:54.824 INFO:teuthology.orchestra.run.trial085.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-20T23:50:54.824 INFO:teuthology.orchestra.run.trial085.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-20T23:50:54.824 INFO:teuthology.orchestra.run.trial085.stdout: python3-ply-3.11-14.el9.noarch 2026-02-20T23:50:54.824 INFO:teuthology.orchestra.run.trial085.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-20T23:50:54.825 INFO:teuthology.orchestra.run.trial085.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-20T23:50:54.825 INFO:teuthology.orchestra.run.trial085.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-20T23:50:54.825 INFO:teuthology.orchestra.run.trial085.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-20T23:50:54.825 INFO:teuthology.orchestra.run.trial085.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-20T23:50:54.825 INFO:teuthology.orchestra.run.trial085.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-20T23:50:54.825 INFO:teuthology.orchestra.run.trial085.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-20T23:50:54.825 INFO:teuthology.orchestra.run.trial085.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-20T23:50:54.825 INFO:teuthology.orchestra.run.trial085.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-20T23:50:54.825 INFO:teuthology.orchestra.run.trial085.stdout: unzip-6.0-59.el9.x86_64 2026-02-20T23:50:54.825 INFO:teuthology.orchestra.run.trial085.stdout: zip-3.0-35.el9.x86_64 2026-02-20T23:50:54.825 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:50:54.826 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:50:54.864 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-base 2026-02-20T23:50:55.177 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:06:55 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:50:55.240 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:50:55.258 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:50:55.258 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-20T23:50:55.259 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:50:55.279 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install cephadm 2026-02-20T23:50:55.585 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:06:55 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:50:55.666 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:50:55.666 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:50:55.667 INFO:teuthology.orchestra.run.trial085.stdout: Package Arch Version Repository Size 2026-02-20T23:50:55.667 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:50:55.667 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:50:55.667 INFO:teuthology.orchestra.run.trial085.stdout: cephadm noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 1.0 M 2026-02-20T23:50:55.667 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:50:55.667 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:50:55.667 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:50:55.667 INFO:teuthology.orchestra.run.trial085.stdout:Install 1 Package 2026-02-20T23:50:55.667 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:50:55.667 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 1.0 M 2026-02-20T23:50:55.668 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 1.0 M 2026-02-20T23:50:55.668 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:50:59.327 INFO:teuthology.orchestra.run.trial085.stdout:cephadm-20.3.0-5343.gbcdd429b.el9.noarch.rpm 284 kB/s | 1.0 MB 00:03 2026-02-20T23:50:59.327 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:50:59.327 INFO:teuthology.orchestra.run.trial085.stdout:Total 284 kB/s | 1.0 MB 00:03 2026-02-20T23:50:59.327 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:50:59.330 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:50:59.330 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:50:59.334 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:50:59.334 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:50:59.352 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:50:59.412 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 1/1 2026-02-20T23:50:59.426 INFO:teuthology.orchestra.run.trial085.stdout: Installing : cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 1/1 2026-02-20T23:50:59.501 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 1/1 2026-02-20T23:50:59.557 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 1/1 2026-02-20T23:50:59.557 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:50:59.557 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-20T23:50:59.557 INFO:teuthology.orchestra.run.trial085.stdout: cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:50:59.557 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:50:59.557 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:50:59.583 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-immutable-object-cache 2026-02-20T23:50:59.888 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:06:59 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:50:59.969 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:50:59.970 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:50:59.971 INFO:teuthology.orchestra.run.trial085.stdout: Package Arch Version Repo Size 2026-02-20T23:50:59.971 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:50:59.971 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:50:59.971 INFO:teuthology.orchestra.run.trial085.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 149 k 2026-02-20T23:50:59.971 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:50:59.971 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:50:59.971 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:50:59.971 INFO:teuthology.orchestra.run.trial085.stdout:Install 1 Package 2026-02-20T23:50:59.971 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:50:59.971 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 149 k 2026-02-20T23:50:59.971 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 455 k 2026-02-20T23:50:59.972 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:51:01.223 INFO:teuthology.orchestra.run.trial085.stdout:ceph-immutable-object-cache-20.3.0-5343.gbcdd42 119 kB/s | 149 kB 00:01 2026-02-20T23:51:01.224 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:51:01.224 INFO:teuthology.orchestra.run.trial085.stdout:Total 119 kB/s | 149 kB 00:01 2026-02-20T23:51:01.224 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:51:01.228 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:51:01.229 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:51:01.254 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:51:01.255 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:51:01.310 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:51:01.311 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b. 1/1 2026-02-20T23:51:01.334 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b. 1/1 2026-02-20T23:51:01.334 INFO:teuthology.orchestra.run.trial085.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:51:01.334 INFO:teuthology.orchestra.run.trial085.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-20T23:51:01.334 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:51:01.630 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b. 1/1 2026-02-20T23:51:01.630 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:51:01.630 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-20T23:51:01.630 INFO:teuthology.orchestra.run.trial085.stdout: ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:51:01.630 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:51:01.630 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:51:01.656 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-mgr 2026-02-20T23:51:01.964 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:07:01 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:51:02.028 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:51:02.045 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:51:02.045 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-20T23:51:02.046 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:51:02.066 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-mgr-dashboard 2026-02-20T23:51:02.374 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:07:02 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:51:02.458 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:51:02.459 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:51:02.459 INFO:teuthology.orchestra.run.trial085.stdout: Package Arch Version Repository Size 2026-02-20T23:51:02.459 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:51:02.459 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:51:02.459 INFO:teuthology.orchestra.run.trial085.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 51 M 2026-02-20T23:51:02.459 INFO:teuthology.orchestra.run.trial085.stdout:Installing dependencies: 2026-02-20T23:51:02.459 INFO:teuthology.orchestra.run.trial085.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-20T23:51:02.459 INFO:teuthology.orchestra.run.trial085.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 57 k 2026-02-20T23:51:02.460 INFO:teuthology.orchestra.run.trial085.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 17 k 2026-02-20T23:51:02.460 INFO:teuthology.orchestra.run.trial085.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-20T23:51:02.460 INFO:teuthology.orchestra.run.trial085.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-20T23:51:02.460 INFO:teuthology.orchestra.run.trial085.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-20T23:51:02.460 INFO:teuthology.orchestra.run.trial085.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-20T23:51:02.460 INFO:teuthology.orchestra.run.trial085.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-20T23:51:02.460 INFO:teuthology.orchestra.run.trial085.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-20T23:51:02.460 INFO:teuthology.orchestra.run.trial085.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-20T23:51:02.460 INFO:teuthology.orchestra.run.trial085.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-20T23:51:02.460 INFO:teuthology.orchestra.run.trial085.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-20T23:51:02.460 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:51:02.461 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:51:02.461 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:51:02.461 INFO:teuthology.orchestra.run.trial085.stdout:Install 13 Packages 2026-02-20T23:51:02.461 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:51:02.461 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 55 M 2026-02-20T23:51:02.461 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 262 M 2026-02-20T23:51:02.461 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:51:03.070 INFO:teuthology.orchestra.run.trial085.stdout:(1/13): ceph-prometheus-alerts-20.3.0-5343.gbcd 48 kB/s | 17 kB 00:00 2026-02-20T23:51:03.177 INFO:teuthology.orchestra.run.trial085.stdout:(2/13): python3-jmespath-1.0.1-1.el9.noarch.rpm 445 kB/s | 48 kB 00:00 2026-02-20T23:51:03.211 INFO:teuthology.orchestra.run.trial085.stdout:(3/13): ceph-grafana-dashboards-20.3.0-5343.gbc 114 kB/s | 57 kB 00:00 2026-02-20T23:51:03.243 INFO:teuthology.orchestra.run.trial085.stdout:(4/13): python3-protobuf-3.14.0-17.el9.noarch.r 3.9 MB/s | 267 kB 00:00 2026-02-20T23:51:03.288 INFO:teuthology.orchestra.run.trial085.stdout:(5/13): abseil-cpp-20211102.0-4.el9.x86_64.rpm 12 MB/s | 551 kB 00:00 2026-02-20T23:51:03.292 INFO:teuthology.orchestra.run.trial085.stdout:(6/13): protobuf-compiler-3.14.0-17.el9.x86_64. 10 MB/s | 862 kB 00:00 2026-02-20T23:51:03.293 INFO:teuthology.orchestra.run.trial085.stdout:(7/13): grpc-data-1.46.7-10.el9.noarch.rpm 3.7 MB/s | 19 kB 00:00 2026-02-20T23:51:03.300 INFO:teuthology.orchestra.run.trial085.stdout:(8/13): python3-grpcio-tools-1.46.7-10.el9.x86_ 21 MB/s | 144 kB 00:00 2026-02-20T23:51:03.304 INFO:teuthology.orchestra.run.trial085.stdout:(9/13): python3-repoze-lru-0.7-16.el9.noarch.rp 7.6 MB/s | 31 kB 00:00 2026-02-20T23:51:03.309 INFO:teuthology.orchestra.run.trial085.stdout:(10/13): python3-routes-2.5.1-5.el9.noarch.rpm 37 MB/s | 188 kB 00:00 2026-02-20T23:51:03.313 INFO:teuthology.orchestra.run.trial085.stdout:(11/13): python3-xmltodict-0.12.0-15.el9.noarch 5.4 MB/s | 22 kB 00:00 2026-02-20T23:51:03.334 INFO:teuthology.orchestra.run.trial085.stdout:(12/13): python3-grpcio-1.46.7-10.el9.x86_64.rp 49 MB/s | 2.0 MB 00:00 2026-02-20T23:52:33.388 INFO:teuthology.orchestra.run.trial085.stdout:(13/13): ceph-mgr-dashboard-20.3.0-5343.gbcdd42 572 kB/s | 51 MB 01:30 2026-02-20T23:52:33.391 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:52:33.391 INFO:teuthology.orchestra.run.trial085.stdout:Total 618 kB/s | 55 MB 01:30 2026-02-20T23:52:33.415 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:52:33.425 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:52:33.425 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:52:33.640 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:52:33.640 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:52:33.870 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:52:33.876 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 1/13 2026-02-20T23:52:33.887 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 2/13 2026-02-20T23:52:33.890 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 3/13 2026-02-20T23:52:33.917 INFO:teuthology.orchestra.run.trial085.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 4/13 2026-02-20T23:52:33.962 INFO:teuthology.orchestra.run.trial085.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 5/13 2026-02-20T23:52:33.971 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 6/13 2026-02-20T23:52:33.999 INFO:teuthology.orchestra.run.trial085.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 7/13 2026-02-20T23:52:34.008 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 8/13 2026-02-20T23:52:34.016 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/13 2026-02-20T23:52:34.018 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 10/13 2026-02-20T23:52:34.028 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5343.gbcdd429b.el9 11/13 2026-02-20T23:52:36.923 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5343.gbcdd429b.el 12/13 2026-02-20T23:52:36.929 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noa 13/13 2026-02-20T23:52:37.200 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noa 13/13 2026-02-20T23:52:37.200 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5343.gbcdd429b.el 1/13 2026-02-20T23:52:37.200 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noa 2/13 2026-02-20T23:52:37.201 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5343.gbcdd429b.el9 3/13 2026-02-20T23:52:37.201 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 4/13 2026-02-20T23:52:37.201 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 5/13 2026-02-20T23:52:37.201 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 6/13 2026-02-20T23:52:37.201 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 7/13 2026-02-20T23:52:37.201 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/13 2026-02-20T23:52:37.201 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/13 2026-02-20T23:52:37.201 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/13 2026-02-20T23:52:37.201 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/13 2026-02-20T23:52:37.201 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/13 2026-02-20T23:52:37.256 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 13/13 2026-02-20T23:52:37.257 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:52:37.257 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-20T23:52:37.257 INFO:teuthology.orchestra.run.trial085.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-20T23:52:37.257 INFO:teuthology.orchestra.run.trial085.stdout: ceph-grafana-dashboards-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:52:37.257 INFO:teuthology.orchestra.run.trial085.stdout: ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:52:37.257 INFO:teuthology.orchestra.run.trial085.stdout: ceph-prometheus-alerts-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:52:37.257 INFO:teuthology.orchestra.run.trial085.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-20T23:52:37.257 INFO:teuthology.orchestra.run.trial085.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-20T23:52:37.257 INFO:teuthology.orchestra.run.trial085.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-20T23:52:37.257 INFO:teuthology.orchestra.run.trial085.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-20T23:52:37.258 INFO:teuthology.orchestra.run.trial085.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-20T23:52:37.258 INFO:teuthology.orchestra.run.trial085.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-20T23:52:37.258 INFO:teuthology.orchestra.run.trial085.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-20T23:52:37.258 INFO:teuthology.orchestra.run.trial085.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-20T23:52:37.258 INFO:teuthology.orchestra.run.trial085.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-20T23:52:37.258 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:52:37.258 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:52:37.292 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-20T23:52:37.599 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:08:37 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:52:37.683 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:52:37.684 INFO:teuthology.orchestra.run.trial085.stdout:======================================================================================= 2026-02-20T23:52:37.684 INFO:teuthology.orchestra.run.trial085.stdout: Package Arch Version Repository Size 2026-02-20T23:52:37.684 INFO:teuthology.orchestra.run.trial085.stdout:======================================================================================= 2026-02-20T23:52:37.684 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:52:37.684 INFO:teuthology.orchestra.run.trial085.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 7.4 M 2026-02-20T23:52:37.684 INFO:teuthology.orchestra.run.trial085.stdout:Installing dependencies: 2026-02-20T23:52:37.684 INFO:teuthology.orchestra.run.trial085.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-20T23:52:37.684 INFO:teuthology.orchestra.run.trial085.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-20T23:52:37.684 INFO:teuthology.orchestra.run.trial085.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-20T23:52:37.685 INFO:teuthology.orchestra.run.trial085.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-20T23:52:37.685 INFO:teuthology.orchestra.run.trial085.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-20T23:52:37.685 INFO:teuthology.orchestra.run.trial085.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-20T23:52:37.685 INFO:teuthology.orchestra.run.trial085.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-20T23:52:37.685 INFO:teuthology.orchestra.run.trial085.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-20T23:52:37.685 INFO:teuthology.orchestra.run.trial085.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-20T23:52:37.685 INFO:teuthology.orchestra.run.trial085.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-20T23:52:37.685 INFO:teuthology.orchestra.run.trial085.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-20T23:52:37.685 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:52:37.685 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:52:37.685 INFO:teuthology.orchestra.run.trial085.stdout:======================================================================================= 2026-02-20T23:52:37.685 INFO:teuthology.orchestra.run.trial085.stdout:Install 12 Packages 2026-02-20T23:52:37.686 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:52:37.686 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 43 M 2026-02-20T23:52:37.686 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 234 M 2026-02-20T23:52:37.686 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:52:38.190 INFO:teuthology.orchestra.run.trial085.stdout:(1/12): libquadmath-11.5.0-14.el9.x86_64.rpm 1.4 MB/s | 184 kB 00:00 2026-02-20T23:52:38.229 INFO:teuthology.orchestra.run.trial085.stdout:(2/12): libgfortran-11.5.0-14.el9.x86_64.rpm 4.5 MB/s | 794 kB 00:00 2026-02-20T23:52:38.595 INFO:teuthology.orchestra.run.trial085.stdout:(3/12): flexiblas-3.0.4-9.el9.x86_64.rpm 73 kB/s | 30 kB 00:00 2026-02-20T23:52:38.677 INFO:teuthology.orchestra.run.trial085.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-9.el9.x 180 kB/s | 15 kB 00:00 2026-02-20T23:52:38.837 INFO:teuthology.orchestra.run.trial085.stdout:(5/12): openblas-0.3.29-1.el9.x86_64.rpm 263 kB/s | 42 kB 00:00 2026-02-20T23:52:42.048 INFO:teuthology.orchestra.run.trial085.stdout:(6/12): flexiblas-netlib-3.0.4-9.el9.x86_64.rpm 801 kB/s | 3.0 MB 00:03 2026-02-20T23:52:42.365 INFO:teuthology.orchestra.run.trial085.stdout:(7/12): python3-devel-3.9.25-3.el9.x86_64.rpm 773 kB/s | 244 kB 00:00 2026-02-20T23:52:44.933 INFO:teuthology.orchestra.run.trial085.stdout:(8/12): openblas-openmp-0.3.29-1.el9.x86_64.rpm 888 kB/s | 5.3 MB 00:06 2026-02-20T23:52:45.483 INFO:teuthology.orchestra.run.trial085.stdout:(9/12): python3-numpy-f2py-1.23.5-2.el9.x86_64. 803 kB/s | 442 kB 00:00 2026-02-20T23:52:48.053 INFO:teuthology.orchestra.run.trial085.stdout:(10/12): python3-numpy-1.23.5-2.el9.x86_64.rpm 1.1 MB/s | 6.1 MB 00:05 2026-02-20T23:52:51.517 INFO:teuthology.orchestra.run.trial085.stdout:(11/12): python3-scipy-1.9.3-2.el9.x86_64.rpm 3.2 MB/s | 19 MB 00:06 2026-02-20T23:52:51.929 INFO:teuthology.orchestra.run.trial085.stdout:(12/12): ceph-mgr-diskprediction-local-20.3.0-5 545 kB/s | 7.4 MB 00:13 2026-02-20T23:52:51.931 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:52:51.931 INFO:teuthology.orchestra.run.trial085.stdout:Total 3.0 MB/s | 43 MB 00:14 2026-02-20T23:52:52.092 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:52:52.106 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:52:52.106 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:52:52.308 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:52:52.308 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:52:52.526 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:52:52.529 INFO:teuthology.orchestra.run.trial085.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 1/12 2026-02-20T23:52:52.535 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 2/12 2026-02-20T23:52:52.569 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 3/12 2026-02-20T23:52:52.573 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 4/12 2026-02-20T23:52:52.632 INFO:teuthology.orchestra.run.trial085.stdout: Installing : openblas-0.3.29-1.el9.x86_64 5/12 2026-02-20T23:52:52.634 INFO:teuthology.orchestra.run.trial085.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 6/12 2026-02-20T23:52:52.655 INFO:teuthology.orchestra.run.trial085.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 7/12 2026-02-20T23:52:52.977 INFO:teuthology.orchestra.run.trial085.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 8/12 2026-02-20T23:52:53.049 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 9/12 2026-02-20T23:52:53.712 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 10/12 2026-02-20T23:52:54.177 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 11/12 2026-02-20T23:52:54.178 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd4 12/12 2026-02-20T23:52:54.486 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd4 12/12 2026-02-20T23:52:54.487 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd4 1/12 2026-02-20T23:52:54.487 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 2/12 2026-02-20T23:52:54.487 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 3/12 2026-02-20T23:52:54.487 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 4/12 2026-02-20T23:52:54.487 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 5/12 2026-02-20T23:52:54.487 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 6/12 2026-02-20T23:52:54.487 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 7/12 2026-02-20T23:52:54.487 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 8/12 2026-02-20T23:52:54.487 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 9/12 2026-02-20T23:52:54.487 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 10/12 2026-02-20T23:52:54.488 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 11/12 2026-02-20T23:52:54.545 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 12/12 2026-02-20T23:52:54.545 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:52:54.545 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-20T23:52:54.545 INFO:teuthology.orchestra.run.trial085.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:52:54.546 INFO:teuthology.orchestra.run.trial085.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-20T23:52:54.546 INFO:teuthology.orchestra.run.trial085.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-20T23:52:54.546 INFO:teuthology.orchestra.run.trial085.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-20T23:52:54.546 INFO:teuthology.orchestra.run.trial085.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-20T23:52:54.546 INFO:teuthology.orchestra.run.trial085.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-20T23:52:54.546 INFO:teuthology.orchestra.run.trial085.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-20T23:52:54.546 INFO:teuthology.orchestra.run.trial085.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-20T23:52:54.546 INFO:teuthology.orchestra.run.trial085.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-20T23:52:54.546 INFO:teuthology.orchestra.run.trial085.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-20T23:52:54.546 INFO:teuthology.orchestra.run.trial085.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-20T23:52:54.546 INFO:teuthology.orchestra.run.trial085.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-20T23:52:54.546 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:52:54.547 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:52:54.584 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-mgr-rook 2026-02-20T23:52:54.895 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:08:54 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:52:54.979 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:52:54.980 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:52:54.980 INFO:teuthology.orchestra.run.trial085.stdout: Package Arch Version Repository Size 2026-02-20T23:52:54.980 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:52:54.980 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:52:54.980 INFO:teuthology.orchestra.run.trial085.stdout: ceph-mgr-rook noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 50 k 2026-02-20T23:52:54.980 INFO:teuthology.orchestra.run.trial085.stdout:Installing dependencies: 2026-02-20T23:52:54.980 INFO:teuthology.orchestra.run.trial085.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-20T23:52:54.980 INFO:teuthology.orchestra.run.trial085.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-20T23:52:54.980 INFO:teuthology.orchestra.run.trial085.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-20T23:52:54.981 INFO:teuthology.orchestra.run.trial085.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-20T23:52:54.981 INFO:teuthology.orchestra.run.trial085.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-20T23:52:54.981 INFO:teuthology.orchestra.run.trial085.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-20T23:52:54.981 INFO:teuthology.orchestra.run.trial085.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-20T23:52:54.981 INFO:teuthology.orchestra.run.trial085.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-20T23:52:54.981 INFO:teuthology.orchestra.run.trial085.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-20T23:52:54.981 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:52:54.981 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:52:54.981 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:52:54.981 INFO:teuthology.orchestra.run.trial085.stdout:Install 10 Packages 2026-02-20T23:52:54.981 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:52:54.982 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 2.0 M 2026-02-20T23:52:54.982 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 25 M 2026-02-20T23:52:54.982 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:52:55.203 INFO:teuthology.orchestra.run.trial085.stdout:(1/10): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.8 MB/s | 159 kB 00:00 2026-02-20T23:52:55.204 INFO:teuthology.orchestra.run.trial085.stdout:(2/10): python3-pyasn1-modules-0.4.8-6.el9.noar 3.2 MB/s | 279 kB 00:00 2026-02-20T23:52:55.227 INFO:teuthology.orchestra.run.trial085.stdout:(3/10): python3-requests-oauthlib-1.3.0-12.el9. 2.2 MB/s | 54 kB 00:00 2026-02-20T23:52:55.308 INFO:teuthology.orchestra.run.trial085.stdout:(4/10): python3-cachetools-4.2.4-1.el9.noarch.r 311 kB/s | 32 kB 00:00 2026-02-20T23:52:55.329 INFO:teuthology.orchestra.run.trial085.stdout:(5/10): python3-certifi-2023.05.07-4.el9.noarch 138 kB/s | 14 kB 00:00 2026-02-20T23:52:55.366 INFO:teuthology.orchestra.run.trial085.stdout:(6/10): python3-google-auth-2.45.0-1.el9.noarch 4.3 MB/s | 254 kB 00:00 2026-02-20T23:52:55.386 INFO:teuthology.orchestra.run.trial085.stdout:(7/10): python3-rsa-4.9-2.el9.noarch.rpm 2.9 MB/s | 59 kB 00:00 2026-02-20T23:52:55.407 INFO:teuthology.orchestra.run.trial085.stdout:(8/10): python3-websocket-client-1.2.3-2.el9.no 4.3 MB/s | 90 kB 00:00 2026-02-20T23:52:55.446 INFO:teuthology.orchestra.run.trial085.stdout:(9/10): python3-kubernetes-26.1.0-3.el9.noarch. 8.7 MB/s | 1.0 MB 00:00 2026-02-20T23:52:55.539 INFO:teuthology.orchestra.run.trial085.stdout:(10/10): ceph-mgr-rook-20.3.0-5343.gbcdd429b.el 119 kB/s | 50 kB 00:00 2026-02-20T23:52:55.541 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:52:55.541 INFO:teuthology.orchestra.run.trial085.stdout:Total 3.5 MB/s | 2.0 MB 00:00 2026-02-20T23:52:55.567 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:52:55.575 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:52:55.575 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:52:55.614 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:52:55.614 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:52:55.688 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:52:55.738 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/10 2026-02-20T23:52:55.751 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/10 2026-02-20T23:52:55.765 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/10 2026-02-20T23:52:55.770 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/10 2026-02-20T23:52:55.777 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/10 2026-02-20T23:52:55.816 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/10 2026-02-20T23:52:55.828 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-20T23:52:56.144 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 8/10 2026-02-20T23:52:56.158 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 9/10 2026-02-20T23:52:56.159 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch 10/10 2026-02-20T23:52:56.269 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch 10/10 2026-02-20T23:52:56.269 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch 1/10 2026-02-20T23:52:56.269 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 2/10 2026-02-20T23:52:56.269 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 3/10 2026-02-20T23:52:56.269 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 4/10 2026-02-20T23:52:56.269 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 5/10 2026-02-20T23:52:56.269 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 6/10 2026-02-20T23:52:56.269 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-20T23:52:56.269 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 8/10 2026-02-20T23:52:56.270 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 9/10 2026-02-20T23:52:56.326 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 10/10 2026-02-20T23:52:56.327 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:52:56.327 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-20T23:52:56.327 INFO:teuthology.orchestra.run.trial085.stdout: ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:52:56.327 INFO:teuthology.orchestra.run.trial085.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-20T23:52:56.327 INFO:teuthology.orchestra.run.trial085.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-20T23:52:56.327 INFO:teuthology.orchestra.run.trial085.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-20T23:52:56.327 INFO:teuthology.orchestra.run.trial085.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-20T23:52:56.327 INFO:teuthology.orchestra.run.trial085.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-20T23:52:56.327 INFO:teuthology.orchestra.run.trial085.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-20T23:52:56.327 INFO:teuthology.orchestra.run.trial085.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-20T23:52:56.328 INFO:teuthology.orchestra.run.trial085.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-20T23:52:56.328 INFO:teuthology.orchestra.run.trial085.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-20T23:52:56.328 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:52:56.328 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:52:56.359 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-mgr-cephadm 2026-02-20T23:52:56.669 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:08:56 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:52:56.752 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:52:56.752 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:52:56.753 INFO:teuthology.orchestra.run.trial085.stdout: Package Arch Version Repository Size 2026-02-20T23:52:56.753 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:52:56.753 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:52:56.753 INFO:teuthology.orchestra.run.trial085.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 184 k 2026-02-20T23:52:56.753 INFO:teuthology.orchestra.run.trial085.stdout:Installing dependencies: 2026-02-20T23:52:56.753 INFO:teuthology.orchestra.run.trial085.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-20T23:52:56.753 INFO:teuthology.orchestra.run.trial085.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-20T23:52:56.753 INFO:teuthology.orchestra.run.trial085.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-20T23:52:56.753 INFO:teuthology.orchestra.run.trial085.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-20T23:52:56.753 INFO:teuthology.orchestra.run.trial085.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-20T23:52:56.753 INFO:teuthology.orchestra.run.trial085.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-20T23:52:56.754 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:52:56.754 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:52:56.754 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:52:56.754 INFO:teuthology.orchestra.run.trial085.stdout:Install 7 Packages 2026-02-20T23:52:56.754 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:52:56.754 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 7.1 M 2026-02-20T23:52:56.754 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 34 M 2026-02-20T23:52:56.754 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:52:57.421 INFO:teuthology.orchestra.run.trial085.stdout:(1/7): python3-jinja2-2.11.3-8.el9.noarch.rpm 494 kB/s | 249 kB 00:00 2026-02-20T23:52:57.510 INFO:teuthology.orchestra.run.trial085.stdout:(2/7): python3-markupsafe-1.1.1-12.el9.x86_64.r 391 kB/s | 35 kB 00:00 2026-02-20T23:52:57.708 INFO:teuthology.orchestra.run.trial085.stdout:(3/7): python3-asyncssh-2.13.2-5.el9.noarch.rpm 2.7 MB/s | 548 kB 00:00 2026-02-20T23:52:57.735 INFO:teuthology.orchestra.run.trial085.stdout:(4/7): python3-natsort-7.1.1-5.el9.noarch.rpm 2.0 MB/s | 58 kB 00:00 2026-02-20T23:52:57.763 INFO:teuthology.orchestra.run.trial085.stdout:(5/7): python3-typing-extensions-4.15.0-1.el9.n 3.1 MB/s | 86 kB 00:00 2026-02-20T23:52:57.786 INFO:teuthology.orchestra.run.trial085.stdout:(6/7): python3-babel-2.9.1-2.el9.noarch.rpm 6.9 MB/s | 6.0 MB 00:00 2026-02-20T23:52:57.826 INFO:teuthology.orchestra.run.trial085.stdout:(7/7): ceph-mgr-cephadm-20.3.0-5343.gbcdd429b.e 202 kB/s | 184 kB 00:00 2026-02-20T23:52:57.827 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:52:57.827 INFO:teuthology.orchestra.run.trial085.stdout:Total 6.6 MB/s | 7.1 MB 00:01 2026-02-20T23:52:57.868 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:52:57.875 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:52:57.875 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:52:57.948 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:52:57.949 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:52:58.039 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:52:58.088 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 1/7 2026-02-20T23:52:58.098 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/7 2026-02-20T23:52:58.104 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/7 2026-02-20T23:52:58.326 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-20T23:52:58.350 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 5/7 2026-02-20T23:52:58.374 INFO:teuthology.orchestra.run.trial085.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 6/7 2026-02-20T23:52:58.376 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 7/7 2026-02-20T23:52:58.485 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 7/7 2026-02-20T23:52:58.485 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 1/7 2026-02-20T23:52:58.486 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/7 2026-02-20T23:52:58.486 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 3/7 2026-02-20T23:52:58.486 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-20T23:52:58.486 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 5/7 2026-02-20T23:52:58.486 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 6/7 2026-02-20T23:52:58.543 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 7/7 2026-02-20T23:52:58.543 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:52:58.543 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-20T23:52:58.543 INFO:teuthology.orchestra.run.trial085.stdout: ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:52:58.543 INFO:teuthology.orchestra.run.trial085.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-20T23:52:58.543 INFO:teuthology.orchestra.run.trial085.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-20T23:52:58.543 INFO:teuthology.orchestra.run.trial085.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-20T23:52:58.544 INFO:teuthology.orchestra.run.trial085.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-20T23:52:58.544 INFO:teuthology.orchestra.run.trial085.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-20T23:52:58.544 INFO:teuthology.orchestra.run.trial085.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-20T23:52:58.544 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:52:58.544 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:52:58.574 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-osd 2026-02-20T23:52:58.883 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:08:58 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:52:58.948 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:52:58.966 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:52:58.967 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-20T23:52:58.967 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:52:58.988 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-osd-classic 2026-02-20T23:52:59.299 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:08:59 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:52:59.365 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:52:59.383 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:52:59.384 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-20T23:52:59.384 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:52:59.405 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-fuse 2026-02-20T23:52:59.713 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:08:59 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:52:59.795 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:52:59.796 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:52:59.797 INFO:teuthology.orchestra.run.trial085.stdout: Package Arch Version Repo Size 2026-02-20T23:52:59.797 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:52:59.797 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:52:59.797 INFO:teuthology.orchestra.run.trial085.stdout: ceph-fuse x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 914 k 2026-02-20T23:52:59.797 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:52:59.797 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:52:59.797 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:52:59.797 INFO:teuthology.orchestra.run.trial085.stdout:Install 1 Package 2026-02-20T23:52:59.797 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:52:59.797 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 914 k 2026-02-20T23:52:59.798 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 2.9 M 2026-02-20T23:52:59.798 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:53:01.736 INFO:teuthology.orchestra.run.trial085.stdout:ceph-fuse-20.3.0-5343.gbcdd429b.el9.x86_64.rpm 471 kB/s | 914 kB 00:01 2026-02-20T23:53:01.736 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:53:01.736 INFO:teuthology.orchestra.run.trial085.stdout:Total 471 kB/s | 914 kB 00:01 2026-02-20T23:53:01.736 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:53:01.741 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:53:01.741 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:53:01.774 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:53:01.774 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:53:01.863 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:53:01.881 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:53:02.125 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:53:02.182 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:53:02.182 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:02.183 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-20T23:53:02.183 INFO:teuthology.orchestra.run.trial085.stdout: ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:53:02.183 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:02.183 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:53:02.210 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-volume 2026-02-20T23:53:02.524 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:09:02 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:53:02.610 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:53:02.611 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:02.611 INFO:teuthology.orchestra.run.trial085.stdout: Package Arch Version Repository Size 2026-02-20T23:53:02.611 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:02.611 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:53:02.611 INFO:teuthology.orchestra.run.trial085.stdout: ceph-volume noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 301 k 2026-02-20T23:53:02.611 INFO:teuthology.orchestra.run.trial085.stdout:Installing dependencies: 2026-02-20T23:53:02.612 INFO:teuthology.orchestra.run.trial085.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-20T23:53:02.612 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:02.612 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:53:02.612 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:02.612 INFO:teuthology.orchestra.run.trial085.stdout:Install 2 Packages 2026-02-20T23:53:02.612 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:02.612 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 652 k 2026-02-20T23:53:02.612 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 2.2 M 2026-02-20T23:53:02.612 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:53:04.248 INFO:teuthology.orchestra.run.trial085.stdout:(1/2): cryptsetup-2.8.1-3.el9.x86_64.rpm 223 kB/s | 351 kB 00:01 2026-02-20T23:53:04.472 INFO:teuthology.orchestra.run.trial085.stdout:(2/2): ceph-volume-20.3.0-5343.gbcdd429b.el9.no 167 kB/s | 301 kB 00:01 2026-02-20T23:53:04.473 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:53:04.473 INFO:teuthology.orchestra.run.trial085.stdout:Total 350 kB/s | 652 kB 00:01 2026-02-20T23:53:04.483 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:53:04.489 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:53:04.489 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:53:04.532 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:53:04.533 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:53:04.613 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:53:04.676 INFO:teuthology.orchestra.run.trial085.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 1/2 2026-02-20T23:53:04.678 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch 2/2 2026-02-20T23:53:04.693 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch 2/2 2026-02-20T23:53:04.693 INFO:teuthology.orchestra.run.trial085.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:53:04.693 INFO:teuthology.orchestra.run.trial085.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-20T23:53:04.693 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:04.910 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch 1/2 2026-02-20T23:53:04.968 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 2/2 2026-02-20T23:53:04.969 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:04.969 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-20T23:53:04.969 INFO:teuthology.orchestra.run.trial085.stdout: ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch cryptsetup-2.8.1-3.el9.x86_64 2026-02-20T23:53:04.969 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:04.969 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:53:04.997 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install librados-devel 2026-02-20T23:53:05.313 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:09:05 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:53:05.396 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:53:05.396 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:05.396 INFO:teuthology.orchestra.run.trial085.stdout: Package Arch Version Repo Size 2026-02-20T23:53:05.396 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:05.396 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:53:05.397 INFO:teuthology.orchestra.run.trial085.stdout: librados-devel x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 124 k 2026-02-20T23:53:05.397 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:05.397 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:53:05.397 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:05.397 INFO:teuthology.orchestra.run.trial085.stdout:Install 1 Package 2026-02-20T23:53:05.397 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:05.397 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 124 k 2026-02-20T23:53:05.397 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 434 k 2026-02-20T23:53:05.397 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:53:06.231 INFO:teuthology.orchestra.run.trial085.stdout:librados-devel-20.3.0-5343.gbcdd429b.el9.x86_64 148 kB/s | 124 kB 00:00 2026-02-20T23:53:06.232 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:53:06.232 INFO:teuthology.orchestra.run.trial085.stdout:Total 148 kB/s | 124 kB 00:00 2026-02-20T23:53:06.232 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:53:06.235 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:53:06.235 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:53:06.266 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:53:06.266 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:53:06.320 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:53:06.337 INFO:teuthology.orchestra.run.trial085.stdout: Installing : librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:53:06.415 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:53:06.474 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:53:06.474 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:06.475 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-20T23:53:06.475 INFO:teuthology.orchestra.run.trial085.stdout: librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:53:06.475 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:06.475 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:53:06.502 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install libcephfs2 2026-02-20T23:53:06.813 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:09:06 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:53:06.877 INFO:teuthology.orchestra.run.trial085.stdout:Package libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:53:06.895 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:53:06.896 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-20T23:53:06.896 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:53:06.918 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install libcephfs-devel 2026-02-20T23:53:07.225 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:09:07 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:53:07.308 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:53:07.308 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:07.308 INFO:teuthology.orchestra.run.trial085.stdout: Package Arch Version Repo Size 2026-02-20T23:53:07.308 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:07.309 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:53:07.309 INFO:teuthology.orchestra.run.trial085.stdout: libcephfs-devel x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 37 k 2026-02-20T23:53:07.309 INFO:teuthology.orchestra.run.trial085.stdout:Installing dependencies: 2026-02-20T23:53:07.309 INFO:teuthology.orchestra.run.trial085.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 26 k 2026-02-20T23:53:07.309 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:07.309 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:53:07.309 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:07.309 INFO:teuthology.orchestra.run.trial085.stdout:Install 2 Packages 2026-02-20T23:53:07.309 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:07.309 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 63 k 2026-02-20T23:53:07.309 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 227 k 2026-02-20T23:53:07.310 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:53:07.662 INFO:teuthology.orchestra.run.trial085.stdout:(1/2): libcephfs-devel-20.3.0-5343.gbcdd429b.el 106 kB/s | 37 kB 00:00 2026-02-20T23:53:07.665 INFO:teuthology.orchestra.run.trial085.stdout:(2/2): libcephfs-proxy2-20.3.0-5343.gbcdd429b.e 72 kB/s | 26 kB 00:00 2026-02-20T23:53:07.666 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:53:07.666 INFO:teuthology.orchestra.run.trial085.stdout:Total 176 kB/s | 63 kB 00:00 2026-02-20T23:53:07.666 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:53:07.670 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:53:07.670 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:53:07.702 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:53:07.702 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:53:07.751 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:53:07.752 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libcephfs-proxy2-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/2 2026-02-20T23:53:07.768 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/2 2026-02-20T23:53:07.784 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/2 2026-02-20T23:53:07.838 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/2 2026-02-20T23:53:07.839 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/2 2026-02-20T23:53:07.896 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/2 2026-02-20T23:53:07.897 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:07.897 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-20T23:53:07.897 INFO:teuthology.orchestra.run.trial085.stdout: libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:53:07.897 INFO:teuthology.orchestra.run.trial085.stdout: libcephfs-proxy2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:53:07.897 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:07.897 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:53:07.923 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install librados2 2026-02-20T23:53:08.236 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:09:08 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:53:08.301 INFO:teuthology.orchestra.run.trial085.stdout:Package librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:53:08.319 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:53:08.320 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-20T23:53:08.320 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:53:08.341 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install librbd1 2026-02-20T23:53:08.655 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:09:08 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:53:08.720 INFO:teuthology.orchestra.run.trial085.stdout:Package librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:53:08.739 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:53:08.739 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-20T23:53:08.739 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:53:08.760 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install python3-rados 2026-02-20T23:53:09.068 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:09:09 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:53:09.135 INFO:teuthology.orchestra.run.trial085.stdout:Package python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:53:09.153 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:53:09.153 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-20T23:53:09.153 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:53:09.175 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install python3-rgw 2026-02-20T23:53:09.483 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:09:09 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:53:09.549 INFO:teuthology.orchestra.run.trial085.stdout:Package python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:53:09.568 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:53:09.568 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-20T23:53:09.568 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:53:09.590 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install python3-cephfs 2026-02-20T23:53:09.895 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:09:09 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:53:09.959 INFO:teuthology.orchestra.run.trial085.stdout:Package python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:53:09.977 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:53:09.978 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-20T23:53:09.978 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:53:09.998 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install python3-rbd 2026-02-20T23:53:10.301 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:09:10 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:53:10.365 INFO:teuthology.orchestra.run.trial085.stdout:Package python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:53:10.383 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:53:10.384 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-20T23:53:10.384 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:53:10.404 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install rbd-fuse 2026-02-20T23:53:10.715 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:09:10 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:53:10.798 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:53:10.798 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:10.798 INFO:teuthology.orchestra.run.trial085.stdout: Package Architecture Version Repository Size 2026-02-20T23:53:10.798 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:10.798 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:53:10.798 INFO:teuthology.orchestra.run.trial085.stdout: rbd-fuse x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 86 k 2026-02-20T23:53:10.799 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:10.799 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:53:10.799 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:10.799 INFO:teuthology.orchestra.run.trial085.stdout:Install 1 Package 2026-02-20T23:53:10.799 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:10.799 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 86 k 2026-02-20T23:53:10.799 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 236 k 2026-02-20T23:53:10.799 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:53:11.428 INFO:teuthology.orchestra.run.trial085.stdout:rbd-fuse-20.3.0-5343.gbcdd429b.el9.x86_64.rpm 137 kB/s | 86 kB 00:00 2026-02-20T23:53:11.428 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:53:11.428 INFO:teuthology.orchestra.run.trial085.stdout:Total 137 kB/s | 86 kB 00:00 2026-02-20T23:53:11.429 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:53:11.433 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:53:11.433 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:53:11.463 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:53:11.463 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:53:11.514 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:53:11.530 INFO:teuthology.orchestra.run.trial085.stdout: Installing : rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:53:11.612 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:53:11.671 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:53:11.671 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:11.671 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-20T23:53:11.671 INFO:teuthology.orchestra.run.trial085.stdout: rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:53:11.671 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:11.671 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:53:11.700 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install rbd-mirror 2026-02-20T23:53:12.009 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:09:12 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:53:12.092 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:53:12.092 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:12.093 INFO:teuthology.orchestra.run.trial085.stdout: Package Arch Version Repo Size 2026-02-20T23:53:12.093 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:12.093 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:53:12.093 INFO:teuthology.orchestra.run.trial085.stdout: rbd-mirror x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 2.6 M 2026-02-20T23:53:12.093 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:12.093 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:53:12.093 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:12.093 INFO:teuthology.orchestra.run.trial085.stdout:Install 1 Package 2026-02-20T23:53:12.093 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:12.093 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 2.6 M 2026-02-20T23:53:12.094 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 11 M 2026-02-20T23:53:12.094 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:53:18.410 INFO:teuthology.orchestra.run.trial085.stdout:rbd-mirror-20.3.0-5343.gbcdd429b.el9.x86_64.rpm 420 kB/s | 2.6 MB 00:06 2026-02-20T23:53:18.411 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:53:18.412 INFO:teuthology.orchestra.run.trial085.stdout:Total 420 kB/s | 2.6 MB 00:06 2026-02-20T23:53:18.412 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:53:18.417 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:53:18.418 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:53:18.456 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:53:18.456 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:53:18.620 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:53:18.622 INFO:teuthology.orchestra.run.trial085.stdout: Installing : rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:53:18.645 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:53:18.645 INFO:teuthology.orchestra.run.trial085.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:53:18.645 INFO:teuthology.orchestra.run.trial085.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-20T23:53:18.645 INFO:teuthology.orchestra.run.trial085.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T23:53:18.645 INFO:teuthology.orchestra.run.trial085.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T23:53:18.645 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:18.945 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:53:18.945 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:18.945 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-20T23:53:18.945 INFO:teuthology.orchestra.run.trial085.stdout: rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:53:18.945 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:18.945 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:53:18.974 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install rbd-nbd 2026-02-20T23:53:19.290 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:09:19 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:53:19.373 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:53:19.373 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:19.373 INFO:teuthology.orchestra.run.trial085.stdout: Package Architecture Version Repository Size 2026-02-20T23:53:19.373 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:19.373 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:53:19.373 INFO:teuthology.orchestra.run.trial085.stdout: rbd-nbd x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 166 k 2026-02-20T23:53:19.373 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:19.374 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:53:19.374 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:19.374 INFO:teuthology.orchestra.run.trial085.stdout:Install 1 Package 2026-02-20T23:53:19.374 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:19.374 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 166 k 2026-02-20T23:53:19.374 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 496 k 2026-02-20T23:53:19.374 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-20T23:53:20.074 INFO:teuthology.orchestra.run.trial085.stdout:rbd-nbd-20.3.0-5343.gbcdd429b.el9.x86_64.rpm 237 kB/s | 166 kB 00:00 2026-02-20T23:53:20.075 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:53:20.075 INFO:teuthology.orchestra.run.trial085.stdout:Total 237 kB/s | 166 kB 00:00 2026-02-20T23:53:20.075 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-20T23:53:20.079 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-20T23:53:20.079 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-20T23:53:20.110 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-20T23:53:20.110 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-20T23:53:20.166 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-20T23:53:20.182 INFO:teuthology.orchestra.run.trial085.stdout: Installing : rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:53:20.263 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:53:20.322 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:53:20.323 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:20.323 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-20T23:53:20.323 INFO:teuthology.orchestra.run.trial085.stdout: rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:53:20.323 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:20.323 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:53:20.352 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-osd-crimson 2026-02-20T23:53:20.669 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:09:20 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:53:20.735 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:53:20.753 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:53:20.754 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-20T23:53:20.754 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-20T23:53:20.775 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-20T23:53:21.085 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:09:21 ago on Fri 20 Feb 2026 11:44:00 PM UTC. 2026-02-20T23:53:21.169 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-20T23:53:21.169 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:21.169 INFO:teuthology.orchestra.run.trial085.stdout: Package Arch Version Repo Size 2026-02-20T23:53:21.169 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:21.169 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-20T23:53:21.170 INFO:teuthology.orchestra.run.trial085.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 433 M 2026-02-20T23:53:21.170 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:21.170 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-20T23:53:21.170 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-20T23:53:21.170 INFO:teuthology.orchestra.run.trial085.stdout:Install 1 Package 2026-02-20T23:53:21.170 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-20T23:53:21.170 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 433 M 2026-02-20T23:53:21.170 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 2.6 G 2026-02-20T23:53:21.170 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-21T00:03:20.137 INFO:teuthology.orchestra.run.trial085.stdout:ceph-osd-crimson-debuginfo-20.3.0-5343.gbcdd429 741 kB/s | 433 MB 09:58 2026-02-21T00:03:20.138 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:03:20.138 INFO:teuthology.orchestra.run.trial085.stdout:Total 741 kB/s | 433 MB 09:58 2026-02-21T00:03:20.138 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-21T00:03:20.140 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-21T00:03:20.140 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-21T00:03:21.440 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-21T00:03:21.441 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-21T00:03:45.586 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-21T00:03:45.602 INFO:teuthology.orchestra.run.trial085.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5343.gbcdd429b.e 1/1 2026-02-21T00:03:46.467 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5343.gbcdd429b.e 1/1 2026-02-21T00:03:46.525 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5343.gbcdd429b.e 1/1 2026-02-21T00:03:46.525 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:03:46.525 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-21T00:03:46.525 INFO:teuthology.orchestra.run.trial085.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:03:46.525 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:03:46.525 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:03:46.598 DEBUG:teuthology.parallel:result is None 2026-02-21T00:03:46.598 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-21T00:03:46.767 DEBUG:teuthology.orchestra.run.trial085:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-21T00:03:46.790 INFO:teuthology.orchestra.run.trial085.stdout:20.3.0-5343.gbcdd429b.el9 2026-02-21T00:03:46.791 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5343.gbcdd429b.el9 2026-02-21T00:03:46.791 INFO:teuthology.task.install:The correct ceph version 20.3.0-5343.gbcdd429b is installed. 2026-02-21T00:03:46.795 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-21T00:03:46.795 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:03:46.795 DEBUG:teuthology.orchestra.run.trial085:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-21T00:03:46.857 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-21T00:03:46.858 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:03:46.858 DEBUG:teuthology.orchestra.run.trial085:> sudo dd of=/usr/bin/daemon-helper 2026-02-21T00:03:46.921 DEBUG:teuthology.orchestra.run.trial085:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-21T00:03:46.986 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-21T00:03:46.986 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:03:46.986 DEBUG:teuthology.orchestra.run.trial085:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-21T00:03:47.050 DEBUG:teuthology.orchestra.run.trial085:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-21T00:03:47.114 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-21T00:03:47.115 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:03:47.115 DEBUG:teuthology.orchestra.run.trial085:> sudo dd of=/usr/bin/stdin-killer 2026-02-21T00:03:47.177 DEBUG:teuthology.orchestra.run.trial085:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-21T00:03:47.243 INFO:teuthology.run_tasks:Running task pexec... 2026-02-21T00:03:47.255 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-21T00:03:47.255 DEBUG:teuthology.orchestra.run.trial085:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-21T00:03:47.286 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial085.front.sepia.ceph.com 2026-02-21T00:03:47.286 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-21T00:03:47.336 DEBUG:teuthology.parallel:result is None 2026-02-21T00:03:47.337 INFO:teuthology.run_tasks:Running task install... 2026-02-21T00:03:47.348 DEBUG:teuthology.task.install:project ceph 2026-02-21T00:03:47.349 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}} 2026-02-21T00:03:47.349 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'} 2026-02-21T00:03:47.349 INFO:teuthology.task.install:Using flavor: debug 2026-02-21T00:03:47.356 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-21T00:03:47.357 INFO:teuthology.task.install:extra packages: [] 2026-02-21T00:03:47.357 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'tag': None, 'wait_for_package': False} 2026-02-21T00:03:47.357 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-21T00:03:47.511 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/ 2026-02-21T00:03:47.511 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5343.gbcdd429b 2026-02-21T00:03:47.876 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-21T00:03:47.876 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:03:47.876 DEBUG:teuthology.orchestra.run.trial085:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-21T00:03:47.900 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-21T00:03:47.901 DEBUG:teuthology.orchestra.run.trial085:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-21T00:03:47.967 DEBUG:teuthology.orchestra.run.trial085:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-21T00:03:48.033 DEBUG:teuthology.orchestra.run.trial085:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-21T00:03:48.088 INFO:teuthology.orchestra.run.trial085.stdout:check_obsoletes = 1 2026-02-21T00:03:48.098 DEBUG:teuthology.orchestra.run.trial085:> sudo yum clean all 2026-02-21T00:03:48.259 INFO:teuthology.orchestra.run.trial085.stdout:56 files removed 2026-02-21T00:03:48.274 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-radosgw 2026-02-21T00:03:49.211 INFO:teuthology.orchestra.run.trial085.stdout:ceph packages for x86_64 117 kB/s | 92 kB 00:00 2026-02-21T00:03:49.870 INFO:teuthology.orchestra.run.trial085.stdout:ceph noarch packages 37 kB/s | 24 kB 00:00 2026-02-21T00:03:50.465 INFO:teuthology.orchestra.run.trial085.stdout:ceph source packages 3.6 kB/s | 2.1 kB 00:00 2026-02-21T00:03:51.515 INFO:teuthology.orchestra.run.trial085.stdout:CentOS Stream 9 - BaseOS 8.6 MB/s | 8.9 MB 00:01 2026-02-21T00:04:11.924 INFO:teuthology.orchestra.run.trial085.stdout:CentOS Stream 9 - AppStream 1.3 MB/s | 27 MB 00:19 2026-02-21T00:04:15.277 INFO:teuthology.orchestra.run.trial085.stdout:CentOS Stream 9 - CRB 7.7 MB/s | 7.8 MB 00:01 2026-02-21T00:04:16.379 INFO:teuthology.orchestra.run.trial085.stdout:CentOS Stream 9 - Extras packages 53 kB/s | 20 kB 00:00 2026-02-21T00:04:17.061 INFO:teuthology.orchestra.run.trial085.stdout:Extra Packages for Enterprise Linux 33 MB/s | 20 MB 00:00 2026-02-21T00:04:20.255 INFO:teuthology.orchestra.run.trial085.stdout:lab-extras 2.8 MB/s | 50 kB 00:00 2026-02-21T00:04:21.429 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:21.448 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:21.448 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:21.449 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:21.473 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-test 2026-02-21T00:04:21.782 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:01 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:21.846 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:21.864 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:21.865 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:21.865 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:21.886 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph 2026-02-21T00:04:22.196 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:02 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:22.261 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:22.280 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:22.281 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:22.281 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:22.303 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-base 2026-02-21T00:04:22.618 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:02 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:22.682 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:22.700 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:22.701 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:22.701 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:22.722 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install cephadm 2026-02-21T00:04:23.032 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:03 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:23.096 INFO:teuthology.orchestra.run.trial085.stdout:Package cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:04:23.114 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:23.115 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:23.115 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:23.137 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-immutable-object-cache 2026-02-21T00:04:23.448 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:03 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:23.513 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:23.532 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:23.532 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:23.532 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:23.554 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-mgr 2026-02-21T00:04:23.866 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:03 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:23.930 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:23.948 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:23.949 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:23.949 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:23.970 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-mgr-dashboard 2026-02-21T00:04:24.281 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:04 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:24.347 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:04:24.365 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:24.366 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:24.366 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:24.387 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-21T00:04:24.700 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:04 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:24.765 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:04:24.784 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:24.784 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:24.784 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:24.805 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-mgr-rook 2026-02-21T00:04:25.110 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:05 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:25.176 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:04:25.195 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:25.195 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:25.196 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:25.217 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-mgr-cephadm 2026-02-21T00:04:25.528 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:05 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:25.593 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:04:25.611 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:25.612 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:25.612 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:25.633 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-osd 2026-02-21T00:04:25.947 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:05 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:26.011 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:26.030 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:26.030 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:26.030 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:26.052 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-osd-classic 2026-02-21T00:04:26.355 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:06 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:26.420 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:26.438 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:26.438 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:26.439 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:26.460 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-fuse 2026-02-21T00:04:26.773 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:06 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:26.838 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:26.856 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:26.857 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:26.857 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:26.878 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install ceph-volume 2026-02-21T00:04:27.190 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:07 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:27.255 INFO:teuthology.orchestra.run.trial085.stdout:Package ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:04:27.274 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:27.274 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:27.274 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:27.296 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install librados-devel 2026-02-21T00:04:27.608 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:07 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:27.673 INFO:teuthology.orchestra.run.trial085.stdout:Package librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:27.691 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:27.691 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:27.692 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:27.713 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install libcephfs2 2026-02-21T00:04:28.025 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:08 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:28.090 INFO:teuthology.orchestra.run.trial085.stdout:Package libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:28.109 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:28.109 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:28.110 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:28.131 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install libcephfs-devel 2026-02-21T00:04:28.438 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:08 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:28.502 INFO:teuthology.orchestra.run.trial085.stdout:Package libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:28.521 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:28.521 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:28.521 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:28.542 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install librados2 2026-02-21T00:04:28.854 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:08 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:28.918 INFO:teuthology.orchestra.run.trial085.stdout:Package librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:28.936 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:28.937 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:28.937 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:28.958 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install librbd1 2026-02-21T00:04:29.271 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:09 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:29.335 INFO:teuthology.orchestra.run.trial085.stdout:Package librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:29.353 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:29.354 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:29.354 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:29.375 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install python3-rados 2026-02-21T00:04:29.684 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:09 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:29.751 INFO:teuthology.orchestra.run.trial085.stdout:Package python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:29.769 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:29.770 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:29.770 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:29.791 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install python3-rgw 2026-02-21T00:04:30.098 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:10 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:30.163 INFO:teuthology.orchestra.run.trial085.stdout:Package python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:30.181 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:30.182 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:30.182 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:30.203 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install python3-cephfs 2026-02-21T00:04:30.511 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:10 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:30.576 INFO:teuthology.orchestra.run.trial085.stdout:Package python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:30.594 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:30.595 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:30.595 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:30.617 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install python3-rbd 2026-02-21T00:04:30.923 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:10 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:30.987 INFO:teuthology.orchestra.run.trial085.stdout:Package python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:31.005 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:31.006 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:31.006 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:31.027 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install rbd-fuse 2026-02-21T00:04:31.333 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:11 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:31.398 INFO:teuthology.orchestra.run.trial085.stdout:Package rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:31.416 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:31.417 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:31.417 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:31.438 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install rbd-mirror 2026-02-21T00:04:31.751 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:11 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:31.816 INFO:teuthology.orchestra.run.trial085.stdout:Package rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:31.834 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:31.835 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:31.835 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:31.857 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install rbd-nbd 2026-02-21T00:04:32.166 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:12 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:04:32.229 INFO:teuthology.orchestra.run.trial085.stdout:Package rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:04:32.247 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:04:32.248 INFO:teuthology.orchestra.run.trial085.stdout:Nothing to do. 2026-02-21T00:04:32.249 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:04:32.273 DEBUG:teuthology.parallel:result is None 2026-02-21T00:04:32.273 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-21T00:04:32.438 DEBUG:teuthology.orchestra.run.trial085:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-21T00:04:32.459 INFO:teuthology.orchestra.run.trial085.stdout:20.3.0-5343.gbcdd429b.el9 2026-02-21T00:04:32.459 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5343.gbcdd429b.el9 2026-02-21T00:04:32.459 INFO:teuthology.task.install:The correct ceph version 20.3.0-5343.gbcdd429b is installed. 2026-02-21T00:04:32.462 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-21T00:04:32.462 INFO:teuthology.run_tasks:Running task ceph... 2026-02-21T00:04:32.572 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-21T00:04:32.572 DEBUG:teuthology.orchestra.run.trial085:> sudo chmod 777 /var/log/ceph 2026-02-21T00:04:32.598 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-21T00:04:32.598 DEBUG:teuthology.orchestra.run.trial085:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-21T00:04:32.661 INFO:tasks.ceph:Creating extra log directories... 2026-02-21T00:04:32.661 DEBUG:teuthology.orchestra.run.trial085:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-21T00:04:32.739 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-21T00:04:32.739 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-21T00:04:32.740 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61901', 'branch': 'wip-liucm-osd-shard-redesign-crimson-only', 'description': 'crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} perf_ignorelist settings/optimized workloads/fio_4M_rand_write}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '61901', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-liucm-osd-shard-redesign-crimson-only'}, 'ceph': {'conf': {'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}}, 'workunit': {'branch': 'wip-liucm-osd-shard-redesign-crimson-only', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}}, 'owner': 'scheduled_liucm@soko04.front.sepia.ceph.com', 'priority': 120, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 6283, 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'wip-liucm-osd-shard-redesign-crimson-only', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'targets': {'trial085.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBG3p1vOMUy5kD/Fxh6GyJJgm54KsYm8wNNX282hwUf+3YqpBmyRhF00BiijjZT4XZl+6ZAd+Ztb5IkZDIlMpjjQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'wait-for-scrub': False, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'cluster': 'ceph'}}, {'ssh_keys': None}, {'cbt': {'benchmarks': {'librbdfio': {'iodepth': [32], 'log_avg_msec': 100, 'mode': ['randwrite'], 'norandommap': True, 'op_size': [4194304], 'osd_ra': [4096], 'pool_profile': 'rbd', 'procs_per_volume': [1], 'time': 60, 'vol_size': 4096, 'volumes_per_client': [2]}}, 'cluster': {'iterations': 1, 'osds_per_node': 3, 'pool_profiles': {'rbd': {'pg_size': 128, 'pgp_size': 128, 'replication': 3}}, 'user': 'ubuntu'}, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-20_22:16:48', 'tube': 'trial', 'user': 'liucm', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699'} 2026-02-21T00:04:32.741 DEBUG:teuthology.orchestra.run.trial085:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-21T00:04:32.786 DEBUG:teuthology.orchestra.run.trial085:> sudo install -d -m0777 -- /var/run/ceph 2026-02-21T00:04:32.850 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:04:32.850 DEBUG:teuthology.orchestra.run.trial085:> dd if=/scratch_devs of=/dev/stdout 2026-02-21T00:04:32.907 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-21T00:04:32.907 DEBUG:teuthology.orchestra.run.trial085:> stat /dev/vg_nvme/lv_1 2026-02-21T00:04:32.963 INFO:teuthology.orchestra.run.trial085.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-21T00:04:32.963 INFO:teuthology.orchestra.run.trial085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T00:04:32.964 INFO:teuthology.orchestra.run.trial085.stdout:Device: 6h/6d Inode: 909 Links: 1 2026-02-21T00:04:32.964 INFO:teuthology.orchestra.run.trial085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T00:04:32.964 INFO:teuthology.orchestra.run.trial085.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T00:04:32.964 INFO:teuthology.orchestra.run.trial085.stdout:Access: 2026-02-20 23:53:18.789622163 +0000 2026-02-21T00:04:32.964 INFO:teuthology.orchestra.run.trial085.stdout:Modify: 2026-02-20 23:43:10.527209904 +0000 2026-02-21T00:04:32.964 INFO:teuthology.orchestra.run.trial085.stdout:Change: 2026-02-20 23:43:10.527209904 +0000 2026-02-21T00:04:32.964 INFO:teuthology.orchestra.run.trial085.stdout: Birth: 2026-02-20 23:43:10.527209904 +0000 2026-02-21T00:04:32.965 DEBUG:teuthology.orchestra.run.trial085:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-21T00:04:33.028 INFO:teuthology.orchestra.run.trial085.stderr:1+0 records in 2026-02-21T00:04:33.029 INFO:teuthology.orchestra.run.trial085.stderr:1+0 records out 2026-02-21T00:04:33.029 INFO:teuthology.orchestra.run.trial085.stderr:512 bytes copied, 0.000152724 s, 3.4 MB/s 2026-02-21T00:04:33.030 DEBUG:teuthology.orchestra.run.trial085:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-21T00:04:33.087 DEBUG:teuthology.orchestra.run.trial085:> stat /dev/vg_nvme/lv_2 2026-02-21T00:04:33.144 INFO:teuthology.orchestra.run.trial085.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-21T00:04:33.144 INFO:teuthology.orchestra.run.trial085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T00:04:33.144 INFO:teuthology.orchestra.run.trial085.stdout:Device: 6h/6d Inode: 935 Links: 1 2026-02-21T00:04:33.144 INFO:teuthology.orchestra.run.trial085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T00:04:33.144 INFO:teuthology.orchestra.run.trial085.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T00:04:33.144 INFO:teuthology.orchestra.run.trial085.stdout:Access: 2026-02-20 23:53:18.790622136 +0000 2026-02-21T00:04:33.144 INFO:teuthology.orchestra.run.trial085.stdout:Modify: 2026-02-20 23:43:10.754201945 +0000 2026-02-21T00:04:33.145 INFO:teuthology.orchestra.run.trial085.stdout:Change: 2026-02-20 23:43:10.754201945 +0000 2026-02-21T00:04:33.145 INFO:teuthology.orchestra.run.trial085.stdout: Birth: 2026-02-20 23:43:10.754201945 +0000 2026-02-21T00:04:33.145 DEBUG:teuthology.orchestra.run.trial085:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-21T00:04:33.206 INFO:teuthology.orchestra.run.trial085.stderr:1+0 records in 2026-02-21T00:04:33.206 INFO:teuthology.orchestra.run.trial085.stderr:1+0 records out 2026-02-21T00:04:33.206 INFO:teuthology.orchestra.run.trial085.stderr:512 bytes copied, 0.000144228 s, 3.5 MB/s 2026-02-21T00:04:33.207 DEBUG:teuthology.orchestra.run.trial085:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-21T00:04:33.264 DEBUG:teuthology.orchestra.run.trial085:> stat /dev/vg_nvme/lv_3 2026-02-21T00:04:33.320 INFO:teuthology.orchestra.run.trial085.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-21T00:04:33.320 INFO:teuthology.orchestra.run.trial085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T00:04:33.320 INFO:teuthology.orchestra.run.trial085.stdout:Device: 6h/6d Inode: 965 Links: 1 2026-02-21T00:04:33.320 INFO:teuthology.orchestra.run.trial085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T00:04:33.320 INFO:teuthology.orchestra.run.trial085.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T00:04:33.320 INFO:teuthology.orchestra.run.trial085.stdout:Access: 2026-02-20 23:53:18.790622136 +0000 2026-02-21T00:04:33.320 INFO:teuthology.orchestra.run.trial085.stdout:Modify: 2026-02-20 23:43:10.979194057 +0000 2026-02-21T00:04:33.320 INFO:teuthology.orchestra.run.trial085.stdout:Change: 2026-02-20 23:43:10.979194057 +0000 2026-02-21T00:04:33.321 INFO:teuthology.orchestra.run.trial085.stdout: Birth: 2026-02-20 23:43:10.979194057 +0000 2026-02-21T00:04:33.321 DEBUG:teuthology.orchestra.run.trial085:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-21T00:04:33.382 INFO:teuthology.orchestra.run.trial085.stderr:1+0 records in 2026-02-21T00:04:33.383 INFO:teuthology.orchestra.run.trial085.stderr:1+0 records out 2026-02-21T00:04:33.383 INFO:teuthology.orchestra.run.trial085.stderr:512 bytes copied, 0.000153646 s, 3.3 MB/s 2026-02-21T00:04:33.384 DEBUG:teuthology.orchestra.run.trial085:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-21T00:04:33.441 DEBUG:teuthology.orchestra.run.trial085:> stat /dev/vg_nvme/lv_4 2026-02-21T00:04:33.499 INFO:teuthology.orchestra.run.trial085.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-21T00:04:33.499 INFO:teuthology.orchestra.run.trial085.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T00:04:33.500 INFO:teuthology.orchestra.run.trial085.stdout:Device: 6h/6d Inode: 993 Links: 1 2026-02-21T00:04:33.500 INFO:teuthology.orchestra.run.trial085.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T00:04:33.500 INFO:teuthology.orchestra.run.trial085.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T00:04:33.500 INFO:teuthology.orchestra.run.trial085.stdout:Access: 2026-02-20 23:53:18.790622136 +0000 2026-02-21T00:04:33.500 INFO:teuthology.orchestra.run.trial085.stdout:Modify: 2026-02-20 23:43:11.211185923 +0000 2026-02-21T00:04:33.500 INFO:teuthology.orchestra.run.trial085.stdout:Change: 2026-02-20 23:43:11.211185923 +0000 2026-02-21T00:04:33.500 INFO:teuthology.orchestra.run.trial085.stdout: Birth: 2026-02-20 23:43:11.211185923 +0000 2026-02-21T00:04:33.500 DEBUG:teuthology.orchestra.run.trial085:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-21T00:04:33.563 INFO:teuthology.orchestra.run.trial085.stderr:1+0 records in 2026-02-21T00:04:33.564 INFO:teuthology.orchestra.run.trial085.stderr:1+0 records out 2026-02-21T00:04:33.564 INFO:teuthology.orchestra.run.trial085.stderr:512 bytes copied, 0.00014539 s, 3.5 MB/s 2026-02-21T00:04:33.565 DEBUG:teuthology.orchestra.run.trial085:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-21T00:04:33.622 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T00:04:33.622 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial085.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-21T00:04:33.622 INFO:tasks.ceph:Generating config... 2026-02-21T00:04:33.625 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-21T00:04:33.625 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-21T00:04:33.625 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-21T00:04:33.625 INFO:tasks.ceph:[osd] debug filestore = 0/0 2026-02-21T00:04:33.626 INFO:tasks.ceph:[osd] debug journal = 0/0 2026-02-21T00:04:33.626 INFO:tasks.ceph:[osd] debug ms = 0/0 2026-02-21T00:04:33.626 INFO:tasks.ceph:[osd] debug osd = 0/0 2026-02-21T00:04:33.626 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-21T00:04:33.626 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-21T00:04:33.626 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-21T00:04:33.626 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-21T00:04:33.626 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-21T00:04:33.626 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-21T00:04:33.626 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-21T00:04:33.626 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-21T00:04:33.626 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-21T00:04:33.627 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-21T00:04:33.627 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-21T00:04:33.627 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-21T00:04:33.627 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-21T00:04:33.627 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-21T00:04:33.627 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-21T00:04:33.627 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-21T00:04:33.627 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-21T00:04:33.627 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-21T00:04:33.627 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-21T00:04:33.627 INFO:tasks.ceph:[global] auth client required = none 2026-02-21T00:04:33.627 INFO:tasks.ceph:[global] auth cluster required = none 2026-02-21T00:04:33.628 INFO:tasks.ceph:[global] auth service required = none 2026-02-21T00:04:33.628 INFO:tasks.ceph:[global] auth supported = none 2026-02-21T00:04:33.628 INFO:tasks.ceph:[global] debug asok = 0/0 2026-02-21T00:04:33.628 INFO:tasks.ceph:[global] debug auth = 0/0 2026-02-21T00:04:33.628 INFO:tasks.ceph:[global] debug bdev = 0/0 2026-02-21T00:04:33.628 INFO:tasks.ceph:[global] debug bluefs = 0/0 2026-02-21T00:04:33.628 INFO:tasks.ceph:[global] debug bluestore = 0/0 2026-02-21T00:04:33.628 INFO:tasks.ceph:[global] debug buffer = 0/0 2026-02-21T00:04:33.628 INFO:tasks.ceph:[global] debug civetweb = 0/0 2026-02-21T00:04:33.628 INFO:tasks.ceph:[global] debug client = 0/0 2026-02-21T00:04:33.628 INFO:tasks.ceph:[global] debug compressor = 0/0 2026-02-21T00:04:33.628 INFO:tasks.ceph:[global] debug context = 0/0 2026-02-21T00:04:33.629 INFO:tasks.ceph:[global] debug crush = 0/0 2026-02-21T00:04:33.629 INFO:tasks.ceph:[global] debug crypto = 0/0 2026-02-21T00:04:33.629 INFO:tasks.ceph:[global] debug dpdk = 0/0 2026-02-21T00:04:33.629 INFO:tasks.ceph:[global] debug eventtrace = 0/0 2026-02-21T00:04:33.629 INFO:tasks.ceph:[global] debug filer = 0/0 2026-02-21T00:04:33.629 INFO:tasks.ceph:[global] debug filestore = 0/0 2026-02-21T00:04:33.629 INFO:tasks.ceph:[global] debug finisher = 0/0 2026-02-21T00:04:33.629 INFO:tasks.ceph:[global] debug fuse = 0/0 2026-02-21T00:04:33.629 INFO:tasks.ceph:[global] debug heartbeatmap = 0/0 2026-02-21T00:04:33.629 INFO:tasks.ceph:[global] debug javaclient = 0/0 2026-02-21T00:04:33.629 INFO:tasks.ceph:[global] debug journal = 0/0 2026-02-21T00:04:33.629 INFO:tasks.ceph:[global] debug journaler = 0/0 2026-02-21T00:04:33.629 INFO:tasks.ceph:[global] debug kstore = 0/0 2026-02-21T00:04:33.630 INFO:tasks.ceph:[global] debug leveldb = 0/0 2026-02-21T00:04:33.630 INFO:tasks.ceph:[global] debug lockdep = 0/0 2026-02-21T00:04:33.630 INFO:tasks.ceph:[global] debug mds = 0/0 2026-02-21T00:04:33.630 INFO:tasks.ceph:[global] debug mds balancer = 0/0 2026-02-21T00:04:33.630 INFO:tasks.ceph:[global] debug mds locker = 0/0 2026-02-21T00:04:33.630 INFO:tasks.ceph:[global] debug mds log = 0/0 2026-02-21T00:04:33.630 INFO:tasks.ceph:[global] debug mds log expire = 0/0 2026-02-21T00:04:33.630 INFO:tasks.ceph:[global] debug mds migrator = 0/0 2026-02-21T00:04:33.630 INFO:tasks.ceph:[global] debug memdb = 0/0 2026-02-21T00:04:33.630 INFO:tasks.ceph:[global] debug mgr = 0/0 2026-02-21T00:04:33.630 INFO:tasks.ceph:[global] debug mgrc = 0/0 2026-02-21T00:04:33.630 INFO:tasks.ceph:[global] debug mon = 0/0 2026-02-21T00:04:33.630 INFO:tasks.ceph:[global] debug monc = 0/0 2026-02-21T00:04:33.631 INFO:tasks.ceph:[global] debug ms = 0/0 2026-02-21T00:04:33.631 INFO:tasks.ceph:[global] debug objclass = 0/0 2026-02-21T00:04:33.631 INFO:tasks.ceph:[global] debug objectcacher = 0/0 2026-02-21T00:04:33.631 INFO:tasks.ceph:[global] debug objecter = 0/0 2026-02-21T00:04:33.631 INFO:tasks.ceph:[global] debug optracker = 0/0 2026-02-21T00:04:33.631 INFO:tasks.ceph:[global] debug osd = 0/0 2026-02-21T00:04:33.631 INFO:tasks.ceph:[global] debug paxos = 0/0 2026-02-21T00:04:33.631 INFO:tasks.ceph:[global] debug perfcounter = 0/0 2026-02-21T00:04:33.631 INFO:tasks.ceph:[global] debug rados = 0/0 2026-02-21T00:04:33.631 INFO:tasks.ceph:[global] debug rbd = 0/0 2026-02-21T00:04:33.631 INFO:tasks.ceph:[global] debug rbd mirror = 0/0 2026-02-21T00:04:33.631 INFO:tasks.ceph:[global] debug rbd replay = 0/0 2026-02-21T00:04:33.631 INFO:tasks.ceph:[global] debug refs = 0/0 2026-02-21T00:04:33.632 INFO:tasks.ceph:[global] debug rgw = 0/0 2026-02-21T00:04:33.632 INFO:tasks.ceph:[global] debug rgw lifecycle = 0/0 2026-02-21T00:04:33.632 INFO:tasks.ceph:[global] debug rgw notification = 0/0 2026-02-21T00:04:33.632 INFO:tasks.ceph:[global] debug rgw sync = 0/0 2026-02-21T00:04:33.632 INFO:tasks.ceph:[global] debug rocksdb = 0/0 2026-02-21T00:04:33.632 INFO:tasks.ceph:[global] debug striper = 0/0 2026-02-21T00:04:33.632 INFO:tasks.ceph:[global] debug throttle = 0/0 2026-02-21T00:04:33.632 INFO:tasks.ceph:[global] debug timer = 0/0 2026-02-21T00:04:33.632 INFO:tasks.ceph:[global] debug tp = 0/0 2026-02-21T00:04:33.632 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-21T00:04:33.632 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-21T00:04:33.632 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-21T00:04:33.632 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-21T00:04:33.633 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-21T00:04:33.633 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-21T00:04:33.633 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-21T00:04:33.633 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-21T00:04:33.633 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-21T00:04:33.633 INFO:tasks.ceph:[mon] debug mon = 0/0 2026-02-21T00:04:33.633 INFO:tasks.ceph:[mon] debug ms = 0/0 2026-02-21T00:04:33.633 INFO:tasks.ceph:[mon] debug paxos = 0/0 2026-02-21T00:04:33.633 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-21T00:04:33.633 INFO:tasks.ceph:Setting up mon.a... 2026-02-21T00:04:33.634 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-21T00:04:33.703 INFO:teuthology.orchestra.run.trial085.stdout:creating /etc/ceph/ceph.keyring 2026-02-21T00:04:33.706 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-21T00:04:33.789 DEBUG:teuthology.orchestra.run.trial085:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T00:04:33.855 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.85')] 2026-02-21T00:04:33.856 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'none', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.85', 'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': '0/0', 'debug mon': '0/0', 'debug paxos': '0/0', 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-21T00:04:33.858 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:04:33.858 DEBUG:teuthology.orchestra.run.trial085:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-21T00:04:33.911 DEBUG:teuthology.orchestra.run.trial085:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.85 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:04:33.992 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:33.990+0000 7f87738e0f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:33.993 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:33.990+0000 7f87738e0f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:33.993 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:33.990+0000 7f87738e0f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:33.994 INFO:teuthology.orchestra.run.trial085.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:04:33.994 INFO:teuthology.orchestra.run.trial085.stdout:monmaptool: generated fsid e5b08652-3d06-4075-8c52-d52bf7e93d0a 2026-02-21T00:04:33.994 INFO:teuthology.orchestra.run.trial085.stdout:setting min_mon_release = tentacle 2026-02-21T00:04:33.994 INFO:teuthology.orchestra.run.trial085.stdout:epoch 0 2026-02-21T00:04:33.994 INFO:teuthology.orchestra.run.trial085.stdout:fsid e5b08652-3d06-4075-8c52-d52bf7e93d0a 2026-02-21T00:04:33.995 INFO:teuthology.orchestra.run.trial085.stdout:last_changed 2026-02-21T00:04:33.991339+0000 2026-02-21T00:04:33.995 INFO:teuthology.orchestra.run.trial085.stdout:created 2026-02-21T00:04:33.991339+0000 2026-02-21T00:04:33.995 INFO:teuthology.orchestra.run.trial085.stdout:min_mon_release 20 (tentacle) 2026-02-21T00:04:33.995 INFO:teuthology.orchestra.run.trial085.stdout:election_strategy: 1 2026-02-21T00:04:33.995 INFO:teuthology.orchestra.run.trial085.stdout:0: [v2:10.20.193.85:3300/0,v1:10.20.193.85:6789/0] mon.a 2026-02-21T00:04:33.995 INFO:teuthology.orchestra.run.trial085.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-21T00:04:33.996 DEBUG:teuthology.orchestra.run.trial085:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-21T00:04:34.051 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID e5b08652-3d06-4075-8c52-d52bf7e93d0a... 2026-02-21T00:04:34.052 DEBUG:teuthology.orchestra.run.trial085:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-21T00:04:34.132 INFO:teuthology.orchestra.run.trial085.stdout:[global] 2026-02-21T00:04:34.132 INFO:teuthology.orchestra.run.trial085.stdout: chdir = "" 2026-02-21T00:04:34.132 INFO:teuthology.orchestra.run.trial085.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-21T00:04:34.132 INFO:teuthology.orchestra.run.trial085.stdout: auth supported = none 2026-02-21T00:04:34.132 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.132 INFO:teuthology.orchestra.run.trial085.stdout: filestore xattr use omap = true 2026-02-21T00:04:34.132 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.133 INFO:teuthology.orchestra.run.trial085.stdout: mon clock drift allowed = 1.000 2026-02-21T00:04:34.133 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.133 INFO:teuthology.orchestra.run.trial085.stdout: osd crush chooseleaf type = 0 2026-02-21T00:04:34.133 INFO:teuthology.orchestra.run.trial085.stdout: auth debug = true 2026-02-21T00:04:34.133 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.133 INFO:teuthology.orchestra.run.trial085.stdout: ms die on old message = true 2026-02-21T00:04:34.133 INFO:teuthology.orchestra.run.trial085.stdout: ms die on bug = true 2026-02-21T00:04:34.133 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.133 INFO:teuthology.orchestra.run.trial085.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-21T00:04:34.133 INFO:teuthology.orchestra.run.trial085.stdout: mon pg warn max object skew = 0 2026-02-21T00:04:34.133 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.134 INFO:teuthology.orchestra.run.trial085.stdout: # disable pg_autoscaler by default for new pools 2026-02-21T00:04:34.134 INFO:teuthology.orchestra.run.trial085.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-21T00:04:34.134 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.134 INFO:teuthology.orchestra.run.trial085.stdout: osd pool default size = 2 2026-02-21T00:04:34.134 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.134 INFO:teuthology.orchestra.run.trial085.stdout: mon osd allow primary affinity = true 2026-02-21T00:04:34.134 INFO:teuthology.orchestra.run.trial085.stdout: mon osd allow pg remap = true 2026-02-21T00:04:34.134 INFO:teuthology.orchestra.run.trial085.stdout: mon warn on legacy crush tunables = false 2026-02-21T00:04:34.134 INFO:teuthology.orchestra.run.trial085.stdout: mon warn on crush straw calc version zero = false 2026-02-21T00:04:34.134 INFO:teuthology.orchestra.run.trial085.stdout: mon warn on no sortbitwise = false 2026-02-21T00:04:34.134 INFO:teuthology.orchestra.run.trial085.stdout: mon warn on osd down out interval zero = false 2026-02-21T00:04:34.134 INFO:teuthology.orchestra.run.trial085.stdout: mon warn on too few osds = false 2026-02-21T00:04:34.135 INFO:teuthology.orchestra.run.trial085.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-21T00:04:34.135 INFO:teuthology.orchestra.run.trial085.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-21T00:04:34.135 INFO:teuthology.orchestra.run.trial085.stdout: mon_allow_pool_size_one = true 2026-02-21T00:04:34.135 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.135 INFO:teuthology.orchestra.run.trial085.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-21T00:04:34.135 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.135 INFO:teuthology.orchestra.run.trial085.stdout: osd default data pool replay window = 5 2026-02-21T00:04:34.135 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.135 INFO:teuthology.orchestra.run.trial085.stdout: mon allow pool delete = true 2026-02-21T00:04:34.135 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.135 INFO:teuthology.orchestra.run.trial085.stdout: mon cluster log file level = debug 2026-02-21T00:04:34.135 INFO:teuthology.orchestra.run.trial085.stdout: debug asserts on shutdown = true 2026-02-21T00:04:34.135 INFO:teuthology.orchestra.run.trial085.stdout: mon health detail to clog = false 2026-02-21T00:04:34.136 INFO:teuthology.orchestra.run.trial085.stdout: mon host = 10.20.193.85 2026-02-21T00:04:34.136 INFO:teuthology.orchestra.run.trial085.stdout: auth client required = none 2026-02-21T00:04:34.136 INFO:teuthology.orchestra.run.trial085.stdout: auth cluster required = none 2026-02-21T00:04:34.136 INFO:teuthology.orchestra.run.trial085.stdout: auth service required = none 2026-02-21T00:04:34.136 INFO:teuthology.orchestra.run.trial085.stdout: debug asok = 0/0 2026-02-21T00:04:34.136 INFO:teuthology.orchestra.run.trial085.stdout: debug auth = 0/0 2026-02-21T00:04:34.136 INFO:teuthology.orchestra.run.trial085.stdout: debug bdev = 0/0 2026-02-21T00:04:34.136 INFO:teuthology.orchestra.run.trial085.stdout: debug bluefs = 0/0 2026-02-21T00:04:34.136 INFO:teuthology.orchestra.run.trial085.stdout: debug bluestore = 0/0 2026-02-21T00:04:34.136 INFO:teuthology.orchestra.run.trial085.stdout: debug buffer = 0/0 2026-02-21T00:04:34.136 INFO:teuthology.orchestra.run.trial085.stdout: debug civetweb = 0/0 2026-02-21T00:04:34.136 INFO:teuthology.orchestra.run.trial085.stdout: debug client = 0/0 2026-02-21T00:04:34.136 INFO:teuthology.orchestra.run.trial085.stdout: debug compressor = 0/0 2026-02-21T00:04:34.137 INFO:teuthology.orchestra.run.trial085.stdout: debug context = 0/0 2026-02-21T00:04:34.137 INFO:teuthology.orchestra.run.trial085.stdout: debug crush = 0/0 2026-02-21T00:04:34.137 INFO:teuthology.orchestra.run.trial085.stdout: debug crypto = 0/0 2026-02-21T00:04:34.137 INFO:teuthology.orchestra.run.trial085.stdout: debug dpdk = 0/0 2026-02-21T00:04:34.137 INFO:teuthology.orchestra.run.trial085.stdout: debug eventtrace = 0/0 2026-02-21T00:04:34.137 INFO:teuthology.orchestra.run.trial085.stdout: debug filer = 0/0 2026-02-21T00:04:34.137 INFO:teuthology.orchestra.run.trial085.stdout: debug filestore = 0/0 2026-02-21T00:04:34.137 INFO:teuthology.orchestra.run.trial085.stdout: debug finisher = 0/0 2026-02-21T00:04:34.137 INFO:teuthology.orchestra.run.trial085.stdout: debug fuse = 0/0 2026-02-21T00:04:34.137 INFO:teuthology.orchestra.run.trial085.stdout: debug heartbeatmap = 0/0 2026-02-21T00:04:34.137 INFO:teuthology.orchestra.run.trial085.stdout: debug javaclient = 0/0 2026-02-21T00:04:34.137 INFO:teuthology.orchestra.run.trial085.stdout: debug journal = 0/0 2026-02-21T00:04:34.138 INFO:teuthology.orchestra.run.trial085.stdout: debug journaler = 0/0 2026-02-21T00:04:34.138 INFO:teuthology.orchestra.run.trial085.stdout: debug kstore = 0/0 2026-02-21T00:04:34.138 INFO:teuthology.orchestra.run.trial085.stdout: debug leveldb = 0/0 2026-02-21T00:04:34.138 INFO:teuthology.orchestra.run.trial085.stdout: debug lockdep = 0/0 2026-02-21T00:04:34.138 INFO:teuthology.orchestra.run.trial085.stdout: debug mds = 0/0 2026-02-21T00:04:34.138 INFO:teuthology.orchestra.run.trial085.stdout: debug mds balancer = 0/0 2026-02-21T00:04:34.138 INFO:teuthology.orchestra.run.trial085.stdout: debug mds locker = 0/0 2026-02-21T00:04:34.138 INFO:teuthology.orchestra.run.trial085.stdout: debug mds log = 0/0 2026-02-21T00:04:34.138 INFO:teuthology.orchestra.run.trial085.stdout: debug mds log expire = 0/0 2026-02-21T00:04:34.138 INFO:teuthology.orchestra.run.trial085.stdout: debug mds migrator = 0/0 2026-02-21T00:04:34.138 INFO:teuthology.orchestra.run.trial085.stdout: debug memdb = 0/0 2026-02-21T00:04:34.138 INFO:teuthology.orchestra.run.trial085.stdout: debug mgr = 0/0 2026-02-21T00:04:34.138 INFO:teuthology.orchestra.run.trial085.stdout: debug mgrc = 0/0 2026-02-21T00:04:34.138 INFO:teuthology.orchestra.run.trial085.stdout: debug mon = 0/0 2026-02-21T00:04:34.139 INFO:teuthology.orchestra.run.trial085.stdout: debug monc = 0/0 2026-02-21T00:04:34.139 INFO:teuthology.orchestra.run.trial085.stdout: debug ms = 0/0 2026-02-21T00:04:34.139 INFO:teuthology.orchestra.run.trial085.stdout: debug objclass = 0/0 2026-02-21T00:04:34.139 INFO:teuthology.orchestra.run.trial085.stdout: debug objectcacher = 0/0 2026-02-21T00:04:34.139 INFO:teuthology.orchestra.run.trial085.stdout: debug objecter = 0/0 2026-02-21T00:04:34.139 INFO:teuthology.orchestra.run.trial085.stdout: debug optracker = 0/0 2026-02-21T00:04:34.139 INFO:teuthology.orchestra.run.trial085.stdout: debug osd = 0/0 2026-02-21T00:04:34.139 INFO:teuthology.orchestra.run.trial085.stdout: debug paxos = 0/0 2026-02-21T00:04:34.139 INFO:teuthology.orchestra.run.trial085.stdout: debug perfcounter = 0/0 2026-02-21T00:04:34.139 INFO:teuthology.orchestra.run.trial085.stdout: debug rados = 0/0 2026-02-21T00:04:34.139 INFO:teuthology.orchestra.run.trial085.stdout: debug rbd = 0/0 2026-02-21T00:04:34.139 INFO:teuthology.orchestra.run.trial085.stdout: debug rbd mirror = 0/0 2026-02-21T00:04:34.139 INFO:teuthology.orchestra.run.trial085.stdout: debug rbd replay = 0/0 2026-02-21T00:04:34.140 INFO:teuthology.orchestra.run.trial085.stdout: debug refs = 0/0 2026-02-21T00:04:34.140 INFO:teuthology.orchestra.run.trial085.stdout: debug rgw = 0/0 2026-02-21T00:04:34.140 INFO:teuthology.orchestra.run.trial085.stdout: debug rgw lifecycle = 0/0 2026-02-21T00:04:34.140 INFO:teuthology.orchestra.run.trial085.stdout: debug rgw notification = 0/0 2026-02-21T00:04:34.140 INFO:teuthology.orchestra.run.trial085.stdout: debug rgw sync = 0/0 2026-02-21T00:04:34.140 INFO:teuthology.orchestra.run.trial085.stdout: debug rocksdb = 0/0 2026-02-21T00:04:34.140 INFO:teuthology.orchestra.run.trial085.stdout: debug striper = 0/0 2026-02-21T00:04:34.140 INFO:teuthology.orchestra.run.trial085.stdout: debug throttle = 0/0 2026-02-21T00:04:34.140 INFO:teuthology.orchestra.run.trial085.stdout: debug timer = 0/0 2026-02-21T00:04:34.140 INFO:teuthology.orchestra.run.trial085.stdout: debug tp = 0/0 2026-02-21T00:04:34.140 INFO:teuthology.orchestra.run.trial085.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-21T00:04:34.140 INFO:teuthology.orchestra.run.trial085.stdout: ms client mode = crc 2026-02-21T00:04:34.140 INFO:teuthology.orchestra.run.trial085.stdout: ms cluster mode = crc 2026-02-21T00:04:34.141 INFO:teuthology.orchestra.run.trial085.stdout: ms mon client mode = crc 2026-02-21T00:04:34.141 INFO:teuthology.orchestra.run.trial085.stdout: ms mon cluster mode = crc 2026-02-21T00:04:34.141 INFO:teuthology.orchestra.run.trial085.stdout: ms mon service mode = crc 2026-02-21T00:04:34.141 INFO:teuthology.orchestra.run.trial085.stdout: ms service mode = crc 2026-02-21T00:04:34.141 INFO:teuthology.orchestra.run.trial085.stdout: fsid = e5b08652-3d06-4075-8c52-d52bf7e93d0a 2026-02-21T00:04:34.141 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.141 INFO:teuthology.orchestra.run.trial085.stdout:[osd] 2026-02-21T00:04:34.141 INFO:teuthology.orchestra.run.trial085.stdout: osd journal size = 100 2026-02-21T00:04:34.141 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.141 INFO:teuthology.orchestra.run.trial085.stdout: osd scrub load threshold = 5.0 2026-02-21T00:04:34.141 INFO:teuthology.orchestra.run.trial085.stdout: osd scrub max interval = 600 2026-02-21T00:04:34.141 INFO:teuthology.orchestra.run.trial085.stdout: osd mclock profile = high_recovery_ops 2026-02-21T00:04:34.141 INFO:teuthology.orchestra.run.trial085.stdout: osd mclock skip benchmark = true 2026-02-21T00:04:34.142 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.142 INFO:teuthology.orchestra.run.trial085.stdout: osd recover clone overlap = true 2026-02-21T00:04:34.142 INFO:teuthology.orchestra.run.trial085.stdout: osd recovery max chunk = 1048576 2026-02-21T00:04:34.142 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.142 INFO:teuthology.orchestra.run.trial085.stdout: osd debug shutdown = true 2026-02-21T00:04:34.142 INFO:teuthology.orchestra.run.trial085.stdout: osd debug op order = true 2026-02-21T00:04:34.142 INFO:teuthology.orchestra.run.trial085.stdout: osd debug verify stray on activate = true 2026-02-21T00:04:34.142 INFO:teuthology.orchestra.run.trial085.stdout: osd debug trim objects = true 2026-02-21T00:04:34.142 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.142 INFO:teuthology.orchestra.run.trial085.stdout: osd open classes on start = true 2026-02-21T00:04:34.142 INFO:teuthology.orchestra.run.trial085.stdout: osd debug pg log writeout = true 2026-02-21T00:04:34.142 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.143 INFO:teuthology.orchestra.run.trial085.stdout: osd deep scrub update digest min age = 30 2026-02-21T00:04:34.143 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.143 INFO:teuthology.orchestra.run.trial085.stdout: osd map max advance = 10 2026-02-21T00:04:34.143 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.143 INFO:teuthology.orchestra.run.trial085.stdout: journal zero on create = true 2026-02-21T00:04:34.143 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.143 INFO:teuthology.orchestra.run.trial085.stdout: filestore ondisk finisher threads = 3 2026-02-21T00:04:34.143 INFO:teuthology.orchestra.run.trial085.stdout: filestore apply finisher threads = 3 2026-02-21T00:04:34.143 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.143 INFO:teuthology.orchestra.run.trial085.stdout: bdev debug aio = true 2026-02-21T00:04:34.143 INFO:teuthology.orchestra.run.trial085.stdout: osd debug misdirected ops = true 2026-02-21T00:04:34.143 INFO:teuthology.orchestra.run.trial085.stdout: debug monc = 20 2026-02-21T00:04:34.143 INFO:teuthology.orchestra.run.trial085.stdout: crimson cpu num = 4 2026-02-21T00:04:34.143 INFO:teuthology.orchestra.run.trial085.stdout: crimson osd obc lru size = 10 2026-02-21T00:04:34.144 INFO:teuthology.orchestra.run.trial085.stdout: debug filestore = 0/0 2026-02-21T00:04:34.144 INFO:teuthology.orchestra.run.trial085.stdout: debug journal = 0/0 2026-02-21T00:04:34.144 INFO:teuthology.orchestra.run.trial085.stdout: debug ms = 0/0 2026-02-21T00:04:34.144 INFO:teuthology.orchestra.run.trial085.stdout: debug osd = 0/0 2026-02-21T00:04:34.144 INFO:teuthology.orchestra.run.trial085.stdout: debug seastore = 20 2026-02-21T00:04:34.144 INFO:teuthology.orchestra.run.trial085.stdout: debug seastore backref = 20 2026-02-21T00:04:34.144 INFO:teuthology.orchestra.run.trial085.stdout: debug seastore cache = 20 2026-02-21T00:04:34.144 INFO:teuthology.orchestra.run.trial085.stdout: debug seastore cleaner = 20 2026-02-21T00:04:34.144 INFO:teuthology.orchestra.run.trial085.stdout: debug seastore device = 20 2026-02-21T00:04:34.144 INFO:teuthology.orchestra.run.trial085.stdout: debug seastore epm = 20 2026-02-21T00:04:34.144 INFO:teuthology.orchestra.run.trial085.stdout: debug seastore fixedkv tree = 20 2026-02-21T00:04:34.144 INFO:teuthology.orchestra.run.trial085.stdout: debug seastore journal = 20 2026-02-21T00:04:34.144 INFO:teuthology.orchestra.run.trial085.stdout: debug seastore lba = 20 2026-02-21T00:04:34.145 INFO:teuthology.orchestra.run.trial085.stdout: debug seastore odata = 20 2026-02-21T00:04:34.145 INFO:teuthology.orchestra.run.trial085.stdout: debug seastore omap = 20 2026-02-21T00:04:34.145 INFO:teuthology.orchestra.run.trial085.stdout: debug seastore onode = 20 2026-02-21T00:04:34.145 INFO:teuthology.orchestra.run.trial085.stdout: debug seastore t = 20 2026-02-21T00:04:34.145 INFO:teuthology.orchestra.run.trial085.stdout: debug seastore tm = 20 2026-02-21T00:04:34.145 INFO:teuthology.orchestra.run.trial085.stdout: osd objectstore = seastore 2026-02-21T00:04:34.145 INFO:teuthology.orchestra.run.trial085.stdout: osd shutdown pgref assert = True 2026-02-21T00:04:34.145 INFO:teuthology.orchestra.run.trial085.stdout: seastore cachepin size pershard = 64M 2026-02-21T00:04:34.145 INFO:teuthology.orchestra.run.trial085.stdout: seastore cachepin type = 2Q 2026-02-21T00:04:34.145 INFO:teuthology.orchestra.run.trial085.stdout: seastore max concurrent transactions = 8 2026-02-21T00:04:34.145 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.145 INFO:teuthology.orchestra.run.trial085.stdout:[mgr] 2026-02-21T00:04:34.145 INFO:teuthology.orchestra.run.trial085.stdout: debug ms = 1 2026-02-21T00:04:34.146 INFO:teuthology.orchestra.run.trial085.stdout: debug mgr = 20 2026-02-21T00:04:34.146 INFO:teuthology.orchestra.run.trial085.stdout: debug mon = 20 2026-02-21T00:04:34.146 INFO:teuthology.orchestra.run.trial085.stdout: debug auth = 20 2026-02-21T00:04:34.146 INFO:teuthology.orchestra.run.trial085.stdout: mon reweight min pgs per osd = 4 2026-02-21T00:04:34.146 INFO:teuthology.orchestra.run.trial085.stdout: mon reweight min bytes per osd = 10 2026-02-21T00:04:34.146 INFO:teuthology.orchestra.run.trial085.stdout: mgr/telemetry/nag = false 2026-02-21T00:04:34.146 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.146 INFO:teuthology.orchestra.run.trial085.stdout:[mon] 2026-02-21T00:04:34.146 INFO:teuthology.orchestra.run.trial085.stdout: debug ms = 0/0 2026-02-21T00:04:34.146 INFO:teuthology.orchestra.run.trial085.stdout: debug mon = 0/0 2026-02-21T00:04:34.146 INFO:teuthology.orchestra.run.trial085.stdout: debug paxos = 0/0 2026-02-21T00:04:34.146 INFO:teuthology.orchestra.run.trial085.stdout: debug auth = 20 2026-02-21T00:04:34.147 INFO:teuthology.orchestra.run.trial085.stdout: mon data avail warn = 5 2026-02-21T00:04:34.147 INFO:teuthology.orchestra.run.trial085.stdout: mon mgr mkfs grace = 240 2026-02-21T00:04:34.147 INFO:teuthology.orchestra.run.trial085.stdout: mon reweight min pgs per osd = 4 2026-02-21T00:04:34.147 INFO:teuthology.orchestra.run.trial085.stdout: mon osd reporter subtree level = osd 2026-02-21T00:04:34.147 INFO:teuthology.orchestra.run.trial085.stdout: mon osd prime pg temp = true 2026-02-21T00:04:34.147 INFO:teuthology.orchestra.run.trial085.stdout: mon reweight min bytes per osd = 10 2026-02-21T00:04:34.147 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.147 INFO:teuthology.orchestra.run.trial085.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-21T00:04:34.147 INFO:teuthology.orchestra.run.trial085.stdout: auth mon ticket ttl = 660 # 11m 2026-02-21T00:04:34.147 INFO:teuthology.orchestra.run.trial085.stdout: auth service ticket ttl = 240 # 4m 2026-02-21T00:04:34.147 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.147 INFO:teuthology.orchestra.run.trial085.stdout: # don't complain about insecure global_id in the test suite 2026-02-21T00:04:34.147 INFO:teuthology.orchestra.run.trial085.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-21T00:04:34.148 INFO:teuthology.orchestra.run.trial085.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-21T00:04:34.148 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.148 INFO:teuthology.orchestra.run.trial085.stdout: # 1m isn't quite enough 2026-02-21T00:04:34.148 INFO:teuthology.orchestra.run.trial085.stdout: mon_down_mkfs_grace = 2m 2026-02-21T00:04:34.148 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.148 INFO:teuthology.orchestra.run.trial085.stdout: mon_warn_on_filestore_osds = false 2026-02-21T00:04:34.148 INFO:teuthology.orchestra.run.trial085.stdout: osd pool default crimson = True 2026-02-21T00:04:34.148 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:34.148 INFO:teuthology.orchestra.run.trial085.stdout:[client] 2026-02-21T00:04:34.148 INFO:teuthology.orchestra.run.trial085.stdout: rgw cache enabled = true 2026-02-21T00:04:34.148 INFO:teuthology.orchestra.run.trial085.stdout: rgw enable ops log = true 2026-02-21T00:04:34.148 INFO:teuthology.orchestra.run.trial085.stdout: rgw enable usage log = true 2026-02-21T00:04:34.148 INFO:teuthology.orchestra.run.trial085.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-21T00:04:34.149 INFO:teuthology.orchestra.run.trial085.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-21T00:04:34.149 INFO:teuthology.orchestra.run.trial085.stdout:[mon.a] 2026-02-21T00:04:34.150 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-21T00:04:34.150 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-21T00:04:34.241 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-21T00:04:34.242 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:04:34.242 DEBUG:teuthology.orchestra.run.trial085:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-21T00:04:34.259 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:04:34.259 DEBUG:teuthology.orchestra.run.trial085:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-21T00:04:34.315 INFO:tasks.ceph:Sending monmap to node ubuntu@trial085.front.sepia.ceph.com 2026-02-21T00:04:34.315 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:04:34.315 DEBUG:teuthology.orchestra.run.trial085:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-21T00:04:34.315 DEBUG:teuthology.orchestra.run.trial085:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T00:04:34.388 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:04:34.388 DEBUG:teuthology.orchestra.run.trial085:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:04:34.443 INFO:tasks.ceph:Setting up mon nodes... 2026-02-21T00:04:34.444 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-21T00:04:34.444 DEBUG:teuthology.orchestra.run.trial085:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-21T00:04:34.534 INFO:teuthology.orchestra.run.trial085.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-21T00:04:34.537 INFO:tasks.ceph:Setting up mds nodes... 2026-02-21T00:04:34.538 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-21T00:04:34.538 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-21T00:04:34.580 INFO:teuthology.orchestra.run.trial085.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-21T00:04:34.592 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-21T00:04:34.592 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial085.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-21T00:04:34.593 DEBUG:teuthology.orchestra.run.trial085:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-21T00:04:34.656 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T00:04:34.656 INFO:tasks.ceph:role: osd.0 2026-02-21T00:04:34.656 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial085.front.sepia.ceph.com 2026-02-21T00:04:34.657 DEBUG:teuthology.orchestra.run.trial085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-21T00:04:34.721 INFO:teuthology.orchestra.run.trial085.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T00:04:34.722 INFO:teuthology.orchestra.run.trial085.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T00:04:34.722 INFO:teuthology.orchestra.run.trial085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T00:04:34.722 INFO:teuthology.orchestra.run.trial085.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T00:04:34.722 INFO:teuthology.orchestra.run.trial085.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T00:04:34.722 INFO:teuthology.orchestra.run.trial085.stdout: = sunit=0 swidth=0 blks 2026-02-21T00:04:34.722 INFO:teuthology.orchestra.run.trial085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T00:04:34.722 INFO:teuthology.orchestra.run.trial085.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T00:04:34.722 INFO:teuthology.orchestra.run.trial085.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T00:04:34.722 INFO:teuthology.orchestra.run.trial085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T00:04:35.293 INFO:teuthology.orchestra.run.trial085.stdout:Discarding blocks...Done. 2026-02-21T00:04:35.295 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial085.front.sepia.ceph.com -o noatime 2026-02-21T00:04:35.295 DEBUG:teuthology.orchestra.run.trial085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-21T00:04:35.364 DEBUG:teuthology.orchestra.run.trial085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-21T00:04:35.433 DEBUG:teuthology.orchestra.run.trial085:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-21T00:04:35.497 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T00:04:35.497 INFO:tasks.ceph:role: osd.1 2026-02-21T00:04:35.497 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial085.front.sepia.ceph.com 2026-02-21T00:04:35.498 DEBUG:teuthology.orchestra.run.trial085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-21T00:04:35.562 INFO:teuthology.orchestra.run.trial085.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T00:04:35.563 INFO:teuthology.orchestra.run.trial085.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T00:04:35.563 INFO:teuthology.orchestra.run.trial085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T00:04:35.563 INFO:teuthology.orchestra.run.trial085.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T00:04:35.563 INFO:teuthology.orchestra.run.trial085.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T00:04:35.563 INFO:teuthology.orchestra.run.trial085.stdout: = sunit=0 swidth=0 blks 2026-02-21T00:04:35.563 INFO:teuthology.orchestra.run.trial085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T00:04:35.563 INFO:teuthology.orchestra.run.trial085.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T00:04:35.563 INFO:teuthology.orchestra.run.trial085.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T00:04:35.563 INFO:teuthology.orchestra.run.trial085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T00:04:36.124 INFO:teuthology.orchestra.run.trial085.stdout:Discarding blocks...Done. 2026-02-21T00:04:36.125 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial085.front.sepia.ceph.com -o noatime 2026-02-21T00:04:36.125 DEBUG:teuthology.orchestra.run.trial085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-21T00:04:36.195 DEBUG:teuthology.orchestra.run.trial085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-21T00:04:36.262 DEBUG:teuthology.orchestra.run.trial085:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-21T00:04:36.326 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T00:04:36.326 INFO:tasks.ceph:role: osd.2 2026-02-21T00:04:36.326 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial085.front.sepia.ceph.com 2026-02-21T00:04:36.326 DEBUG:teuthology.orchestra.run.trial085:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-21T00:04:36.393 INFO:teuthology.orchestra.run.trial085.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T00:04:36.393 INFO:teuthology.orchestra.run.trial085.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T00:04:36.393 INFO:teuthology.orchestra.run.trial085.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T00:04:36.393 INFO:teuthology.orchestra.run.trial085.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T00:04:36.393 INFO:teuthology.orchestra.run.trial085.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T00:04:36.393 INFO:teuthology.orchestra.run.trial085.stdout: = sunit=0 swidth=0 blks 2026-02-21T00:04:36.393 INFO:teuthology.orchestra.run.trial085.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T00:04:36.393 INFO:teuthology.orchestra.run.trial085.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T00:04:36.393 INFO:teuthology.orchestra.run.trial085.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T00:04:36.393 INFO:teuthology.orchestra.run.trial085.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T00:04:36.951 INFO:teuthology.orchestra.run.trial085.stdout:Discarding blocks...Done. 2026-02-21T00:04:36.953 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial085.front.sepia.ceph.com -o noatime 2026-02-21T00:04:36.953 DEBUG:teuthology.orchestra.run.trial085:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-21T00:04:37.022 DEBUG:teuthology.orchestra.run.trial085:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-21T00:04:37.088 DEBUG:teuthology.orchestra.run.trial085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:04:37.241 INFO:teuthology.orchestra.run.trial085.stderr:INFO  osd - OSD::main: parsing early config 2026-02-21T00:04:37.296 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:37,295 seastar - Reactor backend: linux-aio 2026-02-21T00:04:37.312 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:37,310 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:04:37.312 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:37,310 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:04:37.314 INFO:teuthology.orchestra.run.trial085.stderr:==38911==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:04:37.373 INFO:teuthology.orchestra.run.trial085.stderr:DEBUG 2026-02-21 00:04:37,371 [shard 0:main] seastar - reactor::drain 2026-02-21T00:04:39.426 INFO:teuthology.orchestra.run.trial085.stderr:----------------------------------------------------- 2026-02-21T00:04:39.426 INFO:teuthology.orchestra.run.trial085.stderr:Suppressions used: 2026-02-21T00:04:39.426 INFO:teuthology.orchestra.run.trial085.stderr: count bytes template 2026-02-21T00:04:39.426 INFO:teuthology.orchestra.run.trial085.stderr: 1 8 MallocExtension::Register 2026-02-21T00:04:39.426 INFO:teuthology.orchestra.run.trial085.stderr:----------------------------------------------------- 2026-02-21T00:04:39.427 INFO:teuthology.orchestra.run.trial085.stderr: 2026-02-21T00:04:39.445 INFO:teuthology.orchestra.run.trial085.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-21T00:04:39.445 INFO:teuthology.orchestra.run.trial085.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-21T00:04:39.490 INFO:teuthology.orchestra.run.trial085.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-21T00:04:39.491 INFO:teuthology.orchestra.run.trial085.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-21T00:04:39.497 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:39,495 seastar - Reactor backend: linux-aio 2026-02-21T00:04:39.507 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:39,506 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:04:39.508 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:39,506 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:04:39.511 INFO:teuthology.orchestra.run.trial085.stderr:==38909==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:04:39.511 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:39,510 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-21T00:04:39.669 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x9ff1773 0xa0b4067 0xa0b422f 0xa0b4390 0xa0b4884 0xa0d1aff 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:39.669 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0xa0d766e 0xa0d7987 0xa0d7b50 0xa0d7be3 0xa0d8a28 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:39.669 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297db /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x116816a 0xf741c5 0x9fe874e 0x9fe908d 0xa0ce025 0xa0d870b 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:39.702 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe6ca27 0xa0b3f0a 0xa0b40c6 0xa0b422f 0xa0b435a 0xa0b4884 0xa0d1aff 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:04:39.702 INFO:teuthology.orchestra.run.trial085.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff89b27ef7 0xffffffff897f7f16 0xffffffff897fb8d0 0xffffffff897fcc31 0xffffffff89824893 0xffffffff89824a55 0xffffffff897c97c8 0xffffffff897d3aeb 0xffffffff897d3f4b 0xffffffff894910d2 0xffffffff8a14d451 0xffffffff8a200ba2 2026-02-21T00:04:39.731 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 33 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x9fa43e7 0xa0ce48a 0xa0d870b 0x7bc71f4 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:39.763 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1190d8 /lib64/libasan.so.8+0xebbf /lib64/libasan.so.8+0xec776 0xcee0e1 0xa0b450b 0xa0b4884 0xa0d1aff 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:39.764 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x157c7 0x10d1d95 0xf741d0 0xa0ce4e9 0xa0d870b 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:39.789 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f1 0x2cea82b 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:04:39.848 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x2cea0e0 0x8097714 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:39.848 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x2cea708 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:39.848 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x2cea656 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:39.949 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1586f 0x10d1d95 0xf741d0 0x9fe4c31 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5a18 0x9fec227 0x9fec907 0xa0d213c 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x77b6684 0x76972d7 0x787c2e0 0x76e6c34 0x76e6d4d 0xa3a2366 0xa3a2569 0xa3a2607 0xa3a268c 0xa3f5a8e 0xa3f5b47 0xa427489 0xa427830 0xa32dbb7 0xa32dc70 0xa32dcff 0xa3f55f6 0xa3f56b9 0xa3f57c6 0xa3f5981 0x763ea65 0x7697545 0x769777c 0x7538acd 0x726508c 0x781ce68 0x781cfbc 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:04:39.993 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x15197 0x9fe4acd 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5f05 0x9fe6403 0x9fe6a33 0x9fe6b9d 0xa0c1f5d 0xa0d4d60 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:39.993 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119134 /lib64/libasan.so.8+0xebbf /lib64/libasan.so.8+0xec776 0xcee0e1 0x7242c44 0x7eff140 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:39.993 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x9fe4b96 0x9fe4f2e 0x9fe51a1 0x9fe61a0 0x9fec58b 0x9fec907 0xa0d213c 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:40.075 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x2cea0de 0x8097714 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:04:40.109 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x2cea81a 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:40.110 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x2cea82b 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:40.110 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x2cea82b 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:42.313 INFO:teuthology.orchestra.run.trial085.stderr:----------------------------------------------------- 2026-02-21T00:04:42.314 INFO:teuthology.orchestra.run.trial085.stderr:Suppressions used: 2026-02-21T00:04:42.314 INFO:teuthology.orchestra.run.trial085.stderr: count bytes template 2026-02-21T00:04:42.314 INFO:teuthology.orchestra.run.trial085.stderr: 1 8 MallocExtension::Register 2026-02-21T00:04:42.314 INFO:teuthology.orchestra.run.trial085.stderr:----------------------------------------------------- 2026-02-21T00:04:42.314 INFO:teuthology.orchestra.run.trial085.stderr: 2026-02-21T00:04:42.351 DEBUG:teuthology.orchestra.run.trial085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-21T00:04:42.378 DEBUG:teuthology.orchestra.run.trial085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:04:42.532 INFO:teuthology.orchestra.run.trial085.stderr:INFO  osd - OSD::main: parsing early config 2026-02-21T00:04:42.585 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:42,583 seastar - Reactor backend: linux-aio 2026-02-21T00:04:42.595 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:42,593 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:04:42.595 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:42,593 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:04:42.597 INFO:teuthology.orchestra.run.trial085.stderr:==38981==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:04:42.657 INFO:teuthology.orchestra.run.trial085.stderr:DEBUG 2026-02-21 00:04:42,655 [shard 0:main] seastar - reactor::drain 2026-02-21T00:04:44.716 INFO:teuthology.orchestra.run.trial085.stderr:----------------------------------------------------- 2026-02-21T00:04:44.717 INFO:teuthology.orchestra.run.trial085.stderr:Suppressions used: 2026-02-21T00:04:44.717 INFO:teuthology.orchestra.run.trial085.stderr: count bytes template 2026-02-21T00:04:44.717 INFO:teuthology.orchestra.run.trial085.stderr: 1 8 MallocExtension::Register 2026-02-21T00:04:44.717 INFO:teuthology.orchestra.run.trial085.stderr:----------------------------------------------------- 2026-02-21T00:04:44.717 INFO:teuthology.orchestra.run.trial085.stderr: 2026-02-21T00:04:44.737 INFO:teuthology.orchestra.run.trial085.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-21T00:04:44.737 INFO:teuthology.orchestra.run.trial085.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-21T00:04:44.775 INFO:teuthology.orchestra.run.trial085.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-21T00:04:44.775 INFO:teuthology.orchestra.run.trial085.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-21T00:04:44.782 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:44,780 seastar - Reactor backend: linux-aio 2026-02-21T00:04:44.797 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:44,795 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:04:44.797 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:44,795 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:04:44.801 INFO:teuthology.orchestra.run.trial085.stderr:==38979==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:04:44.801 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:44,799 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-21T00:04:44.964 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15835 0x10d1d95 0xf741d0 0x9fe4c31 0x9fe4f2e 0x9fe51a1 0x9fe6ec7 0x9fe7414 0x9fe7613 0xa0c7985 0xa0c80e0 0xa0d236e 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:44.965 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119134 /lib64/libasan.so.8+0xebbf /lib64/libasan.so.8+0xec776 0xcee0e1 0x7242c44 0x7eff140 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:44.965 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0xa0c959e 0xa0cdc00 0xa0ce23e 0xa0d870b 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:44.996 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x11681cc 0xf741c5 0x9fe4a64 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5f05 0x9fe6403 0x9fe6a33 0x9fe6b9d 0xa0c1f5d 0xa0d4d60 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:04:45.026 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c6 /lib64/libasan.so.8+0x15197 0x116816a 0xf741c5 0x9fe4a64 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5a18 0x9fec227 0x9fec907 0xa0d213c 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:45.051 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x2ce9b7d 0x8097707 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:04:45.058 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xfef07 /lib64/libasan.so.8+0xfd214 /lib64/libasan.so.8+0xec7e9 0xcee0e1 0x773f8b4 0x75cad8c 0x74260d4 0x7bc7231 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:45.060 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x104e62 /lib64/libasan.so.8+0xea9a /lib64/libasan.so.8+0xec776 0xcee0e1 0x773f8b4 0x75cad8c 0x74260d4 0x7bc7231 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:45.143 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x2cea185 0x8097714 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:45.143 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x2c959d6 0x2cea7cc 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:45.146 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc657f /lib64/libasan.so.8+0x15197 0x2cea82b 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:45.286 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x9fe862f 0x9fe908d 0xa0ce025 0xa0d870b 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:45.287 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc589 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x776e6e7 0x776e786 0x755bbfb 0x763dfc6 0x7efedc0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:45.287 INFO:teuthology.orchestra.run.trial085.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff897e688d 0xffffffff897cb9d2 0xffffffff897d3aeb 0xffffffff897d3f4b 0xffffffff894910d2 0xffffffff8a14d451 0xffffffff8a200ba2 2026-02-21T00:04:45.287 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x10d1d30 0xf741d0 0x9fe4c31 0x9fe4f2e 0x9fe51a1 0x9fe6ec7 0xa0c58ee 0xa0c7540 0xa0c7b9a 0xa0c80e0 0xa0d236e 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:45.368 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x2cea102 0x8097714 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:04:45.403 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/ld-linux-x86-64.so.2+0x16ec6 /lib64/libasan.so.8+0xc65b1 /lib64/libasan.so.8+0x15197 0x2cea82b 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:45.404 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2cea82b 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:45.404 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x2cea82b 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:47.595 INFO:teuthology.orchestra.run.trial085.stderr:----------------------------------------------------- 2026-02-21T00:04:47.596 INFO:teuthology.orchestra.run.trial085.stderr:Suppressions used: 2026-02-21T00:04:47.596 INFO:teuthology.orchestra.run.trial085.stderr: count bytes template 2026-02-21T00:04:47.596 INFO:teuthology.orchestra.run.trial085.stderr: 1 8 MallocExtension::Register 2026-02-21T00:04:47.596 INFO:teuthology.orchestra.run.trial085.stderr:----------------------------------------------------- 2026-02-21T00:04:47.596 INFO:teuthology.orchestra.run.trial085.stderr: 2026-02-21T00:04:47.634 DEBUG:teuthology.orchestra.run.trial085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-21T00:04:47.660 DEBUG:teuthology.orchestra.run.trial085:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:04:47.815 INFO:teuthology.orchestra.run.trial085.stderr:INFO  osd - OSD::main: parsing early config 2026-02-21T00:04:47.866 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:47,864 seastar - Reactor backend: linux-aio 2026-02-21T00:04:47.883 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:47,882 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:04:47.884 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:47,882 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:04:47.886 INFO:teuthology.orchestra.run.trial085.stderr:==39048==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:04:47.946 INFO:teuthology.orchestra.run.trial085.stderr:DEBUG 2026-02-21 00:04:47,944 [shard 0:main] seastar - reactor::drain 2026-02-21T00:04:49.997 INFO:teuthology.orchestra.run.trial085.stderr:----------------------------------------------------- 2026-02-21T00:04:49.997 INFO:teuthology.orchestra.run.trial085.stderr:Suppressions used: 2026-02-21T00:04:49.997 INFO:teuthology.orchestra.run.trial085.stderr: count bytes template 2026-02-21T00:04:49.997 INFO:teuthology.orchestra.run.trial085.stderr: 1 8 MallocExtension::Register 2026-02-21T00:04:49.997 INFO:teuthology.orchestra.run.trial085.stderr:----------------------------------------------------- 2026-02-21T00:04:49.997 INFO:teuthology.orchestra.run.trial085.stderr: 2026-02-21T00:04:50.017 INFO:teuthology.orchestra.run.trial085.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-21T00:04:50.017 INFO:teuthology.orchestra.run.trial085.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-21T00:04:50.065 INFO:teuthology.orchestra.run.trial085.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-21T00:04:50.066 INFO:teuthology.orchestra.run.trial085.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-21T00:04:50.072 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:50,071 seastar - Reactor backend: linux-aio 2026-02-21T00:04:50.082 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:50,080 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:04:50.082 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:50,080 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:04:50.086 INFO:teuthology.orchestra.run.trial085.stderr:==39046==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:04:50.086 INFO:teuthology.orchestra.run.trial085.stderr:INFO 2026-02-21 00:04:50,085 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-21T00:04:50.247 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xed720 0xcee0e1 0x7ebced5 0xa0d8726 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:50.247 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1583b 0x10d1d95 0xf741d0 0xa0c86d7 0xa0cdc00 0xa0ce23e 0xa0d870b 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:50.248 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16b35d /lib64/libasan.so.8+0xb0541 /lib64/libasan.so.8+0xb1437 0xba66a7 0x10d1d1e 0xf741d0 0x9fe4a64 0x9fe4f2e 0x9fe51a1 0x9fe6ec7 0x9fe7414 0x9fe7613 0xa0c7985 0xa0c80e0 0xa0d236e 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:50.279 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x116816a 0xf741c5 0x9fe4a64 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5f05 0x9fe6403 0x9fe6a33 0x9fe6b9d 0xa0c1f5d 0xa0d4d60 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:04:50.309 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe6ca27 0xe6cd7e 0x77b14b5 0x77b15b6 0x763e19c 0x7bc7158 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:50.309 INFO:teuthology.orchestra.run.trial085.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff897d383c 0xffffffff897d3f4b 0xffffffff894910d2 0xffffffff8a14d451 0xffffffff8a200ba2 2026-02-21T00:04:50.341 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xeaaa1 0xe6c75a 0xe6cb15 0xe6cd7e 0x77b14b5 0x77b15b6 0x763e19c 0x7bc7158 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:50.342 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe6ca27 0xa0d86a2 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:50.342 INFO:teuthology.orchestra.run.trial085.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff89b27ef7 0xffffffff897f7f16 0xffffffff897fb8d0 0xffffffff897fcc31 0xffffffff89824893 0xffffffff89824a55 0xffffffff897c97c8 0xffffffff897d3aeb 0xffffffff897d3f4b 0xffffffff894910d2 0xffffffff8a14d451 0xffffffff8a200ba2 2026-02-21T00:04:50.366 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1521e 0x2cea82b 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:04:50.426 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2cea82b 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:50.426 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297b3 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2cea82b 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:50.426 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65bc /lib64/libasan.so.8+0x15197 0x2cea82b 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:50.526 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0xe6c9d9 0x7ebcea8 0xa0d8726 0x7bc71f4 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x77b6684 0x76972d7 0x787c2e0 0x76e6c34 0x76e6d4d 0xa3a2366 0xa3a2569 0xa3a2607 0xa3a268c 0xa3f5a8e 0xa3f5b47 0xa427489 0xa427830 0xa32dbb7 0xa32dc70 0xa32dcff 0xa3f55f6 0xa3f56b9 0xa3f57c6 0xa3f5981 0x763ea65 0x7697545 0x769777c 0x7538acd 0x726508c 0x781ce68 0x781cfbc 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:04:50.568 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x129896 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x9fa8701 0xa0c8510 0xa0cdc00 0xa0ce23e 0xa0d870b 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:50.568 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc589 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x776e6e7 0x776e786 0x755bbfb 0x763dfc6 0x7efea04 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:50.569 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc5c0 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe6ca27 0xe6cd08 0x77b14b5 0x77b15b6 0x763e19c 0x7efea04 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:50.651 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x2c959ad 0x2cea7cc 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:04:50.685 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1294d2 /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2cea82b 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:50.689 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151e3 0x2cea82b 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:50.689 INFO:teuthology.orchestra.run.trial085.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1006a8 /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2cea82b 0x80976d9 0x816850e 0x8168d75 0x8166a17 0x7b56d4f 0x7b57a44 0x7c6a0e6 0x7b2c132 0x7b8b0ea 0x7c1c497 0x7c1c710 0x7c560b9 0x7c56218 0x7c6a249 0x7b52868 0x7de7d79 0x7c778b0 0x7db8cea 0x76ae786 0x76bcb5b 0x77670af 0x790343c 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:04:52.947 INFO:teuthology.orchestra.run.trial085.stderr:----------------------------------------------------- 2026-02-21T00:04:52.947 INFO:teuthology.orchestra.run.trial085.stderr:Suppressions used: 2026-02-21T00:04:52.947 INFO:teuthology.orchestra.run.trial085.stderr: count bytes template 2026-02-21T00:04:52.947 INFO:teuthology.orchestra.run.trial085.stderr: 1 8 MallocExtension::Register 2026-02-21T00:04:52.948 INFO:teuthology.orchestra.run.trial085.stderr:----------------------------------------------------- 2026-02-21T00:04:52.948 INFO:teuthology.orchestra.run.trial085.stderr: 2026-02-21T00:04:52.987 DEBUG:teuthology.orchestra.run.trial085:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-21T00:04:53.013 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-21T00:04:53.013 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:04:53.014 DEBUG:teuthology.orchestra.run.trial085:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-21T00:04:53.077 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:04:53.078 DEBUG:teuthology.orchestra.run.trial085:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-21T00:04:53.142 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:04:53.142 DEBUG:teuthology.orchestra.run.trial085:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-21T00:04:53.206 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:04:53.206 DEBUG:teuthology.orchestra.run.trial085:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-21T00:04:53.269 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:04:53.269 DEBUG:teuthology.orchestra.run.trial085:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-21T00:04:53.326 INFO:tasks.ceph:Adding keys to all mons... 2026-02-21T00:04:53.326 DEBUG:teuthology.orchestra.run.trial085:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-21T00:04:53.388 INFO:teuthology.orchestra.run.trial085.stdout:[mgr.x] 2026-02-21T00:04:53.389 INFO:teuthology.orchestra.run.trial085.stdout: key = AQCS9phpTyjHHxAAJxgfHalZ9ekymjEV0wwlQA== 2026-02-21T00:04:53.389 INFO:teuthology.orchestra.run.trial085.stdout:[osd.0] 2026-02-21T00:04:53.389 INFO:teuthology.orchestra.run.trial085.stdout: key = AQCX9phpzqw5IhAAM/yn5Ak9iS3//EdVZfYS+A== 2026-02-21T00:04:53.389 INFO:teuthology.orchestra.run.trial085.stdout:[osd.1] 2026-02-21T00:04:53.389 INFO:teuthology.orchestra.run.trial085.stdout: key = AQCc9phpwXnHMxAAsVgUzk9pF4iR6JwsI5EJJA== 2026-02-21T00:04:53.389 INFO:teuthology.orchestra.run.trial085.stdout:[osd.2] 2026-02-21T00:04:53.389 INFO:teuthology.orchestra.run.trial085.stdout: key = AQCi9phpLZDnCBAASvOmmOdV5O8EJJatGvgkeA== 2026-02-21T00:04:53.389 INFO:teuthology.orchestra.run.trial085.stdout:[client.0] 2026-02-21T00:04:53.390 INFO:teuthology.orchestra.run.trial085.stdout: key = AQCS9phpvMCIIhAAdKXIXhfDZz4KSU9EuInb3Q== 2026-02-21T00:04:53.391 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-21T00:04:53.479 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T00:04:53.527 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T00:04:53.576 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T00:04:53.626 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-21T00:04:53.676 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-21T00:04:53.676 DEBUG:teuthology.orchestra.run.trial085:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-21T00:04:53.701 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-21T00:04:53.786 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:53.783+0000 7f976a662c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:53.786 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:53.783+0000 7f976a662c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:53.793 DEBUG:teuthology.orchestra.run.trial085:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-21T00:04:53.818 DEBUG:teuthology.orchestra.run.trial085:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:04:53.875 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-21T00:04:53.875 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-21T00:04:53.876 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-21T00:04:53.917 INFO:tasks.ceph.mon.a:Started 2026-02-21T00:04:53.918 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-21T00:04:53.918 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-21T00:04:53.918 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-21T00:04:53.921 INFO:tasks.ceph.mgr.x:Started 2026-02-21T00:04:53.921 DEBUG:tasks.ceph:set 0 configs 2026-02-21T00:04:53.921 DEBUG:teuthology.orchestra.run.trial085:> sudo ceph --cluster ceph config dump 2026-02-21T00:04:53.981 INFO:tasks.ceph.mon.a.trial085.stderr:2026-02-21T00:04:53.978+0000 7f08d4089c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:53.981 INFO:tasks.ceph.mon.a.trial085.stderr:2026-02-21T00:04:53.978+0000 7f08d4089c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:53.981 INFO:tasks.ceph.mon.a.trial085.stderr:2026-02-21T00:04:53.978+0000 7f08d4089c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:53.983 INFO:tasks.ceph.mgr.x.trial085.stderr:2026-02-21T00:04:53.980+0000 7f9ea53c4fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:53.998 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:53.993+0000 7fb4c957e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:53.999 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:53.996+0000 7fb4c957e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:54.030 INFO:teuthology.orchestra.run.trial085.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-21T00:04:54.038 INFO:tasks.ceph:Setting crush tunables to default 2026-02-21T00:04:54.038 DEBUG:teuthology.orchestra.run.trial085:> sudo ceph --cluster ceph osd crush tunables default 2026-02-21T00:04:54.114 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:54.108+0000 7fbf4be51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:54.115 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:54.112+0000 7fbf4be51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:54.148 INFO:teuthology.orchestra.run.trial085.stderr:adjusted tunables profile to default 2026-02-21T00:04:54.156 INFO:tasks.ceph:check_enable_crimson: True 2026-02-21T00:04:54.156 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-21T00:04:54.157 DEBUG:teuthology.orchestra.run.trial085:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-21T00:04:54.187 INFO:tasks.ceph.mgr.x.trial085.stderr:2026-02-21T00:04:54.184+0000 7f9ea53c4fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:54.187 INFO:tasks.ceph.mgr.x.trial085.stderr:2026-02-21T00:04:54.184+0000 7f9ea53c4fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:54.232 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:54.226+0000 7f25739e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:54.233 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:54.230+0000 7f25739e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:54.272 INFO:tasks.ceph.mon.a.trial085.stderr:2026-02-21T00:04:54.270+0000 7f08cb7f1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-21T00:04:54.273 INFO:tasks.ceph.mon.a.trial085.stderr:Please be aware that this feature is experimental, untested, 2026-02-21T00:04:54.273 INFO:tasks.ceph.mon.a.trial085.stderr:unsupported, and may result in data corruption, data loss, 2026-02-21T00:04:54.273 INFO:tasks.ceph.mon.a.trial085.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-21T00:04:54.273 INFO:tasks.ceph.mon.a.trial085.stderr:feature with important data. 2026-02-21T00:04:54.273 INFO:tasks.ceph.mon.a.trial085.stderr: 2026-02-21T00:04:55.012 INFO:tasks.ceph.mgr.x.trial085.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-21T00:04:55.012 INFO:tasks.ceph.mgr.x.trial085.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-21T00:04:55.012 INFO:tasks.ceph.mgr.x.trial085.stderr: from numpy import show_config as show_numpy_config 2026-02-21T00:04:55.159 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-21T00:04:55.159 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:04:55.159 DEBUG:teuthology.orchestra.run.trial085:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-21T00:04:55.224 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:04:55.224 DEBUG:teuthology.orchestra.run.trial085:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-21T00:04:55.288 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:04:55.288 DEBUG:teuthology.orchestra.run.trial085:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-21T00:04:55.352 DEBUG:teuthology.orchestra.run.trial085:> sudo ceph --cluster ceph osd new b5ac8237-b6ef-4c10-966d-80a9efcdac99 0 2026-02-21T00:04:55.465 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:55.459+0000 7fe17ac5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:55.466 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:55.463+0000 7fe17ac5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:55.503 INFO:teuthology.orchestra.run.trial085.stdout:0 2026-02-21T00:04:55.512 DEBUG:teuthology.orchestra.run.trial085:> sudo ceph --cluster ceph osd new 7308fd20-2bab-4cfb-9afa-dcf92d53965a 1 2026-02-21T00:04:55.587 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:55.582+0000 7fe629d5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:55.588 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:55.586+0000 7fe629d5c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:55.624 INFO:teuthology.orchestra.run.trial085.stdout:1 2026-02-21T00:04:55.633 DEBUG:teuthology.orchestra.run.trial085:> sudo ceph --cluster ceph osd new c64fe3e0-3cb0-4fa1-92c5-c1dfe1a470ba 2 2026-02-21T00:04:55.706 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:55.700+0000 7fd1d7a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:55.707 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:55.705+0000 7fd1d7a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:55.744 INFO:teuthology.orchestra.run.trial085.stdout:2 2026-02-21T00:04:55.752 DEBUG:teuthology.orchestra.run.trial085:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-21T00:04:55.782 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-21T00:04:55.782 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-21T00:04:55.823 INFO:tasks.ceph.osd.0:Started 2026-02-21T00:04:55.823 DEBUG:teuthology.orchestra.run.trial085:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-21T00:04:55.851 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-21T00:04:55.852 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-21T00:04:55.895 INFO:tasks.ceph.osd.1:Started 2026-02-21T00:04:55.895 DEBUG:teuthology.orchestra.run.trial085:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-21T00:04:55.922 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-21T00:04:55.922 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-21T00:04:55.964 INFO:tasks.ceph.osd.2:Started 2026-02-21T00:04:55.965 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T00:04:56.040 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:56.035+0000 7f531bb6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:56.041 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:04:56.039+0000 7f531bb6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:56.074 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:04:56.074 INFO:teuthology.orchestra.run.trial085.stdout:{"epoch":6,"fsid":"e5b08652-3d06-4075-8c52-d52bf7e93d0a","created":"2026-02-21T00:04:53.989909+0000","modified":"2026-02-21T00:04:55.742290+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T00:04:55.742290+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"b5ac8237-b6ef-4c10-966d-80a9efcdac99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7308fd20-2bab-4cfb-9afa-dcf92d53965a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c64fe3e0-3cb0-4fa1-92c5-c1dfe1a470ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:04:56.083 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-21T00:04:56.083 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-21T00:04:56.385 DEBUG:teuthology.orchestra.run.trial085:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T00:04:56.452 INFO:teuthology.misc.health.trial085.stderr:2026-02-21T00:04:56.447+0000 7fa9600cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:56.453 INFO:teuthology.misc.health.trial085.stderr:2026-02-21T00:04:56.451+0000 7fa9600cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:04:56.486 INFO:teuthology.misc.health.trial085.stdout: 2026-02-21T00:04:56.486 INFO:teuthology.misc.health.trial085.stdout:{"epoch":6,"fsid":"e5b08652-3d06-4075-8c52-d52bf7e93d0a","created":"2026-02-21T00:04:53.989909+0000","modified":"2026-02-21T00:04:55.742290+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T00:04:55.742290+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"b5ac8237-b6ef-4c10-966d-80a9efcdac99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7308fd20-2bab-4cfb-9afa-dcf92d53965a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c64fe3e0-3cb0-4fa1-92c5-c1dfe1a470ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:04:56.494 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-21T00:04:57.229 INFO:tasks.ceph.mgr.x.trial085.stderr:2026-02-21T00:04:57.226+0000 7f9dc22e7640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-21T00:05:02.796 DEBUG:teuthology.orchestra.run.trial085:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T00:05:02.883 INFO:teuthology.misc.health.trial085.stderr:2026-02-21T00:05:02.877+0000 7f811b38c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:02.886 INFO:teuthology.misc.health.trial085.stderr:2026-02-21T00:05:02.883+0000 7f811b38c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:03.024 INFO:teuthology.misc.health.trial085.stdout: 2026-02-21T00:05:03.024 INFO:teuthology.misc.health.trial085.stdout:{"epoch":7,"fsid":"e5b08652-3d06-4075-8c52-d52bf7e93d0a","created":"2026-02-21T00:04:53.989909+0000","modified":"2026-02-21T00:05:02.258639+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T00:04:55.742290+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"b5ac8237-b6ef-4c10-966d-80a9efcdac99","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"7308fd20-2bab-4cfb-9afa-dcf92d53965a","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"c64fe3e0-3cb0-4fa1-92c5-c1dfe1a470ba","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:05:03.039 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-21T00:05:09.341 DEBUG:teuthology.orchestra.run.trial085:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T00:05:09.408 INFO:teuthology.misc.health.trial085.stderr:2026-02-21T00:05:09.405+0000 7f3cb2a51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:09.409 INFO:teuthology.misc.health.trial085.stderr:2026-02-21T00:05:09.407+0000 7f3cb2a51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:09.517 INFO:teuthology.misc.health.trial085.stdout: 2026-02-21T00:05:09.518 INFO:teuthology.misc.health.trial085.stdout:{"epoch":13,"fsid":"e5b08652-3d06-4075-8c52-d52bf7e93d0a","created":"2026-02-21T00:04:53.989909+0000","modified":"2026-02-21T00:05:08.279052+0000","last_up_change":"2026-02-21T00:05:05.266644+0000","last_in_change":"2026-02-21T00:04:55.742290+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T00:05:06.239589+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b5ac8237-b6ef-4c10-966d-80a9efcdac99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6807","nonce":3275033524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6806","nonce":3275033524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6813","nonce":3275033524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6804","nonce":3275033524}]},"public_addr":"10.20.193.85:6807/3275033524","cluster_addr":"10.20.193.85:6806/3275033524","heartbeat_back_addr":"10.20.193.85:6813/3275033524","heartbeat_front_addr":"10.20.193.85:6804/3275033524","state":["exists","up"]},{"osd":1,"uuid":"7308fd20-2bab-4cfb-9afa-dcf92d53965a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6803","nonce":1817488071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6802","nonce":1817488071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6810","nonce":1817488071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6811","nonce":1817488071}]},"public_addr":"10.20.193.85:6803/1817488071","cluster_addr":"10.20.193.85:6802/1817488071","heartbeat_back_addr":"10.20.193.85:6810/1817488071","heartbeat_front_addr":"10.20.193.85:6811/1817488071","state":["exists","up"]},{"osd":2,"uuid":"c64fe3e0-3cb0-4fa1-92c5-c1dfe1a470ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6808","nonce":3214192654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6805","nonce":3214192654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6812","nonce":3214192654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6809","nonce":3214192654}]},"public_addr":"10.20.193.85:6808/3214192654","cluster_addr":"10.20.193.85:6805/3214192654","heartbeat_back_addr":"10.20.193.85:6812/3214192654","heartbeat_front_addr":"10.20.193.85:6809/3214192654","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:05:09.525 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-21T00:05:09.525 INFO:tasks.ceph:Creating RBD pool 2026-02-21T00:05:09.526 DEBUG:teuthology.orchestra.run.trial085:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-21T00:05:09.600 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:09.595+0000 7f6ce1c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:09.601 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:09.600+0000 7f6ce1c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:10.286 INFO:teuthology.orchestra.run.trial085.stderr:pool 'rbd' created 2026-02-21T00:05:10.299 DEBUG:teuthology.orchestra.run.trial085:> rbd --cluster ceph pool init rbd 2026-02-21T00:05:10.340 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:10.338+0000 7fc582c83240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:10.340 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:10.338+0000 7fc582c83240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:10.341 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:10.340+0000 7fc582c83240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:13.324 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-21T00:05:13.325 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-21T00:05:13.325 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-21T00:05:13.441 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:13.437+0000 7f00cf85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:13.442 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:13.441+0000 7f00cf85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:13.552 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:05:13.561 INFO:teuthology.orchestra.run.trial085.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-21T00:05:13.561 INFO:tasks.ceph_manager:config epoch is 1 2026-02-21T00:05:13.561 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-21T00:05:13.561 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-21T00:05:13.562 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-21T00:05:13.640 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:13.636+0000 7fafbf36f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:13.641 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:13.639+0000 7fafbf36f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:13.779 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:05:13.788 INFO:teuthology.orchestra.run.trial085.stdout:{"epoch":5,"flags":0,"active_gid":4105,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6800","nonce":3572603151},{"type":"v1","addr":"10.20.193.85:6801","nonce":3572603151}]},"active_addr":"10.20.193.85:6801/3572603151","active_change":"2026-02-21T00:04:56.222436+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.85:0","nonce":1078869727}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.85:0","nonce":3703644088}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.85:0","nonce":2833192117}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.85:0","nonce":2601808249}]}]} 2026-02-21T00:05:13.791 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-21T00:05:13.791 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-21T00:05:13.791 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T00:05:13.867 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:13.864+0000 7fc54b4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:13.869 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:13.867+0000 7fc54b4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:13.977 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:05:13.978 INFO:teuthology.orchestra.run.trial085.stdout:{"epoch":17,"fsid":"e5b08652-3d06-4075-8c52-d52bf7e93d0a","created":"2026-02-21T00:04:53.989909+0000","modified":"2026-02-21T00:05:13.296295+0000","last_up_change":"2026-02-21T00:05:05.266644+0000","last_in_change":"2026-02-21T00:04:55.742290+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T00:05:06.239589+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-21T00:05:09.713082+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b5ac8237-b6ef-4c10-966d-80a9efcdac99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6807","nonce":3275033524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6806","nonce":3275033524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6813","nonce":3275033524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6804","nonce":3275033524}]},"public_addr":"10.20.193.85:6807/3275033524","cluster_addr":"10.20.193.85:6806/3275033524","heartbeat_back_addr":"10.20.193.85:6813/3275033524","heartbeat_front_addr":"10.20.193.85:6804/3275033524","state":["exists","up"]},{"osd":1,"uuid":"7308fd20-2bab-4cfb-9afa-dcf92d53965a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6803","nonce":1817488071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6802","nonce":1817488071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6810","nonce":1817488071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6811","nonce":1817488071}]},"public_addr":"10.20.193.85:6803/1817488071","cluster_addr":"10.20.193.85:6802/1817488071","heartbeat_back_addr":"10.20.193.85:6810/1817488071","heartbeat_front_addr":"10.20.193.85:6811/1817488071","state":["exists","up"]},{"osd":2,"uuid":"c64fe3e0-3cb0-4fa1-92c5-c1dfe1a470ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6808","nonce":3214192654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6805","nonce":3214192654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6812","nonce":3214192654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6809","nonce":3214192654}]},"public_addr":"10.20.193.85:6808/3214192654","cluster_addr":"10.20.193.85:6805/3214192654","heartbeat_back_addr":"10.20.193.85:6812/3214192654","heartbeat_front_addr":"10.20.193.85:6809/3214192654","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:05:13.987 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-21T00:05:13.987 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T00:05:14.067 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.063+0000 7f063d0cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.068 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.066+0000 7f063d0cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.178 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:05:14.178 INFO:teuthology.orchestra.run.trial085.stdout:{"epoch":17,"fsid":"e5b08652-3d06-4075-8c52-d52bf7e93d0a","created":"2026-02-21T00:04:53.989909+0000","modified":"2026-02-21T00:05:13.296295+0000","last_up_change":"2026-02-21T00:05:05.266644+0000","last_in_change":"2026-02-21T00:04:55.742290+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T00:05:06.239589+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-21T00:05:09.713082+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"b5ac8237-b6ef-4c10-966d-80a9efcdac99","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6807","nonce":3275033524}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6806","nonce":3275033524}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6813","nonce":3275033524}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6804","nonce":3275033524}]},"public_addr":"10.20.193.85:6807/3275033524","cluster_addr":"10.20.193.85:6806/3275033524","heartbeat_back_addr":"10.20.193.85:6813/3275033524","heartbeat_front_addr":"10.20.193.85:6804/3275033524","state":["exists","up"]},{"osd":1,"uuid":"7308fd20-2bab-4cfb-9afa-dcf92d53965a","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6803","nonce":1817488071}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6802","nonce":1817488071}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6810","nonce":1817488071}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6811","nonce":1817488071}]},"public_addr":"10.20.193.85:6803/1817488071","cluster_addr":"10.20.193.85:6802/1817488071","heartbeat_back_addr":"10.20.193.85:6810/1817488071","heartbeat_front_addr":"10.20.193.85:6811/1817488071","state":["exists","up"]},{"osd":2,"uuid":"c64fe3e0-3cb0-4fa1-92c5-c1dfe1a470ba","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6808","nonce":3214192654}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6805","nonce":3214192654}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6812","nonce":3214192654}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.85:6809","nonce":3214192654}]},"public_addr":"10.20.193.85:6808/3214192654","cluster_addr":"10.20.193.85:6805/3214192654","heartbeat_back_addr":"10.20.193.85:6812/3214192654","heartbeat_front_addr":"10.20.193.85:6809/3214192654","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:05:14.188 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-21T00:05:14.189 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-21T00:05:14.189 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-21T00:05:14.269 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.265+0000 7f582397e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.270 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.265+0000 7f357696f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.270 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.265+0000 7f1462f6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.271 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.269+0000 7f582397e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.271 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.269+0000 7f357696f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.271 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.269+0000 7f1462f6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.286 INFO:teuthology.orchestra.run.trial085.stdout:42949672968 2026-02-21T00:05:14.286 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T00:05:14.287 INFO:teuthology.orchestra.run.trial085.stdout:42949672968 2026-02-21T00:05:14.288 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T00:05:14.289 INFO:teuthology.orchestra.run.trial085.stdout:38654705674 2026-02-21T00:05:14.289 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T00:05:14.407 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.403+0000 7f4979a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.408 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.403+0000 7fb14ed0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.408 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.407+0000 7f4979a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.409 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.404+0000 7f1756da4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.409 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.408+0000 7fb14ed0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.410 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.408+0000 7f1756da4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.518 INFO:teuthology.orchestra.run.trial085.stdout:42949672968 2026-02-21T00:05:14.524 INFO:teuthology.orchestra.run.trial085.stdout:42949672968 2026-02-21T00:05:14.525 INFO:teuthology.orchestra.run.trial085.stdout:38654705674 2026-02-21T00:05:14.528 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672968 got 42949672968 for osd.2 2026-02-21T00:05:14.529 DEBUG:teuthology.parallel:result is None 2026-02-21T00:05:14.535 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672968 got 42949672968 for osd.0 2026-02-21T00:05:14.535 DEBUG:teuthology.parallel:result is None 2026-02-21T00:05:14.536 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705674 for osd.1 2026-02-21T00:05:14.537 DEBUG:teuthology.parallel:result is None 2026-02-21T00:05:14.537 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-21T00:05:14.537 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T00:05:14.650 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.647+0000 7efe59a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.651 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.650+0000 7efe59a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.758 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:05:14.759 INFO:teuthology.orchestra.run.trial085.stderr:dumped all 2026-02-21T00:05:14.767 INFO:teuthology.orchestra.run.trial085.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2026-02-21T00:05:14.278322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":2492,"kb_used_data":2492,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156497476,"statfs":{"total":160255967232,"available":160253415424,"internally_reserved":0,"allocated":2551808,"data_stored":2551808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.308895+0000","last_change":"2026-02-21T00:05:13.313294+0000","last_active":"2026-02-21T00:05:13.308895+0000","last_peered":"2026-02-21T00:05:13.308895+0000","last_clean":"2026-02-21T00:05:13.308895+0000","last_became_active":"2026-02-21T00:05:11.320227+0000","last_became_peered":"2026-02-21T00:05:11.320227+0000","last_unstale":"2026-02-21T00:05:13.308895+0000","last_undegraded":"2026-02-21T00:05:13.308895+0000","last_fullsized":"2026-02-21T00:05:13.308895+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_clean_scrub_stamp":"2026-02-21T00:05:10.282364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.311236+0000","last_change":"2026-02-21T00:05:13.314673+0000","last_active":"2026-02-21T00:05:13.311236+0000","last_peered":"2026-02-21T00:05:13.311236+0000","last_clean":"2026-02-21T00:05:13.311236+0000","last_became_active":"2026-02-21T00:05:11.319864+0000","last_became_peered":"2026-02-21T00:05:11.319864+0000","last_unstale":"2026-02-21T00:05:13.311236+0000","last_undegraded":"2026-02-21T00:05:13.311236+0000","last_fullsized":"2026-02-21T00:05:13.311236+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_clean_scrub_stamp":"2026-02-21T00:05:10.282364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.311144+0000","last_change":"2026-02-21T00:05:13.315561+0000","last_active":"2026-02-21T00:05:13.311144+0000","last_peered":"2026-02-21T00:05:13.311144+0000","last_clean":"2026-02-21T00:05:13.311144+0000","last_became_active":"2026-02-21T00:05:11.326559+0000","last_became_peered":"2026-02-21T00:05:11.326559+0000","last_unstale":"2026-02-21T00:05:13.311144+0000","last_undegraded":"2026-02-21T00:05:13.311144+0000","last_fullsized":"2026-02-21T00:05:13.311144+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_clean_scrub_stamp":"2026-02-21T00:05:10.282364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.311570+0000","last_change":"2026-02-21T00:05:13.314396+0000","last_active":"2026-02-21T00:05:13.311570+0000","last_peered":"2026-02-21T00:05:13.311570+0000","last_clean":"2026-02-21T00:05:13.311570+0000","last_became_active":"2026-02-21T00:05:11.322884+0000","last_became_peered":"2026-02-21T00:05:11.322884+0000","last_unstale":"2026-02-21T00:05:13.311570+0000","last_undegraded":"2026-02-21T00:05:13.311570+0000","last_fullsized":"2026-02-21T00:05:13.311570+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_clean_scrub_stamp":"2026-02-21T00:05:10.282364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.311310+0000","last_change":"2026-02-21T00:05:13.311310+0000","last_active":"2026-02-21T00:05:13.311310+0000","last_peered":"2026-02-21T00:05:13.311310+0000","last_clean":"2026-02-21T00:05:13.311310+0000","last_became_active":"2026-02-21T00:05:11.335291+0000","last_became_peered":"2026-02-21T00:05:11.335291+0000","last_unstale":"2026-02-21T00:05:13.311310+0000","last_undegraded":"2026-02-21T00:05:13.311310+0000","last_fullsized":"2026-02-21T00:05:13.311310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_clean_scrub_stamp":"2026-02-21T00:05:10.282364+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.308630+0000","last_change":"2026-02-21T00:05:13.311231+0000","last_active":"2026-02-21T00:05:13.308630+0000","last_peered":"2026-02-21T00:05:13.308630+0000","last_clean":"2026-02-21T00:05:13.308630+0000","last_became_active":"2026-02-21T00:05:11.325696+0000","last_became_peered":"2026-02-21T00:05:11.325696+0000","last_unstale":"2026-02-21T00:05:13.308630+0000","last_undegraded":"2026-02-21T00:05:13.308630+0000","last_fullsized":"2026-02-21T00:05:13.308630+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_clean_scrub_stamp":"2026-02-21T00:05:10.282364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.306811+0000","last_change":"2026-02-21T00:05:13.310120+0000","last_active":"2026-02-21T00:05:13.306811+0000","last_peered":"2026-02-21T00:05:13.306811+0000","last_clean":"2026-02-21T00:05:13.306811+0000","last_became_active":"2026-02-21T00:05:11.327110+0000","last_became_peered":"2026-02-21T00:05:11.327110+0000","last_unstale":"2026-02-21T00:05:13.306811+0000","last_undegraded":"2026-02-21T00:05:13.306811+0000","last_fullsized":"2026-02-21T00:05:13.306811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_clean_scrub_stamp":"2026-02-21T00:05:10.282364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.311231+0000","last_change":"2026-02-21T00:05:13.315577+0000","last_active":"2026-02-21T00:05:13.311231+0000","last_peered":"2026-02-21T00:05:13.311231+0000","last_clean":"2026-02-21T00:05:13.311231+0000","last_became_active":"2026-02-21T00:05:11.323002+0000","last_became_peered":"2026-02-21T00:05:11.323002+0000","last_unstale":"2026-02-21T00:05:13.311231+0000","last_undegraded":"2026-02-21T00:05:13.311231+0000","last_fullsized":"2026-02-21T00:05:13.311231+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_clean_scrub_stamp":"2026-02-21T00:05:10.282364+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.308718+0000","last_change":"2026-02-21T00:05:07.308443+0000","last_active":"2026-02-21T00:05:13.308718+0000","last_peered":"2026-02-21T00:05:13.308718+0000","last_clean":"2026-02-21T00:05:13.308718+0000","last_became_active":"2026-02-21T00:05:07.301117+0000","last_became_peered":"2026-02-21T00:05:07.301117+0000","last_unstale":"2026-02-21T00:05:13.308718+0000","last_undegraded":"2026-02-21T00:05:13.308718+0000","last_fullsized":"2026-02-21T00:05:13.308718+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:06.271075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:06.271075+0000","last_clean_scrub_stamp":"2026-02-21T00:05:06.271075+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253366272,"internally_reserved":0,"allocated":2600960,"data_stored":2600960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1116,"kb_used_data":1116,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165540,"statfs":{"total":53418655744,"available":53417512960,"internally_reserved":0,"allocated":1142784,"data_stored":1142784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":308,"kb_used_data":308,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166348,"statfs":{"total":53418655744,"available":53418340352,"internally_reserved":0,"allocated":315392,"data_stored":315392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1068,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165588,"statfs":{"total":53418655744,"available":53417562112,"internally_reserved":0,"allocated":1093632,"data_stored":1093632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417553920,"internally_reserved":0,"allocated":1101824,"data_stored":1101824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418332160,"internally_reserved":0,"allocated":323584,"data_stored":323584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T00:05:14.769 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T00:05:14.843 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.840+0000 7fb5c00cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.844 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:14.843+0000 7fb5c00cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:14.950 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:05:14.951 INFO:teuthology.orchestra.run.trial085.stderr:dumped all 2026-02-21T00:05:14.959 INFO:teuthology.orchestra.run.trial085.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2026-02-21T00:05:14.278322+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":2492,"kb_used_data":2492,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156497476,"statfs":{"total":160255967232,"available":160253415424,"internally_reserved":0,"allocated":2551808,"data_stored":2551808,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.308895+0000","last_change":"2026-02-21T00:05:13.313294+0000","last_active":"2026-02-21T00:05:13.308895+0000","last_peered":"2026-02-21T00:05:13.308895+0000","last_clean":"2026-02-21T00:05:13.308895+0000","last_became_active":"2026-02-21T00:05:11.320227+0000","last_became_peered":"2026-02-21T00:05:11.320227+0000","last_unstale":"2026-02-21T00:05:13.308895+0000","last_undegraded":"2026-02-21T00:05:13.308895+0000","last_fullsized":"2026-02-21T00:05:13.308895+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_clean_scrub_stamp":"2026-02-21T00:05:10.282364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.311236+0000","last_change":"2026-02-21T00:05:13.314673+0000","last_active":"2026-02-21T00:05:13.311236+0000","last_peered":"2026-02-21T00:05:13.311236+0000","last_clean":"2026-02-21T00:05:13.311236+0000","last_became_active":"2026-02-21T00:05:11.319864+0000","last_became_peered":"2026-02-21T00:05:11.319864+0000","last_unstale":"2026-02-21T00:05:13.311236+0000","last_undegraded":"2026-02-21T00:05:13.311236+0000","last_fullsized":"2026-02-21T00:05:13.311236+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_clean_scrub_stamp":"2026-02-21T00:05:10.282364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.311144+0000","last_change":"2026-02-21T00:05:13.315561+0000","last_active":"2026-02-21T00:05:13.311144+0000","last_peered":"2026-02-21T00:05:13.311144+0000","last_clean":"2026-02-21T00:05:13.311144+0000","last_became_active":"2026-02-21T00:05:11.326559+0000","last_became_peered":"2026-02-21T00:05:11.326559+0000","last_unstale":"2026-02-21T00:05:13.311144+0000","last_undegraded":"2026-02-21T00:05:13.311144+0000","last_fullsized":"2026-02-21T00:05:13.311144+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_clean_scrub_stamp":"2026-02-21T00:05:10.282364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.311570+0000","last_change":"2026-02-21T00:05:13.314396+0000","last_active":"2026-02-21T00:05:13.311570+0000","last_peered":"2026-02-21T00:05:13.311570+0000","last_clean":"2026-02-21T00:05:13.311570+0000","last_became_active":"2026-02-21T00:05:11.322884+0000","last_became_peered":"2026-02-21T00:05:11.322884+0000","last_unstale":"2026-02-21T00:05:13.311570+0000","last_undegraded":"2026-02-21T00:05:13.311570+0000","last_fullsized":"2026-02-21T00:05:13.311570+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_clean_scrub_stamp":"2026-02-21T00:05:10.282364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.311310+0000","last_change":"2026-02-21T00:05:13.311310+0000","last_active":"2026-02-21T00:05:13.311310+0000","last_peered":"2026-02-21T00:05:13.311310+0000","last_clean":"2026-02-21T00:05:13.311310+0000","last_became_active":"2026-02-21T00:05:11.335291+0000","last_became_peered":"2026-02-21T00:05:11.335291+0000","last_unstale":"2026-02-21T00:05:13.311310+0000","last_undegraded":"2026-02-21T00:05:13.311310+0000","last_fullsized":"2026-02-21T00:05:13.311310+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_clean_scrub_stamp":"2026-02-21T00:05:10.282364+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.308630+0000","last_change":"2026-02-21T00:05:13.311231+0000","last_active":"2026-02-21T00:05:13.308630+0000","last_peered":"2026-02-21T00:05:13.308630+0000","last_clean":"2026-02-21T00:05:13.308630+0000","last_became_active":"2026-02-21T00:05:11.325696+0000","last_became_peered":"2026-02-21T00:05:11.325696+0000","last_unstale":"2026-02-21T00:05:13.308630+0000","last_undegraded":"2026-02-21T00:05:13.308630+0000","last_fullsized":"2026-02-21T00:05:13.308630+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_clean_scrub_stamp":"2026-02-21T00:05:10.282364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.306811+0000","last_change":"2026-02-21T00:05:13.310120+0000","last_active":"2026-02-21T00:05:13.306811+0000","last_peered":"2026-02-21T00:05:13.306811+0000","last_clean":"2026-02-21T00:05:13.306811+0000","last_became_active":"2026-02-21T00:05:11.327110+0000","last_became_peered":"2026-02-21T00:05:11.327110+0000","last_unstale":"2026-02-21T00:05:13.306811+0000","last_undegraded":"2026-02-21T00:05:13.306811+0000","last_fullsized":"2026-02-21T00:05:13.306811+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_clean_scrub_stamp":"2026-02-21T00:05:10.282364+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.311231+0000","last_change":"2026-02-21T00:05:13.315577+0000","last_active":"2026-02-21T00:05:13.311231+0000","last_peered":"2026-02-21T00:05:13.311231+0000","last_clean":"2026-02-21T00:05:13.311231+0000","last_became_active":"2026-02-21T00:05:11.323002+0000","last_became_peered":"2026-02-21T00:05:11.323002+0000","last_unstale":"2026-02-21T00:05:13.311231+0000","last_undegraded":"2026-02-21T00:05:13.311231+0000","last_fullsized":"2026-02-21T00:05:13.311231+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:10.282364+0000","last_clean_scrub_stamp":"2026-02-21T00:05:10.282364+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:13.308718+0000","last_change":"2026-02-21T00:05:07.308443+0000","last_active":"2026-02-21T00:05:13.308718+0000","last_peered":"2026-02-21T00:05:13.308718+0000","last_clean":"2026-02-21T00:05:13.308718+0000","last_became_active":"2026-02-21T00:05:07.301117+0000","last_became_peered":"2026-02-21T00:05:07.301117+0000","last_unstale":"2026-02-21T00:05:13.308718+0000","last_undegraded":"2026-02-21T00:05:13.308718+0000","last_fullsized":"2026-02-21T00:05:13.308718+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:06.271075+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:06.271075+0000","last_clean_scrub_stamp":"2026-02-21T00:05:06.271075+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253366272,"internally_reserved":0,"allocated":2600960,"data_stored":2600960,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1116,"kb_used_data":1116,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165540,"statfs":{"total":53418655744,"available":53417512960,"internally_reserved":0,"allocated":1142784,"data_stored":1142784,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":10,"seq":42949672968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":308,"kb_used_data":308,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166348,"statfs":{"total":53418655744,"available":53418340352,"internally_reserved":0,"allocated":315392,"data_stored":315392,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1068,"kb_used_data":1068,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165588,"statfs":{"total":53418655744,"available":53417562112,"internally_reserved":0,"allocated":1093632,"data_stored":1093632,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417553920,"internally_reserved":0,"allocated":1101824,"data_stored":1101824,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417480192,"internally_reserved":0,"allocated":1175552,"data_stored":1175552,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418332160,"internally_reserved":0,"allocated":323584,"data_stored":323584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T00:05:14.960 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-21T00:05:14.961 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-21T00:05:14.961 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-21T00:05:14.961 DEBUG:teuthology.orchestra.run.trial085:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-21T00:05:15.036 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:15.032+0000 7fdee52cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:15.037 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:05:15.035+0000 7fdee52cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:15.172 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:05:15.173 INFO:teuthology.orchestra.run.trial085.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-21T00:05:15.182 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-21T00:05:15.183 INFO:teuthology.run_tasks:Running task ssh_keys... 2026-02-21T00:05:15.252 DEBUG:teuthology.orchestra.run.trial085:> echo 'StrictHostKeyChecking no 2026-02-21T00:05:15.252 DEBUG:teuthology.orchestra.run.trial085:> ' > /home/ubuntu/.ssh/config && echo 'UserKnownHostsFile ' /dev/null >> /home/ubuntu/.ssh/config && chmod 600 /home/ubuntu/.ssh/config 2026-02-21T00:05:15.268 INFO:teuthology.task.ssh_keys:generated public key AAAAB3NzaC1yc2EAAAADAQABAAABAQCYdTZRjW1gXLAmv2Manr5cN4UMVgkeBh3hnftZboxD/q1x1gFr343alWTKN6a39X3Gc2jTMR1n6+Qx1spL+sbmwMCrLqkbaVSk8SeVClqA1++sBwKNHIhyVWLSCrl1uegI+MhCoC2Uy0HS23MxkdOgivt9h7Sjo125SakTQD3U79DNM7VvPVy58W57HxueJEFbcXD2Schi/Ro4lrT0sGWCet3R1cwagZsjW4OJUh9uY8moBsL68+u7y51Lc38ExGVAu6g9mYz1BLXStcX82ZfHY0qjJs1nYZ6WjZSVYWjwlbt7I14YDvS91oOJ5ROATn6LL3jJb8m7f4Dr61tGvFY1 2026-02-21T00:05:15.268 INFO:teuthology.task.ssh_keys:pushing keys to trial085.front.sepia.ceph.com for ubuntu 2026-02-21T00:05:15.268 DEBUG:teuthology.orchestra.run.trial085:> rm -f -- /home/ubuntu/.ssh/id_rsa 2026-02-21T00:05:15.324 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:05:15.324 DEBUG:teuthology.orchestra.run.trial085:> dd of=/home/ubuntu/.ssh/id_rsa 2026-02-21T00:05:15.324 DEBUG:teuthology.orchestra.run.trial085:> chmod 0500 /home/ubuntu/.ssh/id_rsa 2026-02-21T00:05:15.380 DEBUG:teuthology.orchestra.run.trial085:> rm -f -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-21T00:05:15.436 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:05:15.436 DEBUG:teuthology.orchestra.run.trial085:> dd of=/home/ubuntu/.ssh/id_rsa.pub 2026-02-21T00:05:15.492 DEBUG:teuthology.orchestra.run.trial085:> cp -v -a /home/ubuntu/.ssh/authorized_keys /home/ubuntu/.ssh/authorized_keys_2026-02-21_00:05:15:492563 2026-02-21T00:05:15.550 INFO:teuthology.orchestra.run.trial085.stdout:'/home/ubuntu/.ssh/authorized_keys' -> '/home/ubuntu/.ssh/authorized_keys_2026-02-21_00:05:15:492563' 2026-02-21T00:05:15.550 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:05:15.550 DEBUG:teuthology.orchestra.run.trial085:> dd of=/home/ubuntu/.ssh/authorized_keys conv=notrunc oflag=append 2026-02-21T00:05:15.606 INFO:teuthology.run_tasks:Running task cbt... 2026-02-21T00:05:15.636 INFO:tasks.cbt:cbt configuration is {'cluster': {'user': 'ubuntu', 'head': 'ubuntu@trial085.front.sepia.ceph.com', 'osds': ['ubuntu@trial085.front.sepia.ceph.com'], 'mons': ['ubuntu@trial085.front.sepia.ceph.com'], 'clients': ['ubuntu@trial085.front.sepia.ceph.com'], 'rgws': {'ubuntu@trial085.front.sepia.ceph.com': None}, 'osds_per_node': 3, 'rebuild_every_test': False, 'use_existing': True, 'is_teuthology': True, 'iterations': 1, 'tmp_dir': '/tmp/cbt', 'pool_profiles': {'rbd': {'pg_size': 128, 'pgp_size': 128, 'replication': 3}}, 'pid_dir': '/var/run/ceph'}, 'benchmarks': {'librbdfio': {'iodepth': [32], 'log_avg_msec': 100, 'mode': ['randwrite'], 'norandommap': True, 'op_size': [4194304], 'osd_ra': [4096], 'pool_profile': 'rbd', 'procs_per_volume': [1], 'time': 60, 'vol_size': 4096, 'volumes_per_client': [2], 'cmd_path': '/home/ubuntu/cephtest/fio/fio'}}, 'client_endpoints': None, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}} 2026-02-21T00:05:15.637 DEBUG:teuthology.orchestra.run.trial085:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/cbt 2026-02-21T00:05:15.667 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:05:15.667 DEBUG:teuthology.orchestra.run.trial085:> dd of=/home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2026-02-21T00:05:15.719 DEBUG:teuthology.orchestra.run.trial085:> git clone --depth 1 -b master https://github.com/ceph/cbt.git /home/ubuntu/cephtest/cbt 2026-02-21T00:05:15.782 INFO:teuthology.orchestra.run.trial085.stderr:Cloning into '/home/ubuntu/cephtest/cbt'... 2026-02-21T00:05:16.017 INFO:tasks.cbt:Installing collectl 2026-02-21T00:05:16.018 DEBUG:teuthology.orchestra.run.trial085:> sudo mkdir collectl && cd collectl && sudo wget https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download -O collectl.tar.gz && sudo tar -xvf collectl.tar.gz && cd collectl-4.3.1 && sudo ./INSTALL 2026-02-21T00:05:16.094 INFO:teuthology.orchestra.run.trial085.stderr:--2026-02-21 00:05:16-- https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download 2026-02-21T00:05:16.152 INFO:teuthology.orchestra.run.trial085.stderr:Resolving sourceforge.net (sourceforge.net)... 104.18.12.149, 104.18.13.149, 2606:4700::6812:c95, ... 2026-02-21T00:05:16.155 INFO:teuthology.orchestra.run.trial085.stderr:Connecting to sourceforge.net (sourceforge.net)|104.18.12.149|:443... connected. 2026-02-21T00:05:16.357 INFO:teuthology.orchestra.run.trial085.stderr:HTTP request sent, awaiting response... 302 Found 2026-02-21T00:05:16.358 INFO:teuthology.orchestra.run.trial085.stderr:Location: https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABpmPa8I1imr5S7hpXC7oKNpY88G4lyKtKRb-w0JN2d9s63BELG9hkaq3CEcrkYyRjQnxofVzwA_NqzUPK44e_R07RyDQ%3D%3D&use_mirror=gigenet&r= [following] 2026-02-21T00:05:16.358 INFO:teuthology.orchestra.run.trial085.stderr:--2026-02-21 00:05:16-- https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABpmPa8I1imr5S7hpXC7oKNpY88G4lyKtKRb-w0JN2d9s63BELG9hkaq3CEcrkYyRjQnxofVzwA_NqzUPK44e_R07RyDQ%3D%3D&use_mirror=gigenet&r= 2026-02-21T00:05:16.401 INFO:teuthology.orchestra.run.trial085.stderr:Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 104.18.13.149, 104.18.12.149, 2606:4700::6812:c95, ... 2026-02-21T00:05:16.404 INFO:teuthology.orchestra.run.trial085.stderr:Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|104.18.13.149|:443... connected. 2026-02-21T00:05:16.576 INFO:teuthology.orchestra.run.trial085.stderr:HTTP request sent, awaiting response... 302 Found 2026-02-21T00:05:16.576 INFO:teuthology.orchestra.run.trial085.stderr:Location: https://gigenet.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 [following] 2026-02-21T00:05:16.577 INFO:teuthology.orchestra.run.trial085.stderr:--2026-02-21 00:05:16-- https://gigenet.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 2026-02-21T00:05:16.583 INFO:teuthology.orchestra.run.trial085.stderr:Resolving gigenet.dl.sourceforge.net (gigenet.dl.sourceforge.net)... 69.65.16.141 2026-02-21T00:05:16.602 INFO:teuthology.orchestra.run.trial085.stderr:Connecting to gigenet.dl.sourceforge.net (gigenet.dl.sourceforge.net)|69.65.16.141|:443... connected. 2026-02-21T00:05:16.746 INFO:teuthology.orchestra.run.trial085.stderr:HTTP request sent, awaiting response... 200 OK 2026-02-21T00:05:16.746 INFO:teuthology.orchestra.run.trial085.stderr:Length: 620693 (606K) [application/x-gzip] 2026-02-21T00:05:16.746 INFO:teuthology.orchestra.run.trial085.stderr:Saving to: ‘collectl.tar.gz’ 2026-02-21T00:05:16.746 INFO:teuthology.orchestra.run.trial085.stderr: 2026-02-21T00:05:16.765 INFO:teuthology.orchestra.run.trial085.stderr: 0K .......... .......... .......... .......... .......... 8% 2.46M 0s 2026-02-21T00:05:16.784 INFO:teuthology.orchestra.run.trial085.stderr: 50K .......... .......... .......... .......... .......... 16% 2.53M 0s 2026-02-21T00:05:16.785 INFO:teuthology.orchestra.run.trial085.stderr: 100K .......... .......... .......... .......... .......... 24% 97.5M 0s 2026-02-21T00:05:16.808 INFO:teuthology.orchestra.run.trial085.stderr: 150K .......... .......... .......... .......... .......... 32% 2.59M 0s 2026-02-21T00:05:16.808 INFO:teuthology.orchestra.run.trial085.stderr: 200K .......... .......... .......... .......... .......... 41% 126M 0s 2026-02-21T00:05:16.808 INFO:teuthology.orchestra.run.trial085.stderr: 250K .......... .......... .......... .......... .......... 49% 127M 0s 2026-02-21T00:05:16.808 INFO:teuthology.orchestra.run.trial085.stderr: 300K .......... .......... .......... .......... .......... 57% 98.1M 0s 2026-02-21T00:05:16.827 INFO:teuthology.orchestra.run.trial085.stderr: 350K .......... .......... .......... .......... .......... 65% 2.67M 0s 2026-02-21T00:05:16.827 INFO:teuthology.orchestra.run.trial085.stderr: 400K .......... .......... .......... .......... .......... 74% 130M 0s 2026-02-21T00:05:16.827 INFO:teuthology.orchestra.run.trial085.stderr: 450K .......... .......... .......... .......... .......... 82% 95.9M 0s 2026-02-21T00:05:16.827 INFO:teuthology.orchestra.run.trial085.stderr: 500K .......... .......... .......... .......... .......... 90% 129M 0s 2026-02-21T00:05:16.828 INFO:teuthology.orchestra.run.trial085.stderr: 550K .......... .......... .......... .......... .......... 98% 125M 0s 2026-02-21T00:05:16.828 INFO:teuthology.orchestra.run.trial085.stderr: 600K ...... 100% 11.4T=0.08s 2026-02-21T00:05:16.828 INFO:teuthology.orchestra.run.trial085.stderr: 2026-02-21T00:05:16.828 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21 00:05:16 (7.42 MB/s) - ‘collectl.tar.gz’ saved [620693/620693] 2026-02-21T00:05:16.828 INFO:teuthology.orchestra.run.trial085.stderr: 2026-02-21T00:05:16.841 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/ 2026-02-21T00:05:16.841 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/collectl.conf 2026-02-21T00:05:16.841 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/ 2026-02-21T00:05:16.841 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/SlowProc.html 2026-02-21T00:05:16.841 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/BuddyInfo.html 2026-02-21T00:05:16.841 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/WhySummary.html 2026-02-21T00:05:16.841 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Socket.html 2026-02-21T00:05:16.841 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/colmux.html 2026-02-21T00:05:16.841 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Tutorial.html 2026-02-21T00:05:16.841 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Gexpr.html 2026-02-21T00:05:16.841 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Import.html 2026-02-21T00:05:16.841 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/style.css 2026-02-21T00:05:16.842 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Ganglia3.jpg 2026-02-21T00:05:16.842 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Ganglia2.jpg 2026-02-21T00:05:16.842 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/FileNaming.html 2026-02-21T00:05:16.842 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/TheMath.html 2026-02-21T00:05:16.842 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Environmental.html 2026-02-21T00:05:16.842 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Process.html 2026-02-21T00:05:16.842 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/OutputFiles.html 2026-02-21T00:05:16.842 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/HiResTime.html 2026-02-21T00:05:16.842 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/OutputFormats.html 2026-02-21T00:05:16.842 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/slub.jpg 2026-02-21T00:05:16.844 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Lustre.html 2026-02-21T00:05:16.844 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Documentation.html 2026-02-21T00:05:16.844 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Inodes.html 2026-02-21T00:05:16.844 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Graphite.html 2026-02-21T00:05:16.844 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/InputFiles.html 2026-02-21T00:05:16.844 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Export.html 2026-02-21T00:05:16.844 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Interrupts.html 2026-02-21T00:05:16.844 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Startup.html 2026-02-21T00:05:16.844 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Performance.html 2026-02-21T00:05:16.844 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Messages.html 2026-02-21T00:05:16.844 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Logging.html 2026-02-21T00:05:16.844 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Data-detail.html 2026-02-21T00:05:16.844 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/CPUs.html 2026-02-21T00:05:16.844 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Matrix.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Data-brief.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/SlabInfo.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Misc.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Examples.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/OpenStack.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Plotfiles.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/ColmuxCPU.jpg 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Tutorial-Lustre.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Data-verbose.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Data.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Exceptions.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/RunningAsAService.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/FAQ-collectl.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Ganglia.jpg 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/OperationalModes.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Playback.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Hello.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Sockets.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Network.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Infiniband.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Memory.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/ColmuxLustre.jpg 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/Disks.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/docs/NfsInfo.html 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/service/ 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/service/collectl.service 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/initd/ 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/initd/collectl-suse 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/initd/collectl-debian 2026-02-21T00:05:16.845 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/initd/collectl-generic 2026-02-21T00:05:16.846 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/initd/collectl 2026-02-21T00:05:16.846 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/misc.ph 2026-02-21T00:05:16.846 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/gexpr.ph 2026-02-21T00:05:16.846 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/statsd.ph 2026-02-21T00:05:16.846 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/envrules.std 2026-02-21T00:05:16.846 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/hello.ph 2026-02-21T00:05:16.846 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/vmsum.ph 2026-02-21T00:05:16.846 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/proctree.ph 2026-02-21T00:05:16.846 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/README-WINDOWS 2026-02-21T00:05:16.846 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/formatit.ph 2026-02-21T00:05:16.846 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/colmux 2026-02-21T00:05:16.847 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/INSTALL 2026-02-21T00:05:16.847 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/graphite.ph 2026-02-21T00:05:16.847 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/RELEASE-collectl 2026-02-21T00:05:16.847 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/vmstat.ph 2026-02-21T00:05:16.847 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/README 2026-02-21T00:05:16.847 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/man1/ 2026-02-21T00:05:16.847 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/man1/collectl.1 2026-02-21T00:05:16.847 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/man1/colmux.1 2026-02-21T00:05:16.847 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/lexpr.ph 2026-02-21T00:05:16.847 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/client.pl 2026-02-21T00:05:16.847 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/collectl 2026-02-21T00:05:16.847 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/COPYING 2026-02-21T00:05:16.847 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/vnet.ph 2026-02-21T00:05:16.847 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/GPL 2026-02-21T00:05:16.847 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/UNINSTALL 2026-02-21T00:05:16.847 INFO:teuthology.orchestra.run.trial085.stdout:collectl-4.3.1/ARTISTIC 2026-02-21T00:05:16.890 INFO:teuthology.orchestra.run.trial085.stderr:./INSTALL: line 90: chkconfig: command not found 2026-02-21T00:05:16.891 DEBUG:teuthology.orchestra.run.trial085:> sudo yum -y install librbd-devel pdsh pdsh-rcmd-ssh perf 2026-02-21T00:05:17.241 INFO:teuthology.orchestra.run.trial085.stdout:Last metadata expiration check: 0:00:57 ago on Sat 21 Feb 2026 12:04:20 AM UTC. 2026-02-21T00:05:17.327 INFO:teuthology.orchestra.run.trial085.stdout:Dependencies resolved. 2026-02-21T00:05:17.328 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-21T00:05:17.328 INFO:teuthology.orchestra.run.trial085.stdout: Package Arch Version Repository Size 2026-02-21T00:05:17.328 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-21T00:05:17.328 INFO:teuthology.orchestra.run.trial085.stdout:Installing: 2026-02-21T00:05:17.329 INFO:teuthology.orchestra.run.trial085.stdout: librbd-devel x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 24 k 2026-02-21T00:05:17.329 INFO:teuthology.orchestra.run.trial085.stdout: pdsh x86_64 2.34-7.el9 epel 118 k 2026-02-21T00:05:17.329 INFO:teuthology.orchestra.run.trial085.stdout: pdsh-rcmd-ssh x86_64 2.34-7.el9 epel 13 k 2026-02-21T00:05:17.329 INFO:teuthology.orchestra.run.trial085.stdout: perf x86_64 5.14.0-681.el9 appstream 3.2 M 2026-02-21T00:05:17.329 INFO:teuthology.orchestra.run.trial085.stdout:Installing dependencies: 2026-02-21T00:05:17.329 INFO:teuthology.orchestra.run.trial085.stdout: libradospp-devel x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 31 k 2026-02-21T00:05:17.329 INFO:teuthology.orchestra.run.trial085.stdout: libtraceevent x86_64 1.8.4-2.el9 baseos 290 k 2026-02-21T00:05:17.329 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:05:17.329 INFO:teuthology.orchestra.run.trial085.stdout:Transaction Summary 2026-02-21T00:05:17.329 INFO:teuthology.orchestra.run.trial085.stdout:================================================================================ 2026-02-21T00:05:17.329 INFO:teuthology.orchestra.run.trial085.stdout:Install 6 Packages 2026-02-21T00:05:17.330 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:05:17.330 INFO:teuthology.orchestra.run.trial085.stdout:Total download size: 3.7 M 2026-02-21T00:05:17.330 INFO:teuthology.orchestra.run.trial085.stdout:Installed size: 14 M 2026-02-21T00:05:17.330 INFO:teuthology.orchestra.run.trial085.stdout:Downloading Packages: 2026-02-21T00:05:18.176 INFO:teuthology.orchestra.run.trial085.stdout:(1/6): libradospp-devel-20.3.0-5343.gbcdd429b.e 89 kB/s | 31 kB 00:00 2026-02-21T00:05:18.177 INFO:teuthology.orchestra.run.trial085.stdout:(2/6): librbd-devel-20.3.0-5343.gbcdd429b.el9.x 68 kB/s | 24 kB 00:00 2026-02-21T00:05:18.231 INFO:teuthology.orchestra.run.trial085.stdout:(3/6): pdsh-2.34-7.el9.x86_64.rpm 2.1 MB/s | 118 kB 00:00 2026-02-21T00:05:18.239 INFO:teuthology.orchestra.run.trial085.stdout:(4/6): pdsh-rcmd-ssh-2.34-7.el9.x86_64.rpm 1.5 MB/s | 13 kB 00:00 2026-02-21T00:05:18.472 INFO:teuthology.orchestra.run.trial085.stdout:(5/6): libtraceevent-1.8.4-2.el9.x86_64.rpm 448 kB/s | 290 kB 00:00 2026-02-21T00:05:22.285 INFO:teuthology.orchestra.run.trial085.stdout:(6/6): perf-5.14.0-681.el9.x86_64.rpm 795 kB/s | 3.2 MB 00:04 2026-02-21T00:05:22.288 INFO:teuthology.orchestra.run.trial085.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:05:22.288 INFO:teuthology.orchestra.run.trial085.stdout:Total 755 kB/s | 3.7 MB 00:04 2026-02-21T00:05:22.312 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction check 2026-02-21T00:05:22.321 INFO:teuthology.orchestra.run.trial085.stdout:Transaction check succeeded. 2026-02-21T00:05:22.321 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction test 2026-02-21T00:05:22.377 INFO:teuthology.orchestra.run.trial085.stdout:Transaction test succeeded. 2026-02-21T00:05:22.378 INFO:teuthology.orchestra.run.trial085.stdout:Running transaction 2026-02-21T00:05:22.450 INFO:teuthology.orchestra.run.trial085.stdout: Preparing : 1/1 2026-02-21T00:05:22.456 INFO:teuthology.orchestra.run.trial085.stdout: Installing : pdsh-rcmd-ssh-2.34-7.el9.x86_64 1/6 2026-02-21T00:05:22.496 INFO:teuthology.orchestra.run.trial085.stdout: Installing : pdsh-2.34-7.el9.x86_64 2/6 2026-02-21T00:05:22.501 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libtraceevent-1.8.4-2.el9.x86_64 3/6 2026-02-21T00:05:22.503 INFO:teuthology.orchestra.run.trial085.stdout: Installing : libradospp-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 4/6 2026-02-21T00:05:22.664 INFO:teuthology.orchestra.run.trial085.stdout: Installing : librbd-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 5/6 2026-02-21T00:05:22.685 INFO:teuthology.orchestra.run.trial085.stdout: Installing : perf-5.14.0-681.el9.x86_64 6/6 2026-02-21T00:05:22.790 INFO:teuthology.orchestra.run.trial085.stdout: Running scriptlet: perf-5.14.0-681.el9.x86_64 6/6 2026-02-21T00:05:22.790 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libradospp-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/6 2026-02-21T00:05:22.790 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : librbd-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/6 2026-02-21T00:05:22.790 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : libtraceevent-1.8.4-2.el9.x86_64 3/6 2026-02-21T00:05:22.790 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : perf-5.14.0-681.el9.x86_64 4/6 2026-02-21T00:05:22.790 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : pdsh-2.34-7.el9.x86_64 5/6 2026-02-21T00:05:22.850 INFO:teuthology.orchestra.run.trial085.stdout: Verifying : pdsh-rcmd-ssh-2.34-7.el9.x86_64 6/6 2026-02-21T00:05:22.850 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:05:22.850 INFO:teuthology.orchestra.run.trial085.stdout:Installed: 2026-02-21T00:05:22.850 INFO:teuthology.orchestra.run.trial085.stdout: libradospp-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:05:22.850 INFO:teuthology.orchestra.run.trial085.stdout: librbd-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:05:22.850 INFO:teuthology.orchestra.run.trial085.stdout: libtraceevent-1.8.4-2.el9.x86_64 2026-02-21T00:05:22.850 INFO:teuthology.orchestra.run.trial085.stdout: pdsh-2.34-7.el9.x86_64 2026-02-21T00:05:22.850 INFO:teuthology.orchestra.run.trial085.stdout: pdsh-rcmd-ssh-2.34-7.el9.x86_64 2026-02-21T00:05:22.851 INFO:teuthology.orchestra.run.trial085.stdout: perf-5.14.0-681.el9.x86_64 2026-02-21T00:05:22.851 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:05:22.851 INFO:teuthology.orchestra.run.trial085.stdout:Complete! 2026-02-21T00:05:22.880 DEBUG:teuthology.orchestra.run.trial085:> python3 -m pip install -r /home/ubuntu/cephtest/cbt/requirements.txt 2026-02-21T00:05:23.018 INFO:teuthology.orchestra.run.trial085.stdout:Defaulting to user installation because normal site-packages is not writeable 2026-02-21T00:05:23.028 INFO:teuthology.orchestra.run.trial085.stdout:Requirement already satisfied: pyyaml in /usr/lib64/python3.9/site-packages (from -r /home/ubuntu/cephtest/cbt/requirements.txt (line 1)) (5.4.1) 2026-02-21T00:05:23.498 INFO:teuthology.orchestra.run.trial085.stdout:Collecting lxml 2026-02-21T00:05:23.527 INFO:teuthology.orchestra.run.trial085.stdout: Downloading lxml-6.0.2-cp39-cp39-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl (5.3 MB) 2026-02-21T00:05:23.880 INFO:teuthology.orchestra.run.trial085.stdout:Collecting matplotlib 2026-02-21T00:05:23.885 INFO:teuthology.orchestra.run.trial085.stdout: Downloading matplotlib-3.9.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.3 MB) 2026-02-21T00:05:24.383 INFO:teuthology.orchestra.run.trial085.stdout:Collecting pillow>=8 2026-02-21T00:05:24.407 INFO:teuthology.orchestra.run.trial085.stdout: Downloading pillow-11.3.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (6.6 MB) 2026-02-21T00:05:24.480 INFO:teuthology.orchestra.run.trial085.stdout:Requirement already satisfied: python-dateutil>=2.7 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (2.9.0.post0) 2026-02-21T00:05:24.515 INFO:teuthology.orchestra.run.trial085.stdout:Collecting importlib-resources>=3.2.0 2026-02-21T00:05:24.520 INFO:teuthology.orchestra.run.trial085.stdout: Downloading importlib_resources-6.5.2-py3-none-any.whl (37 kB) 2026-02-21T00:05:24.681 INFO:teuthology.orchestra.run.trial085.stdout:Collecting kiwisolver>=1.3.1 2026-02-21T00:05:24.685 INFO:teuthology.orchestra.run.trial085.stdout: Downloading kiwisolver-1.4.7-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB) 2026-02-21T00:05:24.712 INFO:teuthology.orchestra.run.trial085.stdout:Collecting cycler>=0.10 2026-02-21T00:05:24.716 INFO:teuthology.orchestra.run.trial085.stdout: Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB) 2026-02-21T00:05:24.717 INFO:teuthology.orchestra.run.trial085.stdout:Requirement already satisfied: packaging>=20.0 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (20.9) 2026-02-21T00:05:24.717 INFO:teuthology.orchestra.run.trial085.stdout:Requirement already satisfied: numpy>=1.23 in /usr/lib64/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (1.23.5) 2026-02-21T00:05:25.071 INFO:teuthology.orchestra.run.trial085.stdout:Collecting fonttools>=4.22.0 2026-02-21T00:05:25.075 INFO:teuthology.orchestra.run.trial085.stdout: Downloading fonttools-4.60.2-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.8 MB) 2026-02-21T00:05:25.259 INFO:teuthology.orchestra.run.trial085.stdout:Collecting contourpy>=1.0.1 2026-02-21T00:05:25.263 INFO:teuthology.orchestra.run.trial085.stdout: Downloading contourpy-1.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (321 kB) 2026-02-21T00:05:25.269 INFO:teuthology.orchestra.run.trial085.stdout:Requirement already satisfied: pyparsing>=2.3.1 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (2.4.7) 2026-02-21T00:05:25.353 INFO:teuthology.orchestra.run.trial085.stdout:Collecting zipp>=3.1.0 2026-02-21T00:05:25.358 INFO:teuthology.orchestra.run.trial085.stdout: Downloading zipp-3.23.0-py3-none-any.whl (10 kB) 2026-02-21T00:05:25.369 INFO:teuthology.orchestra.run.trial085.stdout:Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (1.15.0) 2026-02-21T00:05:25.485 INFO:teuthology.orchestra.run.trial085.stdout:Installing collected packages: zipp, pillow, kiwisolver, importlib-resources, fonttools, cycler, contourpy, matplotlib, lxml 2026-02-21T00:05:26.595 INFO:teuthology.orchestra.run.trial085.stdout:Successfully installed contourpy-1.3.0 cycler-0.12.1 fonttools-4.60.2 importlib-resources-6.5.2 kiwisolver-1.4.7 lxml-6.0.2 matplotlib-3.9.4 pillow-11.3.0 zipp-3.23.0 2026-02-21T00:05:26.728 INFO:tasks.cbt:benchmark: librbdfio 2026-02-21T00:05:26.728 DEBUG:teuthology.orchestra.run.trial085:> git clone -b master https://github.com/axboe/fio.git /home/ubuntu/cephtest/fio 2026-02-21T00:05:26.745 INFO:teuthology.orchestra.run.trial085.stderr:Cloning into '/home/ubuntu/cephtest/fio'... 2026-02-21T00:05:27.535 DEBUG:teuthology.orchestra.run.trial085:> cd /home/ubuntu/cephtest/fio && ./configure && make 2026-02-21T00:05:27.706 INFO:teuthology.orchestra.run.trial085.stdout:Operating system Linux 2026-02-21T00:05:27.706 INFO:teuthology.orchestra.run.trial085.stdout:CPU x86_64 2026-02-21T00:05:27.706 INFO:teuthology.orchestra.run.trial085.stdout:Big endian no 2026-02-21T00:05:27.706 INFO:teuthology.orchestra.run.trial085.stdout:Compiler gcc 2026-02-21T00:05:27.706 INFO:teuthology.orchestra.run.trial085.stdout:Cross compile no 2026-02-21T00:05:27.706 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:05:27.706 INFO:teuthology.orchestra.run.trial085.stdout:Static build no 2026-02-21T00:05:27.754 INFO:teuthology.orchestra.run.trial085.stdout:Wordsize 64 2026-02-21T00:05:27.760 INFO:teuthology.orchestra.run.trial085.stdout:zlib no 2026-02-21T00:05:27.771 INFO:teuthology.orchestra.run.trial085.stdout:fcntl(F_FULLFSYNC) no 2026-02-21T00:05:27.782 INFO:teuthology.orchestra.run.trial085.stdout:Linux AIO support no 2026-02-21T00:05:27.783 INFO:teuthology.orchestra.run.trial085.stdout:Linux AIO support rw flags no 2026-02-21T00:05:27.806 INFO:teuthology.orchestra.run.trial085.stdout:POSIX AIO support yes 2026-02-21T00:05:27.807 INFO:teuthology.orchestra.run.trial085.stdout:POSIX AIO support needs -lrt no 2026-02-21T00:05:27.828 INFO:teuthology.orchestra.run.trial085.stdout:POSIX AIO fsync yes 2026-02-21T00:05:27.849 INFO:teuthology.orchestra.run.trial085.stdout:POSIX pshared support yes 2026-02-21T00:05:27.876 INFO:teuthology.orchestra.run.trial085.stdout:pthread_condattr_setclock() yes 2026-02-21T00:05:27.900 INFO:teuthology.orchestra.run.trial085.stdout:pthread_sigmask() yes 2026-02-21T00:05:27.922 INFO:teuthology.orchestra.run.trial085.stdout:pthread_getaffinity_np() yes 2026-02-21T00:05:27.929 INFO:teuthology.orchestra.run.trial085.stdout:Solaris AIO support no 2026-02-21T00:05:27.951 INFO:teuthology.orchestra.run.trial085.stdout:__sync_fetch_and_add yes 2026-02-21T00:05:27.971 INFO:teuthology.orchestra.run.trial085.stdout:__sync_synchronize yes 2026-02-21T00:05:27.990 INFO:teuthology.orchestra.run.trial085.stdout:__sync_val_compare_and_swap yes 2026-02-21T00:05:27.995 INFO:teuthology.orchestra.run.trial085.stdout:libverbs no 2026-02-21T00:05:28.004 INFO:teuthology.orchestra.run.trial085.stdout:rdmacm no 2026-02-21T00:05:28.025 INFO:teuthology.orchestra.run.trial085.stdout:asprintf() yes 2026-02-21T00:05:28.045 INFO:teuthology.orchestra.run.trial085.stdout:vasprintf() yes 2026-02-21T00:05:28.067 INFO:teuthology.orchestra.run.trial085.stdout:Linux fallocate yes 2026-02-21T00:05:28.088 INFO:teuthology.orchestra.run.trial085.stdout:POSIX fadvise yes 2026-02-21T00:05:28.109 INFO:teuthology.orchestra.run.trial085.stdout:POSIX fallocate yes 2026-02-21T00:05:28.130 INFO:teuthology.orchestra.run.trial085.stdout:sched_setaffinity(3 arg) yes 2026-02-21T00:05:28.131 INFO:teuthology.orchestra.run.trial085.stdout:sched_setaffinity(2 arg) no 2026-02-21T00:05:28.152 INFO:teuthology.orchestra.run.trial085.stdout:clock_gettime yes 2026-02-21T00:05:28.174 INFO:teuthology.orchestra.run.trial085.stdout:CLOCK_MONOTONIC yes 2026-02-21T00:05:28.195 INFO:teuthology.orchestra.run.trial085.stdout:gettimeofday yes 2026-02-21T00:05:28.216 INFO:teuthology.orchestra.run.trial085.stdout:fdatasync yes 2026-02-21T00:05:28.238 INFO:teuthology.orchestra.run.trial085.stdout:pipe() yes 2026-02-21T00:05:28.259 INFO:teuthology.orchestra.run.trial085.stdout:pipe2() yes 2026-02-21T00:05:28.281 INFO:teuthology.orchestra.run.trial085.stdout:pread() yes 2026-02-21T00:05:28.306 INFO:teuthology.orchestra.run.trial085.stdout:sync_file_range yes 2026-02-21T00:05:28.311 INFO:teuthology.orchestra.run.trial085.stdout:ASharedMemory_create no 2026-02-21T00:05:28.319 INFO:teuthology.orchestra.run.trial085.stdout:EXT4 move extent yes 2026-02-21T00:05:28.341 INFO:teuthology.orchestra.run.trial085.stdout:Linux splice(2) yes 2026-02-21T00:05:28.348 INFO:teuthology.orchestra.run.trial085.stdout:libnuma no 2026-02-21T00:05:28.368 INFO:teuthology.orchestra.run.trial085.stdout:strsep yes 2026-02-21T00:05:28.387 INFO:teuthology.orchestra.run.trial085.stdout:strcasestr yes 2026-02-21T00:05:28.396 INFO:teuthology.orchestra.run.trial085.stdout:strlcat no 2026-02-21T00:05:28.417 INFO:teuthology.orchestra.run.trial085.stdout:getopt_long_only() yes 2026-02-21T00:05:28.442 INFO:teuthology.orchestra.run.trial085.stdout:inet_aton yes 2026-02-21T00:05:28.463 INFO:teuthology.orchestra.run.trial085.stdout:socklen_t yes 2026-02-21T00:05:28.484 INFO:teuthology.orchestra.run.trial085.stdout:__thread yes 2026-02-21T00:05:28.505 INFO:teuthology.orchestra.run.trial085.stdout:RUSAGE_THREAD yes 2026-02-21T00:05:28.527 INFO:teuthology.orchestra.run.trial085.stdout:SCHED_IDLE yes 2026-02-21T00:05:28.552 INFO:teuthology.orchestra.run.trial085.stdout:TCP_NODELAY yes 2026-02-21T00:05:28.572 INFO:teuthology.orchestra.run.trial085.stdout:vsock yes 2026-02-21T00:05:28.594 INFO:teuthology.orchestra.run.trial085.stdout:Net engine window_size yes 2026-02-21T00:05:28.618 INFO:teuthology.orchestra.run.trial085.stdout:TCP_MAXSEG yes 2026-02-21T00:05:28.639 INFO:teuthology.orchestra.run.trial085.stdout:RLIMIT_MEMLOCK yes 2026-02-21T00:05:28.662 INFO:teuthology.orchestra.run.trial085.stdout:pwritev/preadv yes 2026-02-21T00:05:28.684 INFO:teuthology.orchestra.run.trial085.stdout:pwritev2/preadv2 yes 2026-02-21T00:05:28.708 INFO:teuthology.orchestra.run.trial085.stdout:IPv6 helpers yes 2026-02-21T00:05:28.719 INFO:teuthology.orchestra.run.trial085.stdout:http engine no 2026-02-21T00:05:28.770 INFO:teuthology.orchestra.run.trial085.stdout:Rados engine yes 2026-02-21T00:05:28.835 INFO:teuthology.orchestra.run.trial085.stdout:Rados Block Device engine yes 2026-02-21T00:05:28.901 INFO:teuthology.orchestra.run.trial085.stdout:rbd_poll yes 2026-02-21T00:05:28.968 INFO:teuthology.orchestra.run.trial085.stdout:rbd_invalidate_cache yes 2026-02-21T00:05:29.032 INFO:teuthology.orchestra.run.trial085.stdout:rbd_encryption_load yes 2026-02-21T00:05:29.053 INFO:teuthology.orchestra.run.trial085.stdout:setvbuf yes 2026-02-21T00:05:29.059 INFO:teuthology.orchestra.run.trial085.stdout:Gluster API engine no 2026-02-21T00:05:29.069 INFO:teuthology.orchestra.run.trial085.stdout:s390_z196_facilities no 2026-02-21T00:05:29.070 INFO:teuthology.orchestra.run.trial085.stdout:HDFS engine no 2026-02-21T00:05:29.090 INFO:teuthology.orchestra.run.trial085.stdout:MTD yes 2026-02-21T00:05:29.095 INFO:teuthology.orchestra.run.trial085.stdout:libpmem no 2026-02-21T00:05:29.095 INFO:teuthology.orchestra.run.trial085.stdout:libpmem1_5 no 2026-02-21T00:05:29.101 INFO:teuthology.orchestra.run.trial085.stdout:libpmem2 no 2026-02-21T00:05:29.101 INFO:teuthology.orchestra.run.trial085.stdout:PMDK dev-dax engine no 2026-02-21T00:05:29.101 INFO:teuthology.orchestra.run.trial085.stdout:PMDK libpmem engine no 2026-02-21T00:05:29.106 INFO:teuthology.orchestra.run.trial085.stdout:DDN's Infinite Memory Engine no 2026-02-21T00:05:29.106 INFO:teuthology.orchestra.run.trial085.stdout:iscsi engine no 2026-02-21T00:05:29.106 INFO:teuthology.orchestra.run.trial085.stdout:NBD engine no 2026-02-21T00:05:29.111 INFO:teuthology.orchestra.run.trial085.stdout:DAOS File System (dfs) Engine no 2026-02-21T00:05:29.117 INFO:teuthology.orchestra.run.trial085.stdout:NFS engine no 2026-02-21T00:05:29.118 INFO:teuthology.orchestra.run.trial085.stdout:lex/yacc for arithmetic no 2026-02-21T00:05:29.138 INFO:teuthology.orchestra.run.trial085.stdout:getmntent yes 2026-02-21T00:05:29.150 INFO:teuthology.orchestra.run.trial085.stdout:getmntinfo no 2026-02-21T00:05:29.179 INFO:teuthology.orchestra.run.trial085.stdout:Static Assert yes 2026-02-21T00:05:29.197 INFO:teuthology.orchestra.run.trial085.stdout:bool yes 2026-02-21T00:05:29.220 INFO:teuthology.orchestra.run.trial085.stdout:strndup yes 2026-02-21T00:05:29.225 INFO:teuthology.orchestra.run.trial085.stdout:Valgrind headers no 2026-02-21T00:05:29.245 INFO:teuthology.orchestra.run.trial085.stdout:Zoned block device support yes 2026-02-21T00:05:29.263 INFO:teuthology.orchestra.run.trial085.stdout:Zoned block device capacity yes 2026-02-21T00:05:29.268 INFO:teuthology.orchestra.run.trial085.stdout:libzbc engine no 2026-02-21T00:05:29.287 INFO:teuthology.orchestra.run.trial085.stdout:NVMe uring command support yes 2026-02-21T00:05:29.299 INFO:teuthology.orchestra.run.trial085.stdout:xnvme engine no 2026-02-21T00:05:29.304 INFO:teuthology.orchestra.run.trial085.stdout:isal no 2026-02-21T00:05:29.305 INFO:teuthology.orchestra.run.trial085.stdout:isal CRC64 no 2026-02-21T00:05:29.317 INFO:teuthology.orchestra.run.trial085.stdout:libblkio engine no 2026-02-21T00:05:29.317 INFO:teuthology.orchestra.run.trial085.stdout:march_armv8_a_crc_crypto no 2026-02-21T00:05:29.317 INFO:teuthology.orchestra.run.trial085.stdout:cuda no 2026-02-21T00:05:29.317 INFO:teuthology.orchestra.run.trial085.stdout:libcufile no 2026-02-21T00:05:29.339 INFO:teuthology.orchestra.run.trial085.stdout:Build march=native yes 2026-02-21T00:05:29.344 INFO:teuthology.orchestra.run.trial085.stdout:CUnit no 2026-02-21T00:05:29.365 INFO:teuthology.orchestra.run.trial085.stdout:__kernel_rwf_t yes 2026-02-21T00:05:29.385 INFO:teuthology.orchestra.run.trial085.stdout:-Wimplicit-fallthrough=2 yes 2026-02-21T00:05:29.406 INFO:teuthology.orchestra.run.trial085.stdout:-Wno-stringop-truncation yes 2026-02-21T00:05:29.426 INFO:teuthology.orchestra.run.trial085.stdout:MADV_HUGEPAGE yes 2026-02-21T00:05:29.447 INFO:teuthology.orchestra.run.trial085.stdout:gettid yes 2026-02-21T00:05:29.470 INFO:teuthology.orchestra.run.trial085.stdout:statx(2)/libc yes 2026-02-21T00:05:29.493 INFO:teuthology.orchestra.run.trial085.stdout:statx(2)/syscall yes 2026-02-21T00:05:29.495 INFO:teuthology.orchestra.run.trial085.stdout:Windows PDB generation no 2026-02-21T00:05:29.516 INFO:teuthology.orchestra.run.trial085.stdout:timerfd_create yes 2026-02-21T00:05:29.517 INFO:teuthology.orchestra.run.trial085.stdout:Consider installing zlib1g-dev (zlib-devel) as some fio features depend on it. 2026-02-21T00:05:29.518 INFO:teuthology.orchestra.run.trial085.stdout:Lib-based ioengines dynamic no 2026-02-21T00:05:29.560 INFO:teuthology.orchestra.run.trial085.stdout:TCMalloc support yes 2026-02-21T00:05:29.561 INFO:teuthology.orchestra.run.trial085.stdout:seed_buckets 4 2026-02-21T00:05:29.607 INFO:teuthology.orchestra.run.trial085.stderr:FIO_VERSION = fio-3.41-110-g1a04e5 2026-02-21T00:05:29.630 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/crc16.o 2026-02-21T00:05:29.665 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/crc32.o 2026-02-21T00:05:29.691 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/crc32c-arm64.o 2026-02-21T00:05:29.737 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/crc32c-intel.o 2026-02-21T00:05:29.768 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/crc32c.o 2026-02-21T00:05:29.795 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/crc64.o 2026-02-21T00:05:29.824 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/crc7.o 2026-02-21T00:05:29.849 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/crct10dif_common.o 2026-02-21T00:05:29.874 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/fnv.o 2026-02-21T00:05:29.904 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/md5.o 2026-02-21T00:05:29.963 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/murmur3.o 2026-02-21T00:05:29.993 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/sha1.o 2026-02-21T00:05:30.090 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/sha256.o 2026-02-21T00:05:30.230 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/sha3.o 2026-02-21T00:05:30.312 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/sha512.o 2026-02-21T00:05:30.379 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/test.o 2026-02-21T00:05:30.472 INFO:teuthology.orchestra.run.trial085.stdout: CC crc/xxhash.o 2026-02-21T00:05:30.528 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/axmap.o 2026-02-21T00:05:30.600 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/bloom.o 2026-02-21T00:05:30.651 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/flist_sort.o 2026-02-21T00:05:30.695 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/gauss.o 2026-02-21T00:05:30.748 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/getrusage.o 2026-02-21T00:05:30.777 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/hweight.o 2026-02-21T00:05:30.805 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/ieee754.o 2026-02-21T00:05:30.839 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/lfsr.o 2026-02-21T00:05:30.899 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/memalign.o 2026-02-21T00:05:30.929 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/memcpy.o 2026-02-21T00:05:31.012 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/mountcheck.o 2026-02-21T00:05:31.042 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/num2str.o 2026-02-21T00:05:31.086 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/output_buffer.o 2026-02-21T00:05:31.120 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/pattern.o 2026-02-21T00:05:31.200 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/prio_tree.o 2026-02-21T00:05:31.270 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/rand.o 2026-02-21T00:05:31.346 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/rbtree.o 2026-02-21T00:05:31.416 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/strntol.o 2026-02-21T00:05:31.449 INFO:teuthology.orchestra.run.trial085.stdout: CC lib/zipf.o 2026-02-21T00:05:31.514 INFO:teuthology.orchestra.run.trial085.stdout: CC gettime.o 2026-02-21T00:05:31.666 INFO:teuthology.orchestra.run.trial085.stdout: CC ioengines.o 2026-02-21T00:05:31.807 INFO:teuthology.orchestra.run.trial085.stdout: CC init.o 2026-02-21T00:05:32.183 INFO:teuthology.orchestra.run.trial085.stdout: CC stat.o 2026-02-21T00:05:32.900 INFO:teuthology.orchestra.run.trial085.stdout: CC log.o 2026-02-21T00:05:32.971 INFO:teuthology.orchestra.run.trial085.stdout: CC time.o 2026-02-21T00:05:33.059 INFO:teuthology.orchestra.run.trial085.stdout: CC filesetup.o 2026-02-21T00:05:33.393 INFO:teuthology.orchestra.run.trial085.stdout: CC eta.o 2026-02-21T00:05:33.555 INFO:teuthology.orchestra.run.trial085.stdout: CC verify.o 2026-02-21T00:05:33.898 INFO:teuthology.orchestra.run.trial085.stdout: CC memory.o 2026-02-21T00:05:33.993 INFO:teuthology.orchestra.run.trial085.stdout: CC io_u.o 2026-02-21T00:05:34.379 INFO:teuthology.orchestra.run.trial085.stdout: CC parse.o 2026-02-21T00:05:34.601 INFO:teuthology.orchestra.run.trial085.stdout: CC fio_sem.o 2026-02-21T00:05:34.665 INFO:teuthology.orchestra.run.trial085.stdout: CC rwlock.o 2026-02-21T00:05:34.719 INFO:teuthology.orchestra.run.trial085.stdout: CC pshared.o 2026-02-21T00:05:34.764 INFO:teuthology.orchestra.run.trial085.stdout: CC options.o 2026-02-21T00:05:35.168 INFO:teuthology.orchestra.run.trial085.stdout: CC fio_shared_sem.o 2026-02-21T00:05:35.202 INFO:teuthology.orchestra.run.trial085.stdout: CC smalloc.o 2026-02-21T00:05:35.308 INFO:teuthology.orchestra.run.trial085.stdout: CC filehash.o 2026-02-21T00:05:35.397 INFO:teuthology.orchestra.run.trial085.stdout: CC profile.o 2026-02-21T00:05:35.465 INFO:teuthology.orchestra.run.trial085.stdout: CC debug.o 2026-02-21T00:05:35.496 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/cpu.o 2026-02-21T00:05:35.574 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/mmap.o 2026-02-21T00:05:35.667 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/sync.o 2026-02-21T00:05:35.787 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/null.o 2026-02-21T00:05:35.858 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/net.o 2026-02-21T00:05:36.066 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/ftruncate.o 2026-02-21T00:05:36.126 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/fileoperations.o 2026-02-21T00:05:36.217 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/exec.o 2026-02-21T00:05:36.304 INFO:teuthology.orchestra.run.trial085.stdout: CC server.o 2026-02-21T00:05:36.660 INFO:teuthology.orchestra.run.trial085.stdout: CC client.o 2026-02-21T00:05:36.938 INFO:teuthology.orchestra.run.trial085.stdout: CC iolog.o 2026-02-21T00:05:37.171 INFO:teuthology.orchestra.run.trial085.stdout: CC backend.o 2026-02-21T00:05:37.586 INFO:teuthology.orchestra.run.trial085.stdout: CC libfio.o 2026-02-21T00:05:37.683 INFO:teuthology.orchestra.run.trial085.stdout: CC flow.o 2026-02-21T00:05:37.749 INFO:teuthology.orchestra.run.trial085.stdout: CC cconv.o 2026-02-21T00:05:37.968 INFO:teuthology.orchestra.run.trial085.stdout: CC gettime-thread.o 2026-02-21T00:05:38.033 INFO:teuthology.orchestra.run.trial085.stdout: CC helpers.o 2026-02-21T00:05:38.075 INFO:teuthology.orchestra.run.trial085.stdout: CC json.o 2026-02-21T00:05:38.173 INFO:teuthology.orchestra.run.trial085.stdout: CC idletime.o 2026-02-21T00:05:38.292 INFO:teuthology.orchestra.run.trial085.stdout: CC td_error.o 2026-02-21T00:05:38.354 INFO:teuthology.orchestra.run.trial085.stdout: CC profiles/tiobench.o 2026-02-21T00:05:38.415 INFO:teuthology.orchestra.run.trial085.stdout: CC profiles/act.o 2026-02-21T00:05:38.502 INFO:teuthology.orchestra.run.trial085.stdout: CC io_u_queue.o 2026-02-21T00:05:38.536 INFO:teuthology.orchestra.run.trial085.stdout: CC filelock.o 2026-02-21T00:05:38.608 INFO:teuthology.orchestra.run.trial085.stdout: CC workqueue.o 2026-02-21T00:05:38.705 INFO:teuthology.orchestra.run.trial085.stdout: CC rate-submit.o 2026-02-21T00:05:38.791 INFO:teuthology.orchestra.run.trial085.stdout: CC optgroup.o 2026-02-21T00:05:38.824 INFO:teuthology.orchestra.run.trial085.stdout: CC helper_thread.o 2026-02-21T00:05:38.926 INFO:teuthology.orchestra.run.trial085.stdout: CC steadystate.o 2026-02-21T00:05:39.072 INFO:teuthology.orchestra.run.trial085.stdout: CC zone-dist.o 2026-02-21T00:05:39.150 INFO:teuthology.orchestra.run.trial085.stdout: CC zbd.o 2026-02-21T00:05:39.515 INFO:teuthology.orchestra.run.trial085.stdout: CC dedupe.o 2026-02-21T00:05:39.596 INFO:teuthology.orchestra.run.trial085.stdout: CC dataplacement.o 2026-02-21T00:05:39.689 INFO:teuthology.orchestra.run.trial085.stdout: CC sprandom.o 2026-02-21T00:05:39.848 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/posixaio.o 2026-02-21T00:05:39.926 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/falloc.o 2026-02-21T00:05:39.995 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/e4defrag.o 2026-02-21T00:05:40.070 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/splice.o 2026-02-21T00:05:40.171 INFO:teuthology.orchestra.run.trial085.stdout: CC oslib/asprintf.o 2026-02-21T00:05:40.198 INFO:teuthology.orchestra.run.trial085.stdout: CC oslib/strlcat.o 2026-02-21T00:05:40.230 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/mtd.o 2026-02-21T00:05:40.320 INFO:teuthology.orchestra.run.trial085.stdout: CC oslib/libmtd.o 2026-02-21T00:05:40.513 INFO:teuthology.orchestra.run.trial085.stdout: CC oslib/libmtd_legacy.o 2026-02-21T00:05:40.586 INFO:teuthology.orchestra.run.trial085.stdout: CC diskutil.o 2026-02-21T00:05:40.699 INFO:teuthology.orchestra.run.trial085.stdout: CC fifo.o 2026-02-21T00:05:40.737 INFO:teuthology.orchestra.run.trial085.stdout: CC blktrace.o 2026-02-21T00:05:40.904 INFO:teuthology.orchestra.run.trial085.stdout: CC cgroup.o 2026-02-21T00:05:40.988 INFO:teuthology.orchestra.run.trial085.stdout: CC trim.o 2026-02-21T00:05:41.056 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/sg.o 2026-02-21T00:05:41.253 INFO:teuthology.orchestra.run.trial085.stdout: CC oslib/linux-dev-lookup.o 2026-02-21T00:05:41.294 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/io_uring.o 2026-02-21T00:05:41.524 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/nvme.o 2026-02-21T00:05:41.687 INFO:teuthology.orchestra.run.trial085.stdout: CC oslib/linux-blkzoned.o 2026-02-21T00:05:41.777 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/cmdprio.o 2026-02-21T00:05:41.904 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/rados.o 2026-02-21T00:05:42.007 INFO:teuthology.orchestra.run.trial085.stdout: CC engines/rbd.o 2026-02-21T00:05:42.116 INFO:teuthology.orchestra.run.trial085.stdout: CC fio.o 2026-02-21T00:05:42.172 INFO:teuthology.orchestra.run.trial085.stdout: LINK fio 2026-02-21T00:05:42.295 INFO:teuthology.orchestra.run.trial085.stdout: CC t/log.o 2026-02-21T00:05:42.326 INFO:teuthology.orchestra.run.trial085.stdout: CC t/genzipf.o 2026-02-21T00:05:42.400 INFO:teuthology.orchestra.run.trial085.stdout: CC oslib/strcasestr.o 2026-02-21T00:05:42.421 INFO:teuthology.orchestra.run.trial085.stdout: CC oslib/strndup.o 2026-02-21T00:05:42.441 INFO:teuthology.orchestra.run.trial085.stdout: LINK t/fio-genzipf 2026-02-21T00:05:42.495 INFO:teuthology.orchestra.run.trial085.stdout: CC t/btrace2fio.o 2026-02-21T00:05:42.695 INFO:teuthology.orchestra.run.trial085.stdout: LINK t/fio-btrace2fio 2026-02-21T00:05:42.751 INFO:teuthology.orchestra.run.trial085.stdout: CC t/verify-state.o 2026-02-21T00:05:42.811 INFO:teuthology.orchestra.run.trial085.stdout: CC t/debug.o 2026-02-21T00:05:42.839 INFO:teuthology.orchestra.run.trial085.stdout: LINK t/fio-verify-state 2026-02-21T00:05:42.893 INFO:teuthology.orchestra.run.trial085.stdout: CC t/stest.o 2026-02-21T00:05:42.942 INFO:teuthology.orchestra.run.trial085.stdout: CC t/arch.o 2026-02-21T00:05:42.966 INFO:teuthology.orchestra.run.trial085.stdout: LINK t/stest 2026-02-21T00:05:43.021 INFO:teuthology.orchestra.run.trial085.stdout: CC t/ieee754.o 2026-02-21T00:05:43.050 INFO:teuthology.orchestra.run.trial085.stdout: LINK t/ieee754 2026-02-21T00:05:43.102 INFO:teuthology.orchestra.run.trial085.stdout: CC t/axmap.o 2026-02-21T00:05:43.161 INFO:teuthology.orchestra.run.trial085.stdout: LINK t/axmap 2026-02-21T00:05:43.214 INFO:teuthology.orchestra.run.trial085.stdout: CC t/lfsr-test.o 2026-02-21T00:05:43.269 INFO:teuthology.orchestra.run.trial085.stdout: LINK t/lfsr-test 2026-02-21T00:05:43.326 INFO:teuthology.orchestra.run.trial085.stdout: CC t/gen-rand.o 2026-02-21T00:05:43.385 INFO:teuthology.orchestra.run.trial085.stdout: LINK t/gen-rand 2026-02-21T00:05:43.441 INFO:teuthology.orchestra.run.trial085.stdout: CC t/memlock.o 2026-02-21T00:05:43.482 INFO:teuthology.orchestra.run.trial085.stdout: LINK t/memlock 2026-02-21T00:05:43.534 INFO:teuthology.orchestra.run.trial085.stdout: CC t/read-to-pipe-async.o 2026-02-21T00:05:43.636 INFO:teuthology.orchestra.run.trial085.stdout: LINK t/read-to-pipe-async 2026-02-21T00:05:43.690 INFO:teuthology.orchestra.run.trial085.stdout: CC t/io_uring.o 2026-02-21T00:05:43.962 INFO:teuthology.orchestra.run.trial085.stdout: LINK t/io_uring 2026-02-21T00:05:44.015 DEBUG:teuthology.orchestra.run.trial085:> sudo /sbin/sysctl -q -w kernel.perf_event_paranoid=0 2026-02-21T00:05:44.042 DEBUG:teuthology.orchestra.run.trial085:> /home/ubuntu/cephtest/cbt/cbt.py -a /home/ubuntu/cephtest/archive/cbt /home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2026-02-21T00:05:44.155 INFO:teuthology.orchestra.run.trial085.stderr:00:05:44 - DEBUG - cbt - Settings.cluster: 2026-02-21T00:05:44.155 INFO:teuthology.orchestra.run.trial085.stderr: {'archive_dir': '/home/ubuntu/cephtest/archive/cbt', 2026-02-21T00:05:44.156 INFO:teuthology.orchestra.run.trial085.stderr: 'clients': ['ubuntu@trial085.front.sepia.ceph.com'], 2026-02-21T00:05:44.156 INFO:teuthology.orchestra.run.trial085.stderr: 'conf_file': '/etc/ceph/ceph.conf', 2026-02-21T00:05:44.156 INFO:teuthology.orchestra.run.trial085.stderr: 'head': 'ubuntu@trial085.front.sepia.ceph.com', 2026-02-21T00:05:44.156 INFO:teuthology.orchestra.run.trial085.stderr: 'is_teuthology': True, 2026-02-21T00:05:44.156 INFO:teuthology.orchestra.run.trial085.stderr: 'iterations': 1, 2026-02-21T00:05:44.156 INFO:teuthology.orchestra.run.trial085.stderr: 'mons': ['ubuntu@trial085.front.sepia.ceph.com'], 2026-02-21T00:05:44.156 INFO:teuthology.orchestra.run.trial085.stderr: 'osds': ['ubuntu@trial085.front.sepia.ceph.com'], 2026-02-21T00:05:44.156 INFO:teuthology.orchestra.run.trial085.stderr: 'osds_per_node': 3, 2026-02-21T00:05:44.156 INFO:teuthology.orchestra.run.trial085.stderr: 'pid_dir': '/var/run/ceph', 2026-02-21T00:05:44.156 INFO:teuthology.orchestra.run.trial085.stderr: 'pool_profiles': {'rbd': {'pg_size': 128, 'pgp_size': 128, 'replication': 3}}, 2026-02-21T00:05:44.156 INFO:teuthology.orchestra.run.trial085.stderr: 'rebuild_every_test': False, 2026-02-21T00:05:44.157 INFO:teuthology.orchestra.run.trial085.stderr: 'rgws': {'ubuntu@trial085.front.sepia.ceph.com': None}, 2026-02-21T00:05:44.157 INFO:teuthology.orchestra.run.trial085.stderr: 'tmp_dir': '/tmp/cbt', 2026-02-21T00:05:44.157 INFO:teuthology.orchestra.run.trial085.stderr: 'use_existing': True, 2026-02-21T00:05:44.157 INFO:teuthology.orchestra.run.trial085.stderr: 'user': 'ubuntu'} 2026-02-21T00:05:44.157 INFO:teuthology.orchestra.run.trial085.stderr:00:05:44 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-7d4bb5d5 2026-02-21T00:05:44.157 INFO:teuthology.orchestra.run.trial085.stderr:00:05:44 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com if [ -d "/tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite" ]; then rm -rf /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite; fi 2026-02-21T00:05:44.322 INFO:teuthology.orchestra.run.trial085.stderr:00:05:44 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite 2026-02-21T00:05:44.476 INFO:teuthology.orchestra.run.trial085.stderr:00:05:44 - INFO - cbt - Pausing for 60s for idle monitoring. 2026-02-21T00:05:44.477 INFO:teuthology.orchestra.run.trial085.stderr:00:05:44 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/idle_monitoring/collectl 2026-02-21T00:05:44.634 INFO:teuthology.orchestra.run.trial085.stderr:00:05:44 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/idle_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-21T00:05:44.635 INFO:teuthology.orchestra.run.trial085.stderr:00:05:44 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/idle_monitoring/perf 2026-02-21T00:05:44.798 INFO:teuthology.orchestra.run.trial085.stderr:00:05:44 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-21T00:05:44.799 INFO:teuthology.orchestra.run.trial085.stderr:00:05:44 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/idle_monitoring/perf/perf_stat.${pid} & done 2026-02-21T00:06:44.859 INFO:teuthology.orchestra.run.trial085.stderr:00:06:44 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-21T00:06:45.032 INFO:teuthology.orchestra.run.trial085.stderr:00:06:45 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-21T00:06:45.208 INFO:teuthology.orchestra.run.trial085.stderr:00:06:45 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/* 2026-02-21T00:06:45.375 INFO:teuthology.orchestra.run.trial085.stderr:00:06:45 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial085.front.sepia.ceph.com -r /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-7d4bb5d5 2026-02-21T00:06:45.541 INFO:teuthology.orchestra.run.trial085.stderr:00:06:45 - INFO - cbt - Creating fio images... 2026-02-21T00:06:45.542 INFO:teuthology.orchestra.run.trial085.stderr:00:06:45 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/pool_monitoring/collectl 2026-02-21T00:06:45.697 INFO:teuthology.orchestra.run.trial085.stderr:00:06:45 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/pool_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-21T00:06:45.698 INFO:teuthology.orchestra.run.trial085.stderr:00:06:45 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/pool_monitoring/perf 2026-02-21T00:06:45.859 INFO:teuthology.orchestra.run.trial085.stderr:00:06:45 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-21T00:06:45.860 INFO:teuthology.orchestra.run.trial085.stderr:00:06:45 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/pool_monitoring/perf/perf_stat.${pid} & done 2026-02-21T00:06:45.861 INFO:teuthology.orchestra.run.trial085.stderr:00:06:45 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool delete cbt-librbdfio cbt-librbdfio --yes-i-really-really-mean-it 2026-02-21T00:06:46.198 INFO:teuthology.orchestra.run.trial085.stderr:00:06:46 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool create cbt-librbdfio 128 128 2026-02-21T00:06:47.364 INFO:teuthology.orchestra.run.trial085.stderr:00:06:47 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool application enable cbt-librbdfio rbd 2026-02-21T00:06:48.368 INFO:teuthology.orchestra.run.trial085.stderr:00:06:48 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool set cbt-librbdfio size 3 2026-02-21T00:06:49.370 INFO:teuthology.orchestra.run.trial085.stderr:00:06:49 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool set cbt-librbdfio min_size 2 2026-02-21T00:06:50.378 INFO:teuthology.orchestra.run.trial085.stderr:00:06:50 - INFO - cbt - Checking Health after pool creation. 2026-02-21T00:06:55.381 INFO:teuthology.orchestra.run.trial085.stderr:00:06:55 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com /usr/bin/ceph -c /etc/ceph/ceph.conf health 2026-02-21T00:06:55.730 INFO:teuthology.orchestra.run.trial085.stderr:00:06:55 - INFO - cbt - Final Pool Health Check. 2026-02-21T00:07:00.736 INFO:teuthology.orchestra.run.trial085.stderr:00:07:00 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com /usr/bin/ceph -c /etc/ceph/ceph.conf health 2026-02-21T00:07:01.085 INFO:teuthology.orchestra.run.trial085.stderr:00:07:01 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com hostname -f 2026-02-21T00:07:01.246 INFO:teuthology.orchestra.run.trial085.stderr:00:07:01 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com /usr/bin/rbd -c /etc/ceph/ceph.conf create cbt-librbdfio-trial085-0 --size 4096 --pool cbt-librbdfio --order 22 2026-02-21T00:07:03.503 INFO:teuthology.orchestra.run.trial085.stderr:00:07:03 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com /usr/bin/rbd -c /etc/ceph/ceph.conf create cbt-librbdfio-trial085-1 --size 4096 --pool cbt-librbdfio --order 22 2026-02-21T00:07:03.756 INFO:teuthology.orchestra.run.trial085.stderr:00:07:03 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-21T00:07:03.923 INFO:teuthology.orchestra.run.trial085.stderr:00:07:03 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-21T00:07:04.103 INFO:teuthology.orchestra.run.trial085.stderr:00:07:04 - INFO - cbt - Attempting to prefill fio images... 2026-02-21T00:07:04.104 INFO:teuthology.orchestra.run.trial085.stderr:00:07:04 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo /home/ubuntu/cephtest/fio/fio --ioengine=rbd --clientname=admin --pool=cbt-librbdfio --rbdname=cbt-librbdfio-`hostname -f`-0 --invalidate=0 --rw=write --numjobs=1 --bs=4M --size 4096M --name=cbt-librbdfio-`hostname -f`-file-0 --output-format=json,normal > /dev/null 2026-02-21T00:07:04.105 INFO:teuthology.orchestra.run.trial085.stderr:00:07:04 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo /home/ubuntu/cephtest/fio/fio --ioengine=rbd --clientname=admin --pool=cbt-librbdfio --rbdname=cbt-librbdfio-`hostname -f`-1 --invalidate=0 --rw=write --numjobs=1 --bs=4M --size 4096M --name=cbt-librbdfio-`hostname -f`-file-0 --output-format=json,normal > /dev/null 2026-02-21T00:07:55.444 INFO:teuthology.orchestra.run.trial085.stderr:00:07:55 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo /home/ubuntu/cephtest/fio/fio --ioengine=rbd --clientname=admin --pool=cbt-librbdfio --rbdname=cbt-librbdfio-`hostname -f`-0 --invalidate=0 --rw=write --numjobs=1 --bs=4M --size 4096M --name=cbt-librbdfio-`hostname -f`-file-0 --output-format=json,normal > /dev/null 2026-02-21T00:07:55.446 INFO:teuthology.orchestra.run.trial085.stderr:00:07:55 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo /home/ubuntu/cephtest/fio/fio --ioengine=rbd --clientname=admin --pool=cbt-librbdfio --rbdname=cbt-librbdfio-`hostname -f`-1 --invalidate=0 --rw=write --numjobs=1 --bs=4M --size 4096M --name=cbt-librbdfio-`hostname -f`-file-0 --output-format=json,normal > /dev/null 2026-02-21T00:08:19.494 INFO:teuthology.orchestra.run.trial085.stderr:00:08:19 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-7d4bb5d5 2026-02-21T00:08:19.494 INFO:teuthology.orchestra.run.trial085.stderr:00:08:19 - INFO - cbt - Skipping existing test in /home/ubuntu/cephtest/archive/cbt/results/00000000/id-7d4bb5d5. 2026-02-21T00:08:19.494 INFO:teuthology.orchestra.run.trial085.stderr:00:08:19 - INFO - cbt - Running benchmark /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite 2026-02-21T00:08:19.494 INFO:teuthology.orchestra.run.trial085.stderr:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-7d4bb5d5 2026-02-21T00:08:19.494 INFO:teuthology.orchestra.run.trial085.stderr:{'cmd_path': '/home/ubuntu/cephtest/fio/fio', 'log_avg_msec': 100, 'norandommap': True, 'pool_profile': 'rbd', 'time': 60, 'vol_size': 4096, 'iodepth': 32, 'mode': 'randwrite', 'op_size': 4194304, 'osd_ra': 4096, 'procs_per_volume': 1, 'volumes_per_client': 2, 'benchmark': 'librbdfio', 'iteration': 0} == iteration 0 == 2026-02-21T00:08:19.495 INFO:teuthology.orchestra.run.trial085.stderr:00:08:19 - INFO - cbt - Setting OSD Read Ahead to: 4096 2026-02-21T00:08:19.495 INFO:teuthology.orchestra.run.trial085.stderr:00:08:19 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com find /dev/disk/by-partlabel/osd-device-*data -exec readlink {} \; | cut -d"/" -f 3 | sed "s/[0-9]$//" | xargs -I{} sudo sh -c "echo 4096 > /sys/block/'{}'/queue/read_ahead_kb" 2026-02-21T00:08:19.495 INFO:teuthology.orchestra.run.trial085.stderr:00:08:19 - DEBUG - cbt - Cleaning existing temporary run directory: /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite 2026-02-21T00:08:19.496 INFO:teuthology.orchestra.run.trial085.stderr:00:08:19 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo rm -rf /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite 2026-02-21T00:08:19.669 INFO:teuthology.orchestra.run.trial085.stderr:00:08:19 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sync 2026-02-21T00:08:19.957 INFO:teuthology.orchestra.run.trial085.stderr:00:08:19 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com echo 3 | sudo tee /proc/sys/vm/drop_caches 2026-02-21T00:08:21.119 INFO:teuthology.orchestra.run.trial085.stderr:00:08:21 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite 2026-02-21T00:08:21.306 INFO:teuthology.orchestra.run.trial085.stderr:00:08:21 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf daemon osd.0 config show > /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/ceph_settings.out 2026-02-21T00:08:46.609 INFO:teuthology.orchestra.run.trial085.stderr:00:08:46 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com echo "[`date`] ceph progress" >> /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/pgautoscaler.log;/usr/bin/ceph -c /etc/ceph/ceph.conf progress | tee -a /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/pgautoscaler.log 2026-02-21T00:08:46.973 INFO:teuthology.orchestra.run.trial085.stderr:00:08:46 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/collectl 2026-02-21T00:08:47.136 INFO:teuthology.orchestra.run.trial085.stderr:00:08:47 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-21T00:08:47.138 INFO:teuthology.orchestra.run.trial085.stderr:00:08:47 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/perf 2026-02-21T00:08:47.296 INFO:teuthology.orchestra.run.trial085.stderr:00:08:47 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-21T00:08:47.297 INFO:teuthology.orchestra.run.trial085.stderr:00:08:47 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/perf/perf_stat.${pid} & done 2026-02-21T00:08:47.298 INFO:teuthology.orchestra.run.trial085.stderr:00:08:47 - INFO - cbt - Running rbd fio randwrite test. 2026-02-21T00:08:47.298 INFO:teuthology.orchestra.run.trial085.stderr:00:08:47 - DEBUG - cbt - Using rbdname cbt-librbdfio-`hostname -f`-0 2026-02-21T00:08:47.298 INFO:teuthology.orchestra.run.trial085.stderr:00:08:47 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo /home/ubuntu/cephtest/fio/fio --ioengine=rbd --clientname=admin --pool=cbt-librbdfio --rbdname=cbt-librbdfio-`hostname -f`-0 --invalidate=0 --rw=randwrite --output-format=json,normal --runtime=60 --numjobs=1 --direct=1 --bs=4194304B --iodepth=32 --end_fsync=0 --norandommap --write_iops_log=/tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/output.0 --write_bw_log=/tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/output.0 --write_lat_log=/tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/output.0 --log_avg_msec=100 --name=cbt-librbdfio-`hostname -f`-file-0 > /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/output.0 2026-02-21T00:08:47.299 INFO:teuthology.orchestra.run.trial085.stderr:00:08:47 - DEBUG - cbt - Using rbdname cbt-librbdfio-`hostname -f`-1 2026-02-21T00:08:47.300 INFO:teuthology.orchestra.run.trial085.stderr:00:08:47 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo /home/ubuntu/cephtest/fio/fio --ioengine=rbd --clientname=admin --pool=cbt-librbdfio --rbdname=cbt-librbdfio-`hostname -f`-1 --invalidate=0 --rw=randwrite --output-format=json,normal --runtime=60 --numjobs=1 --direct=1 --bs=4194304B --iodepth=32 --end_fsync=0 --norandommap --write_iops_log=/tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/output.1 --write_bw_log=/tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/output.1 --write_lat_log=/tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/output.1 --log_avg_msec=100 --name=cbt-librbdfio-`hostname -f`-file-0 > /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/output.1 2026-02-21T00:09:13.416 INFO:teuthology.orchestra.run.trial085.stderr:00:09:13 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-21T00:09:13.593 INFO:teuthology.orchestra.run.trial085.stderr:00:09:13 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-21T00:09:13.772 INFO:teuthology.orchestra.run.trial085.stderr:00:09:13 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/perf/perf.data 2026-02-21T00:09:13.774 INFO:teuthology.orchestra.run.trial085.stderr:00:09:13 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/perf/perf_stat.* 2026-02-21T00:09:13.776 INFO:teuthology.orchestra.run.trial085.stderr:00:09:13 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com find /var/run/ceph/ceph-osd*.asok -maxdepth 1 -exec sudo /usr/bin/ceph --admin-daemon {} dump_historic_ops \; > /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/historic_ops.out 2026-02-21T00:09:14.123 INFO:teuthology.orchestra.run.trial085.stderr:00:09:14 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial085.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/* 2026-02-21T00:09:14.291 INFO:teuthology.orchestra.run.trial085.stderr:00:09:14 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial085.front.sepia.ceph.com -r /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-7d4bb5d5 2026-02-21T00:09:14.459 INFO:teuthology.orchestra.run.trial085.stderr:00:09:14 - INFO - cbt - Convert results to json format. 2026-02-21T00:09:14.459 INFO:teuthology.orchestra.run.trial085.stdout:cleaning remote dir /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-04194304/concurrent_procs-002/iodepth-032/randwrite 2026-02-21T00:09:14.459 INFO:teuthology.orchestra.run.trial085.stdout:trial085: trial085 2026-02-21T00:09:14.459 INFO:teuthology.orchestra.run.trial085.stdout: 2026-02-21T00:09:14.460 INFO:teuthology.orchestra.run.trial085.stdout:['trial085'] 2026-02-21T00:09:14.470 DEBUG:teuthology.run_tasks:Unwinding manager cbt 2026-02-21T00:09:14.486 DEBUG:teuthology.orchestra.run.trial085:> rm --one-file-system -rf -- /home/ubuntu/cephtest/cbt 2026-02-21T00:09:14.512 DEBUG:teuthology.orchestra.run.trial085:> rm --one-file-system -rf -- /home/ubuntu/cephtest/fio 2026-02-21T00:09:14.590 INFO:tasks.cbt_performance:Collecting CBT performance data config 2026-02-21T00:09:14.590 INFO:tasks.cbt_performance:reading cbt preferences 2026-02-21T00:09:14.590 DEBUG:teuthology.orchestra.run.trial085:> find /home/ubuntu/cephtest -name 'json_output.*' 2026-02-21T00:09:14.648 INFO:teuthology.orchestra.run.trial085.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-7d4bb5d5/json_output.1.trial085.front.sepia.ceph.com 2026-02-21T00:09:14.648 INFO:teuthology.orchestra.run.trial085.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-7d4bb5d5/json_output.0.trial085.front.sepia.ceph.com 2026-02-21T00:09:14.649 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:09:14.650 DEBUG:teuthology.orchestra.run.trial085:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-7d4bb5d5/json_output.1.trial085.front.sepia.ceph.com of=/dev/stdout 2026-02-21T00:09:14.707 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-21T00:09:14.707 DEBUG:teuthology.orchestra.run.trial085:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-7d4bb5d5 -name 'perf_stat.*' 2026-02-21T00:09:14.762 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-21T00:09:14.762 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-21T00:09:14.763 DEBUG:teuthology.orchestra.run.trial085:> set -ex 2026-02-21T00:09:14.763 DEBUG:teuthology.orchestra.run.trial085:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-7d4bb5d5/json_output.0.trial085.front.sepia.ceph.com of=/dev/stdout 2026-02-21T00:09:14.820 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-21T00:09:14.821 DEBUG:teuthology.orchestra.run.trial085:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-7d4bb5d5 -name 'perf_stat.*' 2026-02-21T00:09:14.876 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-21T00:09:14.876 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-21T00:09:14.879 WARNING:tasks.cbt_performance:Could not send performance data to http://mira118.front.sepia.ceph.com:4000/cbt_performance: HTTPConnectionPool(host='mira118.front.sepia.ceph.com', port=4000): Max retries exceeded with url: /cbt_performance (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known')) 2026-02-21T00:09:14.880 WARNING:tasks.cbt_performance:Continuing without performance data collection 2026-02-21T00:09:14.880 DEBUG:teuthology.run_tasks:Unwinding manager ssh_keys 2026-02-21T00:09:14.891 INFO:teuthology.task.ssh_keys:Cleaning up SSH keys 2026-02-21T00:09:14.891 INFO:teuthology.task.ssh_keys:cleaning up keys added for testing 2026-02-21T00:09:14.891 INFO:teuthology.task.ssh_keys: cleaning up keys for user ubuntu on trial085.front.sepia.ceph.com 2026-02-21T00:09:14.891 DEBUG:teuthology.orchestra.run.trial085:> rm -- /home/ubuntu/.ssh/id_rsa 2026-02-21T00:09:14.934 DEBUG:teuthology.orchestra.run.trial085:> rm -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-21T00:09:14.990 DEBUG:teuthology.orchestra.run.trial085:> stat -c '"%a"' /home/ubuntu/.ssh/authorized_keys 2026-02-21T00:09:15.047 INFO:teuthology.orchestra.run.trial085.stdout:"600" 2026-02-21T00:09:15.047 DEBUG:teuthology.orchestra.run.trial085:> mv -- /home/ubuntu/.ssh/authorized_keys_2026-02-21_00:05:15:492563 /home/ubuntu/.ssh/authorized_keys 2026-02-21T00:09:15.104 DEBUG:teuthology.orchestra.run.trial085:> chmod 600 /home/ubuntu/.ssh/authorized_keys 2026-02-21T00:09:15.162 DEBUG:teuthology.orchestra.run.trial085:> rm /home/ubuntu/.ssh/config 2026-02-21T00:09:15.219 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-21T00:09:15.235 DEBUG:teuthology.orchestra.run.trial085:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-21T00:09:15.335 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:09:15.329+0000 7fb3f15a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:15.337 INFO:teuthology.orchestra.run.trial085.stderr:2026-02-21T00:09:15.334+0000 7fb3f15a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:09:15.461 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-21T00:09:15.462 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-21T00:09:15.462 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-21T00:09:15.462 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:09:15.542 INFO:tasks.ceph.osd.0:Stopped 2026-02-21T00:09:15.543 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-21T00:09:15.543 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:09:15.635 INFO:tasks.ceph.osd.1:Stopped 2026-02-21T00:09:15.636 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-21T00:09:15.636 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:09:15.729 INFO:tasks.ceph.osd.2:Stopped 2026-02-21T00:09:15.729 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-21T00:09:15.730 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-21T00:09:15.730 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:09:15.767 INFO:tasks.ceph.mgr.x:Stopped 2026-02-21T00:09:15.768 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-21T00:09:15.768 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-21T00:09:15.768 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:09:15.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2026-02-21T00:09:15.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 is failed for ~0s 2026-02-21T00:09:15.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2026-02-21T00:09:15.817 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a is failed for ~0s 2026-02-21T00:09:15.818 INFO:tasks.ceph.mon.a:Stopped 2026-02-21T00:09:15.818 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-21T00:09:15.818 DEBUG:teuthology.orchestra.run.trial085:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v '\(PG_' | grep -E -v '\(OSD_' | grep -E -v '\(OBJECT_' | grep -E -v 'overall HEALTH' | head -n 1 2026-02-21T00:09:15.884 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial085.front.sepia.ceph.com 2026-02-21T00:09:15.884 DEBUG:teuthology.orchestra.run.trial085:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-21T00:09:16.058 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial085.front.sepia.ceph.com 2026-02-21T00:09:16.058 DEBUG:teuthology.orchestra.run.trial085:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-21T00:09:16.084 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial085.front.sepia.ceph.com 2026-02-21T00:09:16.084 DEBUG:teuthology.orchestra.run.trial085:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-21T00:09:16.152 INFO:tasks.ceph:Archiving mon data... 2026-02-21T00:09:16.154 DEBUG:teuthology.misc:Transferring archived files from trial085:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61901/data/mon.a.tgz 2026-02-21T00:09:16.154 DEBUG:teuthology.orchestra.run.trial085:> mktemp 2026-02-21T00:09:16.209 INFO:teuthology.orchestra.run.trial085.stdout:/tmp/tmp.4pgI3YE4IY 2026-02-21T00:09:16.209 DEBUG:teuthology.orchestra.run.trial085:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.4pgI3YE4IY 2026-02-21T00:09:16.299 DEBUG:teuthology.orchestra.run.trial085:> sudo chmod 0666 /tmp/tmp.4pgI3YE4IY 2026-02-21T00:09:16.384 DEBUG:teuthology.orchestra.remote:trial085:/tmp/tmp.4pgI3YE4IY is 200KB 2026-02-21T00:09:16.437 DEBUG:teuthology.orchestra.run.trial085:> rm -fr /tmp/tmp.4pgI3YE4IY 2026-02-21T00:09:16.453 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-21T00:09:16.453 DEBUG:teuthology.orchestra.run.trial085:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-21T00:09:16.574 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-21T00:09:16.574 INFO:tasks.ceph:Archiving crash dumps... 2026-02-21T00:09:16.577 DEBUG:teuthology.misc:Transferring archived files from trial085:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61901/remote/trial085/crash 2026-02-21T00:09:16.578 DEBUG:teuthology.orchestra.run.trial085:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-21T00:09:16.639 INFO:tasks.ceph:Compressing logs... 2026-02-21T00:09:16.639 DEBUG:teuthology.orchestra.run.trial085:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T00:09:16.717 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-21T00:09:16.717 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46963.log 2026-02-21T00:09:16.717 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-21T00:09:16.718 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.46963.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40273.log 2026-02-21T00:09:16.718 INFO:teuthology.orchestra.run.trial085.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46963.log.gz 2026-02-21T00:09:16.718 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47715.log 2026-02-21T00:09:16.718 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.40273.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-21T00:09:16.718 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.47715.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40273.log.gz 2026-02-21T00:09:16.718 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46384.log 2026-02-21T00:09:16.718 INFO:teuthology.orchestra.run.trial085.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.47715.log.gz 2026-02-21T00:09:16.718 INFO:teuthology.orchestra.run.trial085.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-21T00:09:16.718 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40424.log 2026-02-21T00:09:16.718 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.46384.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41123.log 2026-02-21T00:09:16.718 INFO:teuthology.orchestra.run.trial085.stderr: 50.8% -- replaced with /var/log/ceph/ceph-client.admin.46384.log.gz 2026-02-21T00:09:16.718 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.40424.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46651.log 2026-02-21T00:09:16.719 INFO:teuthology.orchestra.run.trial085.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40424.log.gz 2026-02-21T00:09:16.719 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.41123.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47716.log 2026-02-21T00:09:16.719 INFO:teuthology.orchestra.run.trial085.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41123.log.gz 2026-02-21T00:09:16.719 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.46651.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41074.log 2026-02-21T00:09:16.719 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.47716.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.47716.log.gz 2026-02-21T00:09:16.719 INFO:teuthology.orchestra.run.trial085.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46651.log.gz 2026-02-21T00:09:16.719 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46442.log 2026-02-21T00:09:16.719 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39946.log 2026-02-21T00:09:16.719 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.41074.log: /var/log/ceph/ceph-client.admin.46442.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41074.log.gz 2026-02-21T00:09:16.719 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.38276.log 2026-02-21T00:09:16.719 INFO:teuthology.orchestra.run.trial085.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.46442.log.gz 2026-02-21T00:09:16.719 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.39946.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40519.log 2026-02-21T00:09:16.720 INFO:teuthology.orchestra.run.trial085.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39946.log.gz 2026-02-21T00:09:16.720 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph.tmp-client.admin.38276.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41172.log 2026-02-21T00:09:16.720 INFO:teuthology.orchestra.run.trial085.stderr: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.38276.log.gz 2026-02-21T00:09:16.720 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.40519.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40682.log 2026-02-21T00:09:16.720 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.41172.log: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.40519.log.gz 2026-02-21T00:09:16.720 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-21T00:09:16.720 INFO:teuthology.orchestra.run.trial085.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41172.log.gz 2026-02-21T00:09:16.720 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.40682.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40978.log 2026-02-21T00:09:16.720 INFO:teuthology.orchestra.run.trial085.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40682.log.gz 2026-02-21T00:09:16.720 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40979.log 2026-02-21T00:09:16.720 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.40978.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40731.log 2026-02-21T00:09:16.720 INFO:teuthology.orchestra.run.trial085.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40978.log.gz 2026-02-21T00:09:16.720 INFO:teuthology.orchestra.run.trial085.stderr: 86.4% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-21T00:09:16.721 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.40979.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46555.log 2026-02-21T00:09:16.721 INFO:teuthology.orchestra.run.trial085.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40979.log.gz 2026-02-21T00:09:16.721 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.40731.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-21T00:09:16.721 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.46555.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40731.log.gz 2026-02-21T00:09:16.721 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46587.log 2026-02-21T00:09:16.721 INFO:teuthology.orchestra.run.trial085.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46555.log.gz 2026-02-21T00:09:16.721 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40165.log 2026-02-21T00:09:16.721 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.46587.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48373.log 2026-02-21T00:09:16.721 INFO:teuthology.orchestra.run.trial085.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.46587.log.gz 2026-02-21T00:09:16.721 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.40165.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40834.log 2026-02-21T00:09:16.721 INFO:teuthology.orchestra.run.trial085.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40165.log.gz 2026-02-21T00:09:16.721 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.48373.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47480.log 2026-02-21T00:09:16.722 INFO:teuthology.orchestra.run.trial085.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.48373.log.gz 2026-02-21T00:09:16.722 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.40834.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39899.log 2026-02-21T00:09:16.722 INFO:teuthology.orchestra.run.trial085.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40834.log.gz 2026-02-21T00:09:16.722 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46737.log 2026-02-21T00:09:16.722 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.47480.log: /var/log/ceph/ceph-client.admin.39899.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.47480.log.gz 2026-02-21T00:09:16.722 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39573.log 2026-02-21T00:09:16.722 INFO:teuthology.orchestra.run.trial085.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39899.log.gz 2026-02-21T00:09:16.722 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.46737.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46498.log 2026-02-21T00:09:16.722 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.39573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40584.log 2026-02-21T00:09:16.722 INFO:teuthology.orchestra.run.trial085.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39573.log.gz 2026-02-21T00:09:16.722 INFO:teuthology.orchestra.run.trial085.stderr: 66.9% -- replaced with /var/log/ceph/ceph-client.admin.46737.log.gz 2026-02-21T00:09:16.722 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.46498.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40494.log 2026-02-21T00:09:16.723 INFO:teuthology.orchestra.run.trial085.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46498.log.gz 2026-02-21T00:09:16.723 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.40584.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40981.log 2026-02-21T00:09:16.723 INFO:teuthology.orchestra.run.trial085.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40584.log.gz 2026-02-21T00:09:16.723 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.40494.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-21T00:09:16.723 INFO:teuthology.orchestra.run.trial085.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40494.log.gz 2026-02-21T00:09:16.723 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40833.log 2026-02-21T00:09:16.723 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.40981.log: 50.4%/var/log/ceph/ceph-mgr.x.log: -- replaced with /var/log/ceph/ceph-client.admin.40981.log.gz 2026-02-21T00:09:16.723 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40633.log 2026-02-21T00:09:16.723 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.40833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40350.log 2026-02-21T00:09:16.723 INFO:teuthology.orchestra.run.trial085.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40833.log.gz 2026-02-21T00:09:16.723 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.40633.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-21T00:09:16.723 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.40350.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40633.log.gz 2026-02-21T00:09:16.723 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39668.log 2026-02-21T00:09:16.724 INFO:teuthology.orchestra.run.trial085.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40350.log.gz 2026-02-21T00:09:16.724 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39715.log 2026-02-21T00:09:16.724 INFO:teuthology.orchestra.run.trial085.stderr: 86.6% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-21T00:09:16.724 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46962.log 2026-02-21T00:09:16.724 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.39668.log: /var/log/ceph/ceph-client.admin.39715.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46328.log 2026-02-21T00:09:16.724 INFO:teuthology.orchestra.run.trial085.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39715.log.gz 2026-02-21T00:09:16.724 INFO:teuthology.orchestra.run.trial085.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39668.log.gz 2026-02-21T00:09:16.724 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.46962.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46785.log 2026-02-21T00:09:16.724 INFO:teuthology.orchestra.run.trial085.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46962.log.gz 2026-02-21T00:09:16.724 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.46328.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47135.log 2026-02-21T00:09:16.724 INFO:teuthology.orchestra.run.trial085.stderr: 48.4% -- replaced with /var/log/ceph/ceph-client.admin.46328.log.gz 2026-02-21T00:09:16.724 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.46785.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39852.log 2026-02-21T00:09:16.724 INFO:teuthology.orchestra.run.trial085.stderr: 66.9% -- replaced with /var/log/ceph/ceph-client.admin.46785.log.gz 2026-02-21T00:09:16.725 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.47135.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40832.log 2026-02-21T00:09:16.725 INFO:teuthology.orchestra.run.trial085.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.47135.log.gz 2026-02-21T00:09:16.725 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.39852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47136.log 2026-02-21T00:09:16.725 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.40832.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39852.log.gz 2026-02-21T00:09:16.725 INFO:teuthology.orchestra.run.trial085.stderr:/var/log/ceph/ceph-client.admin.47136.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40832.log.gz 2026-02-21T00:09:16.725 INFO:teuthology.orchestra.run.trial085.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.47136.log.gz 2026-02-21T00:09:16.725 INFO:teuthology.orchestra.run.trial085.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-21T00:09:20.138 INFO:teuthology.orchestra.run.trial085.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-21T00:09:20.179 INFO:teuthology.orchestra.run.trial085.stderr: 92.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-21T00:09:20.234 INFO:teuthology.orchestra.run.trial085.stderr: 92.5% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-21T00:09:20.235 INFO:teuthology.orchestra.run.trial085.stderr: 2026-02-21T00:09:20.235 INFO:teuthology.orchestra.run.trial085.stderr:real 0m3.541s 2026-02-21T00:09:20.235 INFO:teuthology.orchestra.run.trial085.stderr:user 0m9.780s 2026-02-21T00:09:20.235 INFO:teuthology.orchestra.run.trial085.stderr:sys 0m0.671s 2026-02-21T00:09:20.237 INFO:tasks.ceph:Archiving logs... 2026-02-21T00:09:20.237 DEBUG:teuthology.misc:Transferring archived files from trial085:/var/log/ceph to /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61901/remote/trial085/log 2026-02-21T00:09:20.239 DEBUG:teuthology.orchestra.run.trial085:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-21T00:09:20.824 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2026-02-21T00:09:20.824 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.1 has been restored 2026-02-21T00:09:20.824 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2026-02-21T00:09:20.824 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.mon.a has been restored 2026-02-21T00:09:21.271 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-21T00:09:21.339 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-21T00:09:21.351 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-21T00:09:21.352 DEBUG:teuthology.orchestra.run.trial085:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-21T00:09:21.378 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-21T00:09:21.390 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-21T00:09:21.390 DEBUG:teuthology.orchestra.run.trial085:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T00:09:21.433 INFO:teuthology.orchestra.run.trial085.stderr:bash: line 1: ntpq: command not found 2026-02-21T00:09:21.462 INFO:teuthology.orchestra.run.trial085.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-21T00:09:21.462 INFO:teuthology.orchestra.run.trial085.stdout:=============================================================================== 2026-02-21T00:09:21.462 INFO:teuthology.orchestra.run.trial085.stdout:^+ 170-187-142-180.ip.linod> 2 6 377 4 -9103us[-9103us] +/- 81ms 2026-02-21T00:09:21.462 INFO:teuthology.orchestra.run.trial085.stdout:^+ 172-234-44-141.ip.linode> 2 6 377 5 +13ms[ +12ms] +/- 90ms 2026-02-21T00:09:21.462 INFO:teuthology.orchestra.run.trial085.stdout:^* time2.trtnw.net 2 6 377 4 -570us[ -937us] +/- 42ms 2026-02-21T00:09:21.462 INFO:teuthology.orchestra.run.trial085.stdout:^- ntp.maxhost.io 2 6 135 63 +1289us[ +937us] +/- 56ms 2026-02-21T00:09:21.463 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-21T00:09:21.474 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-21T00:09:21.475 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-21T00:09:21.491 DEBUG:teuthology.orchestra.run.trial085:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-21T00:09:21.564 DEBUG:teuthology.orchestra.run.trial085:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-21T00:09:21.590 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T00:09:21.591 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-21T00:09:21.608 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-21T00:09:21.625 INFO:teuthology.task.internal:Duration was 1720.245892 seconds 2026-02-21T00:09:21.625 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-21T00:09:21.642 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-21T00:09:21.642 DEBUG:teuthology.orchestra.run.trial085:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-21T00:09:21.682 INFO:teuthology.orchestra.run.trial085.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-21T00:09:21.857 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-21T00:09:21.857 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial085.front.sepia.ceph.com 2026-02-21T00:09:21.857 DEBUG:teuthology.orchestra.run.trial085:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-21T00:09:21.916 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-21T00:09:21.916 DEBUG:teuthology.orchestra.run.trial085:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T00:09:22.021 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-21T00:09:22.022 DEBUG:teuthology.orchestra.run.trial085:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T00:09:22.044 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T00:09:22.045 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T00:09:22.045 INFO:teuthology.orchestra.run.trial085.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-21T00:09:22.045 INFO:teuthology.orchestra.run.trial085.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T00:09:22.046 INFO:teuthology.orchestra.run.trial085.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-21T00:09:22.055 INFO:teuthology.orchestra.run.trial085.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.9% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-21T00:09:22.057 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-21T00:09:22.075 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-21T00:09:22.075 DEBUG:teuthology.orchestra.run.trial085:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-21T00:09:22.121 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-21T00:09:22.138 DEBUG:teuthology.orchestra.run.trial085:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-21T00:09:22.184 INFO:teuthology.orchestra.run.trial085.stdout:kernel.core_pattern = core 2026-02-21T00:09:22.215 DEBUG:teuthology.orchestra.run.trial085:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-21T00:09:22.270 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T00:09:22.271 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-21T00:09:22.288 INFO:teuthology.task.internal:Transferring archived files... 2026-02-21T00:09:22.290 DEBUG:teuthology.misc:Transferring archived files from trial085:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61901/remote/trial085 2026-02-21T00:09:22.290 DEBUG:teuthology.orchestra.run.trial085:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-21T00:09:22.374 INFO:teuthology.task.internal:Removing archive directory... 2026-02-21T00:09:22.374 DEBUG:teuthology.orchestra.run.trial085:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-21T00:09:22.401 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-21T00:09:22.423 INFO:teuthology.task.internal:Not uploading archives. 2026-02-21T00:09:22.423 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-21T00:09:22.441 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-21T00:09:22.441 DEBUG:teuthology.orchestra.run.trial085:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-21T00:09:22.457 INFO:teuthology.orchestra.run.trial085.stdout: 917576 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 21 00:09 /home/ubuntu/cephtest 2026-02-21T00:09:22.459 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-21T00:09:22.476 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-21T00:09:22.516 INFO:teuthology.run:Summary data: description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} perf_ignorelist settings/optimized workloads/fio_4M_rand_write} duration: 1720.245892047882 flavor: debug owner: scheduled_liucm@soko04.front.sepia.ceph.com success: true 2026-02-21T00:09:22.517 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T00:09:22.630 INFO:teuthology.run:pass