2026-02-20T23:43:24.838 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-20T23:43:24.853 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-20T23:43:24.933 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61909 branch: wip-liucm-osd-shard-redesign-crimson-only description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} perf_ignorelist settings/optimized workloads/radosbench_4K_seq_read} email: null first_in_suite: false flavor: debug job_id: '61909' kernel: branch: distro kdb: true sha1: distro last_in_suite: false machine_type: trial name: liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-liucm-osd-shard-redesign-crimson-only ceph: conf: global: auth client required: none auth cluster required: none auth service required: none auth supported: none debug asok: 0/0 debug auth: 0/0 debug bdev: 0/0 debug bluefs: 0/0 debug bluestore: 0/0 debug buffer: 0/0 debug civetweb: 0/0 debug client: 0/0 debug compressor: 0/0 debug context: 0/0 debug crush: 0/0 debug crypto: 0/0 debug dpdk: 0/0 debug eventtrace: 0/0 debug filer: 0/0 debug filestore: 0/0 debug finisher: 0/0 debug fuse: 0/0 debug heartbeatmap: 0/0 debug javaclient: 0/0 debug journal: 0/0 debug journaler: 0/0 debug kstore: 0/0 debug leveldb: 0/0 debug lockdep: 0/0 debug mds: 0/0 debug mds balancer: 0/0 debug mds locker: 0/0 debug mds log: 0/0 debug mds log expire: 0/0 debug mds migrator: 0/0 debug memdb: 0/0 debug mgr: 0/0 debug mgrc: 0/0 debug mon: 0/0 debug monc: 0/0 debug ms: 0/0 debug objclass: 0/0 debug objectcacher: 0/0 debug objecter: 0/0 debug optracker: 0/0 debug osd: 0/0 debug paxos: 0/0 debug perfcounter: 0/0 debug rados: 0/0 debug rbd: 0/0 debug rbd mirror: 0/0 debug rbd replay: 0/0 debug refs: 0/0 debug rgw: 0/0 debug rgw lifecycle: 0/0 debug rgw notification: 0/0 debug rgw sync: 0/0 debug rocksdb: 0/0 debug striper: 0/0 debug throttle: 0/0 debug timer: 0/0 debug tp: 0/0 enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 0/0 debug ms: 0/0 debug paxos: 0/0 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug filestore: 0/0 debug journal: 0/0 debug ms: 0/0 debug osd: 0/0 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore main device type: RANDOM_BLOCK_SSD seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - \(PG_ - \(OSD_ - \(OBJECT_ - overall HEALTH sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: flavor: debug sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f workunit: branch: wip-liucm-osd-shard-redesign-crimson-only env: CRIMSON_COMPAT: '1' sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f owner: scheduled_liucm@soko04.front.sepia.ceph.com priority: 120 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 6283 sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f sleep_before_teardown: 0 suite: crimson-rados suite_branch: wip-liucm-osd-shard-redesign-crimson-only suite_path: /home/teuthworker/src/git.ceph.com_ceph-c_bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/qa suite_relpath: qa suite_repo: https://git.ceph.com/ceph-ci.git suite_sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f targets: trial138.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL1i29wZ1CwcdTV0uC5Pdul/Z4fMZCQq2Y+tS5d5EwzVMMKM/kACnTOtzztUSUDtJ73GUXQ6FlH6I3mf5vXRCKQ= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: osd: debug monc: 20 wait-for-scrub: false - ssh_keys: null - cbt: benchmarks: radosbench: concurrent_ops: 4 concurrent_procs: 2 op_size: - 4096 pool_monitoring_list: - collectl pool_profile: replicated run_monitoring_list: - collectl time: 60 write_only: false cluster: iterations: 1 osds_per_node: 3 pool_profiles: replicated: pg_size: 256 pgp_size: 256 replication: replicated user: ubuntu monitoring_profiles: perf: args: stat -p {pid} -o {perf_dir}/perf_stat.{pid} nodes: - osds perf_cmd: perf pid_glob: crimson-osd.*.pid teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-20_22:16:48 tube: trial user: liucm verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-20T23:43:24.933 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/git.ceph.com_ceph-c_bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/qa; will attempt to use it 2026-02-20T23:43:24.934 INFO:teuthology.run:Found tasks at /home/teuthworker/src/git.ceph.com_ceph-c_bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/qa/tasks 2026-02-20T23:43:24.934 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-20T23:43:24.936 INFO:teuthology.task.internal:Checking packages... 2026-02-20T23:43:24.936 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f' 2026-02-20T23:43:24.936 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-20T23:43:24.936 INFO:teuthology.packaging:ref: None 2026-02-20T23:43:24.936 INFO:teuthology.packaging:tag: None 2026-02-20T23:43:24.936 INFO:teuthology.packaging:branch: wip-liucm-osd-shard-redesign-crimson-only 2026-02-20T23:43:24.936 INFO:teuthology.packaging:sha1: bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-20T23:43:24.936 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=wip-liucm-osd-shard-redesign-crimson-only 2026-02-20T23:43:25.057 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5343.gbcdd429b 2026-02-20T23:43:25.059 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-20T23:43:25.070 INFO:teuthology.task.internal:no buildpackages task found 2026-02-20T23:43:25.070 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-20T23:43:25.083 INFO:teuthology.task.internal:Saving configuration 2026-02-20T23:43:25.090 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-20T23:43:25.102 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-20T23:43:25.174 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial138.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61909', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-20 23:39:57.940458', 'locked_by': 'scheduled_liucm@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL1i29wZ1CwcdTV0uC5Pdul/Z4fMZCQq2Y+tS5d5EwzVMMKM/kACnTOtzztUSUDtJ73GUXQ6FlH6I3mf5vXRCKQ='} 2026-02-20T23:43:25.174 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-20T23:43:25.189 INFO:teuthology.task.internal:roles: ubuntu@trial138.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-20T23:43:25.189 INFO:teuthology.run_tasks:Running task console_log... 2026-02-20T23:43:25.272 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7adb1b242de0>, signals=[15]) 2026-02-20T23:43:25.272 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-20T23:43:25.283 INFO:teuthology.task.internal:Opening connections... 2026-02-20T23:43:25.283 DEBUG:teuthology.task.internal:connecting to ubuntu@trial138.front.sepia.ceph.com 2026-02-20T23:43:25.284 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T23:43:25.346 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-20T23:43:25.357 DEBUG:teuthology.orchestra.run.trial138:> uname -m 2026-02-20T23:43:25.421 INFO:teuthology.orchestra.run.trial138.stdout:x86_64 2026-02-20T23:43:25.421 DEBUG:teuthology.orchestra.run.trial138:> cat /etc/os-release 2026-02-20T23:43:25.477 INFO:teuthology.orchestra.run.trial138.stdout:NAME="CentOS Stream" 2026-02-20T23:43:25.477 INFO:teuthology.orchestra.run.trial138.stdout:VERSION="9" 2026-02-20T23:43:25.477 INFO:teuthology.orchestra.run.trial138.stdout:ID="centos" 2026-02-20T23:43:25.477 INFO:teuthology.orchestra.run.trial138.stdout:ID_LIKE="rhel fedora" 2026-02-20T23:43:25.477 INFO:teuthology.orchestra.run.trial138.stdout:VERSION_ID="9" 2026-02-20T23:43:25.477 INFO:teuthology.orchestra.run.trial138.stdout:PLATFORM_ID="platform:el9" 2026-02-20T23:43:25.477 INFO:teuthology.orchestra.run.trial138.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-20T23:43:25.477 INFO:teuthology.orchestra.run.trial138.stdout:ANSI_COLOR="0;31" 2026-02-20T23:43:25.477 INFO:teuthology.orchestra.run.trial138.stdout:LOGO="fedora-logo-icon" 2026-02-20T23:43:25.477 INFO:teuthology.orchestra.run.trial138.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-20T23:43:25.477 INFO:teuthology.orchestra.run.trial138.stdout:HOME_URL="https://centos.org/" 2026-02-20T23:43:25.478 INFO:teuthology.orchestra.run.trial138.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-20T23:43:25.478 INFO:teuthology.orchestra.run.trial138.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-20T23:43:25.478 INFO:teuthology.orchestra.run.trial138.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-20T23:43:25.478 INFO:teuthology.lock.ops:Updating trial138.front.sepia.ceph.com on lock server 2026-02-20T23:43:25.580 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-20T23:43:25.597 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-20T23:43:25.608 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-20T23:43:25.609 DEBUG:teuthology.orchestra.run.trial138:> test '!' -e /home/ubuntu/cephtest 2026-02-20T23:43:25.624 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-20T23:43:25.636 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-20T23:43:25.636 DEBUG:teuthology.orchestra.run.trial138:> test -z $(ls -A /var/lib/ceph) 2026-02-20T23:43:25.680 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-20T23:43:25.918 INFO:teuthology.run_tasks:Running task kernel... 2026-02-20T23:43:25.947 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': True, 'sha1': 'distro'} 2026-02-20T23:43:25.947 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, timeout 300 2026-02-20T23:43:25.947 DEBUG:teuthology.orchestra.run.trial138:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-20T23:43:25.963 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T23:43:25.963 DEBUG:teuthology.orchestra.run.trial138:> uname -r 2026-02-20T23:43:26.020 INFO:teuthology.orchestra.run.trial138.stdout:5.14.0-681.el9.x86_64 2026-02-20T23:43:26.020 INFO:teuthology.task.kernel:Running kernel on trial138: 5.14.0-681.el9.x86_64 2026-02-20T23:43:26.020 DEBUG:teuthology.orchestra.run.trial138:> sudo yum install -y kernel 2026-02-20T23:43:28.025 INFO:teuthology.orchestra.run.trial138.stdout:CentOS Stream 9 - BaseOS 5.5 MB/s | 8.9 MB 00:01 2026-02-20T23:43:31.502 INFO:teuthology.orchestra.run.trial138.stdout:CentOS Stream 9 - AppStream 9.1 MB/s | 27 MB 00:02 2026-02-20T23:43:34.970 INFO:teuthology.orchestra.run.trial138.stdout:CentOS Stream 9 - CRB 7.0 MB/s | 7.8 MB 00:01 2026-02-20T23:43:36.048 INFO:teuthology.orchestra.run.trial138.stdout:CentOS Stream 9 - Extras packages 58 kB/s | 20 kB 00:00 2026-02-20T23:43:36.367 INFO:teuthology.orchestra.run.trial138.stdout:Extra Packages for Enterprise Linux 79 MB/s | 20 MB 00:00 2026-02-20T23:43:39.571 INFO:teuthology.orchestra.run.trial138.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-20T23:43:40.575 INFO:teuthology.orchestra.run.trial138.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-20T23:43:40.575 INFO:teuthology.orchestra.run.trial138.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-20T23:43:40.594 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:43:40.594 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-20T23:43:40.594 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:43:40.619 DEBUG:teuthology.orchestra.run.trial138:> echo no | sudo yum reinstall kernel || true 2026-02-20T23:43:40.918 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:01 ago on Fri 20 Feb 2026 11:43:39 PM UTC. 2026-02-20T23:43:40.998 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:43:40.999 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:43:40.999 INFO:teuthology.orchestra.run.trial138.stdout: Package Architecture Version Repository Size 2026-02-20T23:43:40.999 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:43:40.999 INFO:teuthology.orchestra.run.trial138.stdout:Reinstalling: 2026-02-20T23:43:41.000 INFO:teuthology.orchestra.run.trial138.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T23:43:41.000 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:43:41.000 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:43:41.000 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:43:41.000 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:43:41.000 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 803 k 2026-02-20T23:43:41.000 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 0 2026-02-20T23:43:41.000 INFO:teuthology.orchestra.run.trial138.stderr:Operation aborted. 2026-02-20T23:43:41.019 INFO:teuthology.orchestra.run.trial138.stdout:Is this ok [y/N]: 2026-02-20T23:43:41.019 DEBUG:teuthology.orchestra.run.trial138:> sudo yum reinstall -y kernel || true 2026-02-20T23:43:41.318 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 11:43:39 PM UTC. 2026-02-20T23:43:41.397 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:43:41.398 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:43:41.398 INFO:teuthology.orchestra.run.trial138.stdout: Package Architecture Version Repository Size 2026-02-20T23:43:41.398 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:43:41.398 INFO:teuthology.orchestra.run.trial138.stdout:Reinstalling: 2026-02-20T23:43:41.398 INFO:teuthology.orchestra.run.trial138.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T23:43:41.398 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:43:41.398 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:43:41.398 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:43:41.398 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:43:41.398 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 803 k 2026-02-20T23:43:41.398 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 0 2026-02-20T23:43:41.399 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:43:42.559 INFO:teuthology.orchestra.run.trial138.stdout:kernel-5.14.0-681.el9.x86_64.rpm 899 kB/s | 803 kB 00:00 2026-02-20T23:43:42.560 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:43:42.560 INFO:teuthology.orchestra.run.trial138.stdout:Total 692 kB/s | 803 kB 00:01 2026-02-20T23:43:42.570 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:43:42.603 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:43:42.603 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:43:42.608 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:43:42.608 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:43:42.631 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:43:42.636 INFO:teuthology.orchestra.run.trial138.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T23:43:42.636 INFO:teuthology.orchestra.run.trial138.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T23:43:42.748 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T23:43:42.748 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T23:43:42.802 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T23:43:42.802 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:43:42.802 INFO:teuthology.orchestra.run.trial138.stdout:Reinstalled: 2026-02-20T23:43:42.802 INFO:teuthology.orchestra.run.trial138.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-20T23:43:42.803 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:43:42.803 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:43:42.831 DEBUG:teuthology.orchestra.run.trial138:> rpm -q kernel | sort -rV | head -n 1 2026-02-20T23:43:42.855 INFO:teuthology.orchestra.run.trial138.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-20T23:43:42.856 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-20T23:43:42.856 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-20T23:43:42.856 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-20T23:43:42.856 DEBUG:teuthology.orchestra.run.trial138:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-20T23:43:42.919 INFO:teuthology.orchestra.run.trial138.stdout:ttyS1 2026-02-20T23:43:42.947 DEBUG:teuthology.parallel:result is None 2026-02-20T23:43:42.947 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-20T23:43:42.958 INFO:teuthology.task.internal:Creating test directory... 2026-02-20T23:43:42.958 DEBUG:teuthology.orchestra.run.trial138:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-20T23:43:43.002 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-20T23:43:43.014 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-20T23:43:43.027 INFO:teuthology.task.internal:Creating archive directory... 2026-02-20T23:43:43.027 DEBUG:teuthology.orchestra.run.trial138:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-20T23:43:43.061 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-20T23:43:43.073 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-20T23:43:43.073 DEBUG:teuthology.orchestra.run.trial138:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-20T23:43:43.122 INFO:teuthology.orchestra.run.trial138.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T23:43:43.131 INFO:teuthology.orchestra.run.trial138.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T23:43:43.132 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-20T23:43:43.144 INFO:teuthology.task.internal:Configuring sudo... 2026-02-20T23:43:43.144 DEBUG:teuthology.orchestra.run.trial138:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-20T23:43:43.196 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-20T23:43:43.210 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-20T23:43:43.210 DEBUG:teuthology.orchestra.run.trial138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-20T23:43:43.252 DEBUG:teuthology.orchestra.run.trial138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T23:43:43.416 DEBUG:teuthology.orchestra.run.trial138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T23:43:43.440 DEBUG:teuthology.orchestra.run.trial138:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T23:43:43.568 DEBUG:teuthology.orchestra.run.trial138:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T23:43:43.591 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-20T23:43:43.591 DEBUG:teuthology.orchestra.run.trial138:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-20T23:43:43.655 DEBUG:teuthology.orchestra.run.trial138:> sudo service rsyslog restart 2026-02-20T23:43:43.723 INFO:teuthology.orchestra.run.trial138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-20T23:43:44.131 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-20T23:43:44.144 INFO:teuthology.task.internal:Starting timer... 2026-02-20T23:43:44.144 INFO:teuthology.run_tasks:Running task pcp... 2026-02-20T23:43:44.167 INFO:teuthology.run_tasks:Running task selinux... 2026-02-20T23:43:44.185 DEBUG:teuthology.orchestra.run.trial138:> sudo service auditd rotate 2026-02-20T23:43:44.213 INFO:teuthology.orchestra.run.trial138.stdout:Rotating logs: 2026-02-20T23:43:44.215 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-20T23:43:44.216 DEBUG:teuthology.orchestra.run.trial138:> /usr/sbin/getenforce 2026-02-20T23:43:44.272 INFO:teuthology.orchestra.run.trial138.stdout:Permissive 2026-02-20T23:43:44.272 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial138.front.sepia.ceph.com': 'permissive'} 2026-02-20T23:43:44.272 DEBUG:teuthology.orchestra.run.trial138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-20T23:43:44.336 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T23:43:44.337 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-20T23:43:44.337 DEBUG:teuthology.orchestra.run.trial138:> sudo /usr/sbin/setenforce permissive 2026-02-20T23:43:44.400 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-20T23:43:44.416 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-20T23:43:44.429 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-20T23:43:44.431 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial138.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-20T23:46:37.324 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial138.front.sepia.ceph.com')] 2026-02-20T23:46:37.326 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial138.front.sepia.ceph.com' 2026-02-20T23:46:37.328 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial138.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T23:46:37.440 DEBUG:teuthology.orchestra.run.trial138:> true 2026-02-20T23:46:37.515 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial138.front.sepia.ceph.com' 2026-02-20T23:46:37.515 INFO:teuthology.run_tasks:Running task clock... 2026-02-20T23:46:37.532 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-20T23:46:37.533 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-20T23:46:37.533 DEBUG:teuthology.orchestra.run.trial138:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-20T23:46:37.583 INFO:teuthology.orchestra.run.trial138.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-20T23:46:37.595 INFO:teuthology.orchestra.run.trial138.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-20T23:46:37.615 INFO:teuthology.orchestra.run.trial138.stderr:sudo: ntpd: command not found 2026-02-20T23:46:37.625 INFO:teuthology.orchestra.run.trial138.stdout:506 Cannot talk to daemon 2026-02-20T23:46:37.637 INFO:teuthology.orchestra.run.trial138.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-20T23:46:37.649 INFO:teuthology.orchestra.run.trial138.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-20T23:46:37.699 INFO:teuthology.orchestra.run.trial138.stderr:bash: line 1: ntpq: command not found 2026-02-20T23:46:37.721 INFO:teuthology.orchestra.run.trial138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-20T23:46:37.721 INFO:teuthology.orchestra.run.trial138.stdout:=============================================================================== 2026-02-20T23:46:37.721 INFO:teuthology.orchestra.run.trial138.stdout:^? ntp.netlinkify.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-20T23:46:37.721 INFO:teuthology.orchestra.run.trial138.stdout:^? 172-235-154-118.ip.linod> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-20T23:46:37.721 INFO:teuthology.orchestra.run.trial138.stdout:^? time2.beanman.net 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-20T23:46:37.722 INFO:teuthology.orchestra.run.trial138.stdout:^? 172-233-177-198.ip.linod> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-20T23:46:37.723 INFO:teuthology.run_tasks:Running task install... 2026-02-20T23:46:37.736 DEBUG:teuthology.task.install:project ceph 2026-02-20T23:46:37.736 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}} 2026-02-20T23:46:37.736 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'} 2026-02-20T23:46:37.736 INFO:teuthology.task.install:Using flavor: debug 2026-02-20T23:46:37.740 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-20T23:46:37.740 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-20T23:46:37.740 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'tag': None, 'wait_for_package': False} 2026-02-20T23:46:37.740 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-20T23:46:37.895 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/ 2026-02-20T23:46:37.896 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5343.gbcdd429b 2026-02-20T23:46:38.277 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-20T23:46:38.277 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-20T23:46:38.277 DEBUG:teuthology.orchestra.run.trial138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-20T23:46:38.303 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-20T23:46:38.303 DEBUG:teuthology.orchestra.run.trial138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-20T23:46:38.369 DEBUG:teuthology.orchestra.run.trial138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-20T23:46:38.441 DEBUG:teuthology.orchestra.run.trial138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-20T23:46:38.507 INFO:teuthology.orchestra.run.trial138.stdout:check_obsoletes = 1 2026-02-20T23:46:38.509 DEBUG:teuthology.orchestra.run.trial138:> sudo yum clean all 2026-02-20T23:46:38.668 INFO:teuthology.orchestra.run.trial138.stdout:41 files removed 2026-02-20T23:46:38.682 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-radosgw 2026-02-20T23:46:39.542 INFO:teuthology.orchestra.run.trial138.stdout:ceph packages for x86_64 128 kB/s | 92 kB 00:00 2026-02-20T23:46:40.207 INFO:teuthology.orchestra.run.trial138.stdout:ceph noarch packages 37 kB/s | 24 kB 00:00 2026-02-20T23:46:40.795 INFO:teuthology.orchestra.run.trial138.stdout:ceph source packages 3.6 kB/s | 2.1 kB 00:00 2026-02-20T23:46:41.522 INFO:teuthology.orchestra.run.trial138.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.9 MB 00:00 2026-02-20T23:46:44.569 INFO:teuthology.orchestra.run.trial138.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-20T23:46:47.829 INFO:teuthology.orchestra.run.trial138.stdout:CentOS Stream 9 - CRB 8.7 MB/s | 7.8 MB 00:00 2026-02-20T23:46:49.200 INFO:teuthology.orchestra.run.trial138.stdout:CentOS Stream 9 - Extras packages 31 kB/s | 20 kB 00:00 2026-02-20T23:46:50.898 INFO:teuthology.orchestra.run.trial138.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2026-02-20T23:46:54.126 INFO:teuthology.orchestra.run.trial138.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-20T23:46:55.323 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:46:55.324 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:46:55.324 INFO:teuthology.orchestra.run.trial138.stdout: Package Arch Version Repository Size 2026-02-20T23:46:55.324 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:46:55.324 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:46:55.324 INFO:teuthology.orchestra.run.trial138.stdout: ceph-radosgw x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 11 M 2026-02-20T23:46:55.324 INFO:teuthology.orchestra.run.trial138.stdout:Upgrading: 2026-02-20T23:46:55.324 INFO:teuthology.orchestra.run.trial138.stdout: librados2 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 3.3 M 2026-02-20T23:46:55.324 INFO:teuthology.orchestra.run.trial138.stdout: librbd1 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 2.7 M 2026-02-20T23:46:55.324 INFO:teuthology.orchestra.run.trial138.stdout:Installing dependencies: 2026-02-20T23:46:55.324 INFO:teuthology.orchestra.run.trial138.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-20T23:46:55.324 INFO:teuthology.orchestra.run.trial138.stdout: ceph-base x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 5.1 M 2026-02-20T23:46:55.324 INFO:teuthology.orchestra.run.trial138.stdout: ceph-common x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 21 M 2026-02-20T23:46:55.324 INFO:teuthology.orchestra.run.trial138.stdout: ceph-selinux x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 25 k 2026-02-20T23:46:55.324 INFO:teuthology.orchestra.run.trial138.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-20T23:46:55.324 INFO:teuthology.orchestra.run.trial138.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-20T23:46:55.324 INFO:teuthology.orchestra.run.trial138.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: libcephfs2 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 860 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: libradosstriper1 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 263 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: librgw2 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 5.6 M 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: python3-ceph-argparse x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 45 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: python3-ceph-common x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 185 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: python3-cephfs x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 203 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: python3-rados x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 324 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: python3-rbd x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 305 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: python3-rgw x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 100 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout:Installing weak dependencies: 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout:Install 32 Packages 2026-02-20T23:46:55.325 INFO:teuthology.orchestra.run.trial138.stdout:Upgrade 2 Packages 2026-02-20T23:46:55.326 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:46:55.326 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 60 M 2026-02-20T23:46:55.326 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:47:06.983 INFO:teuthology.orchestra.run.trial138.stdout:(1/34): ceph-base-20.3.0-5343.gbcdd429b.el9.x86 468 kB/s | 5.1 MB 00:11 2026-02-20T23:47:07.054 INFO:teuthology.orchestra.run.trial138.stdout:(2/34): ceph-selinux-20.3.0-5343.gbcdd429b.el9. 356 kB/s | 25 kB 00:00 2026-02-20T23:47:08.791 INFO:teuthology.orchestra.run.trial138.stdout:(3/34): libcephfs2-20.3.0-5343.gbcdd429b.el9.x8 495 kB/s | 860 kB 00:01 2026-02-20T23:47:09.211 INFO:teuthology.orchestra.run.trial138.stdout:(4/34): libradosstriper1-20.3.0-5343.gbcdd429b. 625 kB/s | 263 kB 00:00 2026-02-20T23:47:17.246 INFO:teuthology.orchestra.run.trial138.stdout:(5/34): librgw2-20.3.0-5343.gbcdd429b.el9.x86_6 717 kB/s | 5.6 MB 00:08 2026-02-20T23:47:17.315 INFO:teuthology.orchestra.run.trial138.stdout:(6/34): python3-ceph-argparse-20.3.0-5343.gbcdd 658 kB/s | 45 kB 00:00 2026-02-20T23:47:17.592 INFO:teuthology.orchestra.run.trial138.stdout:(7/34): python3-ceph-common-20.3.0-5343.gbcdd42 667 kB/s | 185 kB 00:00 2026-02-20T23:47:17.870 INFO:teuthology.orchestra.run.trial138.stdout:(8/34): python3-cephfs-20.3.0-5343.gbcdd429b.el 730 kB/s | 203 kB 00:00 2026-02-20T23:47:18.417 INFO:teuthology.orchestra.run.trial138.stdout:(9/34): python3-rados-20.3.0-5343.gbcdd429b.el9 594 kB/s | 324 kB 00:00 2026-02-20T23:47:18.974 INFO:teuthology.orchestra.run.trial138.stdout:(10/34): python3-rbd-20.3.0-5343.gbcdd429b.el9. 548 kB/s | 305 kB 00:00 2026-02-20T23:47:19.182 INFO:teuthology.orchestra.run.trial138.stdout:(11/34): python3-rgw-20.3.0-5343.gbcdd429b.el9. 484 kB/s | 100 kB 00:00 2026-02-20T23:47:19.249 INFO:teuthology.orchestra.run.trial138.stdout:(12/34): mailcap-2.1.49-5.el9.noarch.rpm 492 kB/s | 33 kB 00:00 2026-02-20T23:47:19.292 INFO:teuthology.orchestra.run.trial138.stdout:(13/34): pciutils-3.7.0-7.el9.x86_64.rpm 2.1 MB/s | 93 kB 00:00 2026-02-20T23:47:19.696 INFO:teuthology.orchestra.run.trial138.stdout:(14/34): boost-program-options-1.75.0-13.el9.x8 257 kB/s | 104 kB 00:00 2026-02-20T23:47:19.876 INFO:teuthology.orchestra.run.trial138.stdout:(15/34): libnbd-1.20.3-4.el9.x86_64.rpm 915 kB/s | 164 kB 00:00 2026-02-20T23:47:19.990 INFO:teuthology.orchestra.run.trial138.stdout:(16/34): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.4 MB/s | 160 kB 00:00 2026-02-20T23:47:20.077 INFO:teuthology.orchestra.run.trial138.stdout:(17/34): librabbitmq-0.11.0-7.el9.x86_64.rpm 518 kB/s | 45 kB 00:00 2026-02-20T23:47:20.301 INFO:teuthology.orchestra.run.trial138.stdout:(18/34): librdkafka-1.6.1-102.el9.x86_64.rpm 2.9 MB/s | 662 kB 00:00 2026-02-20T23:47:20.400 INFO:teuthology.orchestra.run.trial138.stdout:(19/34): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.9 MB/s | 292 kB 00:00 2026-02-20T23:47:20.495 INFO:teuthology.orchestra.run.trial138.stdout:(20/34): qatlib-25.08.0-2.el9.x86_64.rpm 2.5 MB/s | 240 kB 00:00 2026-02-20T23:47:20.575 INFO:teuthology.orchestra.run.trial138.stdout:(21/34): qatlib-service-25.08.0-2.el9.x86_64.rp 466 kB/s | 37 kB 00:00 2026-02-20T23:47:20.658 INFO:teuthology.orchestra.run.trial138.stdout:(22/34): qatzip-libs-1.3.1-1.el9.x86_64.rpm 803 kB/s | 66 kB 00:00 2026-02-20T23:47:20.772 INFO:teuthology.orchestra.run.trial138.stdout:(23/34): gperftools-libs-2.9.1-3.el9.x86_64.rpm 2.6 MB/s | 308 kB 00:00 2026-02-20T23:47:20.850 INFO:teuthology.orchestra.run.trial138.stdout:(24/34): libarrow-9.0.0-15.el9.x86_64.rpm 57 MB/s | 4.4 MB 00:00 2026-02-20T23:47:20.865 INFO:teuthology.orchestra.run.trial138.stdout:(25/34): libarrow-doc-9.0.0-15.el9.noarch.rpm 1.7 MB/s | 25 kB 00:00 2026-02-20T23:47:20.880 INFO:teuthology.orchestra.run.trial138.stdout:(26/34): liboath-2.6.12-1.el9.x86_64.rpm 3.3 MB/s | 49 kB 00:00 2026-02-20T23:47:20.894 INFO:teuthology.orchestra.run.trial138.stdout:(27/34): libunwind-1.6.2-1.el9.x86_64.rpm 4.4 MB/s | 67 kB 00:00 2026-02-20T23:47:20.912 INFO:teuthology.orchestra.run.trial138.stdout:(28/34): parquet-libs-9.0.0-15.el9.x86_64.rpm 46 MB/s | 838 kB 00:00 2026-02-20T23:47:20.929 INFO:teuthology.orchestra.run.trial138.stdout:(29/34): re2-20211101-20.el9.x86_64.rpm 12 MB/s | 191 kB 00:00 2026-02-20T23:47:20.948 INFO:teuthology.orchestra.run.trial138.stdout:(30/34): thrift-0.15.0-4.el9.x86_64.rpm 80 MB/s | 1.6 MB 00:00 2026-02-20T23:47:21.834 INFO:teuthology.orchestra.run.trial138.stdout:(31/34): ceph-radosgw-20.3.0-5343.gbcdd429b.el9 440 kB/s | 11 MB 00:26 2026-02-20T23:47:25.549 INFO:teuthology.orchestra.run.trial138.stdout:(32/34): ceph-common-20.3.0-5343.gbcdd429b.el9. 717 kB/s | 21 MB 00:29 2026-02-20T23:47:26.794 INFO:teuthology.orchestra.run.trial138.stdout:(33/34): librbd1-20.3.0-5343.gbcdd429b.el9.x86_ 555 kB/s | 2.7 MB 00:04 2026-02-20T23:47:28.810 INFO:teuthology.orchestra.run.trial138.stdout:(34/34): librados2-20.3.0-5343.gbcdd429b.el9.x8 428 kB/s | 3.3 MB 00:07 2026-02-20T23:47:28.813 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:47:28.813 INFO:teuthology.orchestra.run.trial138.stdout:Total 1.8 MB/s | 60 MB 00:33 2026-02-20T23:47:28.901 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:47:28.920 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:47:28.920 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:47:29.166 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:47:29.166 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:47:29.448 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:47:29.590 INFO:teuthology.orchestra.run.trial138.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/36 2026-02-20T23:47:29.594 INFO:teuthology.orchestra.run.trial138.stdout: Upgrading : librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/36 2026-02-20T23:47:29.625 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/36 2026-02-20T23:47:29.662 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 3/36 2026-02-20T23:47:29.663 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 4/36 2026-02-20T23:47:29.675 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 4/36 2026-02-20T23:47:29.683 INFO:teuthology.orchestra.run.trial138.stdout: Installing : liboath-2.6.12-1.el9.x86_64 5/36 2026-02-20T23:47:29.686 INFO:teuthology.orchestra.run.trial138.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/36 2026-02-20T23:47:29.689 INFO:teuthology.orchestra.run.trial138.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/36 2026-02-20T23:47:29.693 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 8/36 2026-02-20T23:47:29.704 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-ceph-argparse-2:20.3.0-5343.gbcdd429b.el9. 9/36 2026-02-20T23:47:29.717 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 10/36 2026-02-20T23:47:29.718 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libradosstriper1-2:20.3.0-5343.gbcdd429b.el9.x86_6 11/36 2026-02-20T23:47:29.743 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5343.gbcdd429b.el9.x86_6 11/36 2026-02-20T23:47:29.747 INFO:teuthology.orchestra.run.trial138.stdout: Installing : thrift-0.15.0-4.el9.x86_64 12/36 2026-02-20T23:47:29.782 INFO:teuthology.orchestra.run.trial138.stdout: Installing : re2-1:20211101-20.el9.x86_64 13/36 2026-02-20T23:47:29.785 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 14/36 2026-02-20T23:47:29.791 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 15/36 2026-02-20T23:47:29.794 INFO:teuthology.orchestra.run.trial138.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 16/36 2026-02-20T23:47:29.799 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 17/36 2026-02-20T23:47:29.916 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/36 2026-02-20T23:47:29.917 INFO:teuthology.orchestra.run.trial138.stdout: Upgrading : librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 19/36 2026-02-20T23:47:29.943 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 19/36 2026-02-20T23:47:29.947 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 20/36 2026-02-20T23:47:29.953 INFO:teuthology.orchestra.run.trial138.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/36 2026-02-20T23:47:30.206 INFO:teuthology.orchestra.run.trial138.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/36 2026-02-20T23:47:30.207 INFO:teuthology.orchestra.run.trial138.stdout: Installing : librgw2-2:20.3.0-5343.gbcdd429b.el9.x86_64 23/36 2026-02-20T23:47:30.224 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: librgw2-2:20.3.0-5343.gbcdd429b.el9.x86_64 23/36 2026-02-20T23:47:30.228 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 24/36 2026-02-20T23:47:30.233 INFO:teuthology.orchestra.run.trial138.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/36 2026-02-20T23:47:30.236 INFO:teuthology.orchestra.run.trial138.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/36 2026-02-20T23:47:30.236 INFO:teuthology.orchestra.run.trial138.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-20T23:47:30.255 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-20T23:47:30.258 INFO:teuthology.orchestra.run.trial138.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/36 2026-02-20T23:47:30.292 INFO:teuthology.orchestra.run.trial138.stdout: Installing : mailcap-2.1.49-5.el9.noarch 29/36 2026-02-20T23:47:30.294 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-ceph-common-2:20.3.0-5343.gbcdd429b.el9.x8 30/36 2026-02-20T23:47:31.247 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 31/36 2026-02-20T23:47:31.250 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 31/36 2026-02-20T23:47:31.515 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 31/36 2026-02-20T23:47:31.519 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 32/36 2026-02-20T23:47:31.560 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 32/36 2026-02-20T23:47:31.560 INFO:teuthology.orchestra.run.trial138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-20T23:47:31.560 INFO:teuthology.orchestra.run.trial138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-20T23:47:31.560 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:47:31.563 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-selinux-2:20.3.0-5343.gbcdd429b.el9.x86_64 33/36 2026-02-20T23:47:36.032 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5343.gbcdd429b.el9.x86_64 33/36 2026-02-20T23:47:36.033 INFO:teuthology.orchestra.run.trial138.stdout:skipping the directory /sys 2026-02-20T23:47:36.033 INFO:teuthology.orchestra.run.trial138.stdout:skipping the directory /proc 2026-02-20T23:47:36.033 INFO:teuthology.orchestra.run.trial138.stdout:skipping the directory /mnt 2026-02-20T23:47:36.033 INFO:teuthology.orchestra.run.trial138.stdout:skipping the directory /var/tmp 2026-02-20T23:47:36.033 INFO:teuthology.orchestra.run.trial138.stdout:skipping the directory /home 2026-02-20T23:47:36.033 INFO:teuthology.orchestra.run.trial138.stdout:skipping the directory /root 2026-02-20T23:47:36.033 INFO:teuthology.orchestra.run.trial138.stdout:skipping the directory /tmp 2026-02-20T23:47:36.033 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:47:36.536 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 34/36 2026-02-20T23:47:36.560 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 34/36 2026-02-20T23:47:36.560 INFO:teuthology.orchestra.run.trial138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:47:36.560 INFO:teuthology.orchestra.run.trial138.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-20T23:47:36.560 INFO:teuthology.orchestra.run.trial138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T23:47:36.560 INFO:teuthology.orchestra.run.trial138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T23:47:36.560 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:47:36.561 INFO:teuthology.orchestra.run.trial138.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-20T23:47:36.574 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-20T23:47:36.574 INFO:teuthology.orchestra.run.trial138.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-20T23:47:36.629 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-20T23:47:36.629 INFO:teuthology.orchestra.run.trial138.stdout:Creating group 'qat' with GID 992. 2026-02-20T23:47:36.629 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:47:36.916 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/36 2026-02-20T23:47:36.916 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/36 2026-02-20T23:47:36.916 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 3/36 2026-02-20T23:47:36.917 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-selinux-2:20.3.0-5343.gbcdd429b.el9.x86_64 4/36 2026-02-20T23:47:36.917 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 5/36 2026-02-20T23:47:36.917 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libradosstriper1-2:20.3.0-5343.gbcdd429b.el9.x86_6 6/36 2026-02-20T23:47:36.917 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : librgw2-2:20.3.0-5343.gbcdd429b.el9.x86_64 7/36 2026-02-20T23:47:36.917 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5343.gbcdd429b.el9. 8/36 2026-02-20T23:47:36.917 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-ceph-common-2:20.3.0-5343.gbcdd429b.el9.x8 9/36 2026-02-20T23:47:36.917 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 10/36 2026-02-20T23:47:36.917 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 11/36 2026-02-20T23:47:36.917 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 12/36 2026-02-20T23:47:36.917 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 13/36 2026-02-20T23:47:36.917 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/36 2026-02-20T23:47:36.917 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 15/36 2026-02-20T23:47:36.918 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 16/36 2026-02-20T23:47:36.918 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 17/36 2026-02-20T23:47:36.918 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 18/36 2026-02-20T23:47:36.918 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/36 2026-02-20T23:47:36.918 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 20/36 2026-02-20T23:47:36.918 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 21/36 2026-02-20T23:47:36.918 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 22/36 2026-02-20T23:47:36.918 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 23/36 2026-02-20T23:47:36.918 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 24/36 2026-02-20T23:47:36.918 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 25/36 2026-02-20T23:47:36.918 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 26/36 2026-02-20T23:47:36.918 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 27/36 2026-02-20T23:47:36.918 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 28/36 2026-02-20T23:47:36.919 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 29/36 2026-02-20T23:47:36.919 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 30/36 2026-02-20T23:47:36.919 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : re2-1:20211101-20.el9.x86_64 31/36 2026-02-20T23:47:36.919 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 32/36 2026-02-20T23:47:36.919 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 33/36 2026-02-20T23:47:36.919 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 34/36 2026-02-20T23:47:36.919 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 35/36 2026-02-20T23:47:36.976 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 36/36 2026-02-20T23:47:36.976 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:47:36.976 INFO:teuthology.orchestra.run.trial138.stdout:Upgraded: 2026-02-20T23:47:36.977 INFO:teuthology.orchestra.run.trial138.stdout: librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:36.977 INFO:teuthology.orchestra.run.trial138.stdout: librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:36.977 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-20T23:47:36.977 INFO:teuthology.orchestra.run.trial138.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-20T23:47:36.977 INFO:teuthology.orchestra.run.trial138.stdout: ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:36.977 INFO:teuthology.orchestra.run.trial138.stdout: ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:36.977 INFO:teuthology.orchestra.run.trial138.stdout: ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:36.977 INFO:teuthology.orchestra.run.trial138.stdout: ceph-selinux-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:36.977 INFO:teuthology.orchestra.run.trial138.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-20T23:47:36.977 INFO:teuthology.orchestra.run.trial138.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-20T23:47:36.977 INFO:teuthology.orchestra.run.trial138.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-20T23:47:36.977 INFO:teuthology.orchestra.run.trial138.stdout: libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:36.977 INFO:teuthology.orchestra.run.trial138.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-20T23:47:36.978 INFO:teuthology.orchestra.run.trial138.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-20T23:47:36.978 INFO:teuthology.orchestra.run.trial138.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-20T23:47:36.978 INFO:teuthology.orchestra.run.trial138.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-20T23:47:36.978 INFO:teuthology.orchestra.run.trial138.stdout: libradosstriper1-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:36.978 INFO:teuthology.orchestra.run.trial138.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-20T23:47:36.978 INFO:teuthology.orchestra.run.trial138.stdout: librgw2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:36.978 INFO:teuthology.orchestra.run.trial138.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-20T23:47:36.978 INFO:teuthology.orchestra.run.trial138.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-20T23:47:36.978 INFO:teuthology.orchestra.run.trial138.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-20T23:47:36.978 INFO:teuthology.orchestra.run.trial138.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-20T23:47:36.978 INFO:teuthology.orchestra.run.trial138.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-20T23:47:36.978 INFO:teuthology.orchestra.run.trial138.stdout: python3-ceph-argparse-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:36.979 INFO:teuthology.orchestra.run.trial138.stdout: python3-ceph-common-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:36.979 INFO:teuthology.orchestra.run.trial138.stdout: python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:36.979 INFO:teuthology.orchestra.run.trial138.stdout: python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:36.979 INFO:teuthology.orchestra.run.trial138.stdout: python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:36.979 INFO:teuthology.orchestra.run.trial138.stdout: python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:47:36.979 INFO:teuthology.orchestra.run.trial138.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-20T23:47:36.979 INFO:teuthology.orchestra.run.trial138.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-20T23:47:36.979 INFO:teuthology.orchestra.run.trial138.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-20T23:47:36.979 INFO:teuthology.orchestra.run.trial138.stdout: re2-1:20211101-20.el9.x86_64 2026-02-20T23:47:36.979 INFO:teuthology.orchestra.run.trial138.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-20T23:47:36.979 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:47:36.979 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:47:37.016 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-test 2026-02-20T23:47:37.327 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:43 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:47:37.411 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:47:37.411 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:47:37.411 INFO:teuthology.orchestra.run.trial138.stdout: Package Arch Version Repository Size 2026-02-20T23:47:37.411 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:47:37.412 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:47:37.412 INFO:teuthology.orchestra.run.trial138.stdout: ceph-test x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 126 M 2026-02-20T23:47:37.412 INFO:teuthology.orchestra.run.trial138.stdout:Installing dependencies: 2026-02-20T23:47:37.412 INFO:teuthology.orchestra.run.trial138.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-20T23:47:37.412 INFO:teuthology.orchestra.run.trial138.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-20T23:47:37.412 INFO:teuthology.orchestra.run.trial138.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-20T23:47:37.412 INFO:teuthology.orchestra.run.trial138.stdout: libcephsqlite x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 153 k 2026-02-20T23:47:37.412 INFO:teuthology.orchestra.run.trial138.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-20T23:47:37.412 INFO:teuthology.orchestra.run.trial138.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-20T23:47:37.412 INFO:teuthology.orchestra.run.trial138.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-20T23:47:37.412 INFO:teuthology.orchestra.run.trial138.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-20T23:47:37.412 INFO:teuthology.orchestra.run.trial138.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-20T23:47:37.413 INFO:teuthology.orchestra.run.trial138.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-20T23:47:37.413 INFO:teuthology.orchestra.run.trial138.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-20T23:47:37.413 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:47:37.413 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:47:37.413 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:47:37.413 INFO:teuthology.orchestra.run.trial138.stdout:Install 12 Packages 2026-02-20T23:47:37.413 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:47:37.414 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 131 M 2026-02-20T23:47:37.414 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 981 M 2026-02-20T23:47:37.414 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:47:38.794 INFO:teuthology.orchestra.run.trial138.stdout:(1/12): libcephsqlite-20.3.0-5343.gbcdd429b.el9 170 kB/s | 153 kB 00:00 2026-02-20T23:47:38.893 INFO:teuthology.orchestra.run.trial138.stdout:(2/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 2.1 MB/s | 2.1 MB 00:00 2026-02-20T23:47:39.318 INFO:teuthology.orchestra.run.trial138.stdout:(3/12): libatomic-11.5.0-14.el9.x86_64.rpm 44 kB/s | 23 kB 00:00 2026-02-20T23:47:39.377 INFO:teuthology.orchestra.run.trial138.stdout:(4/12): libasan8-15.2.1-7.el9.x86_64.rpm 8.8 MB/s | 525 kB 00:00 2026-02-20T23:47:39.387 INFO:teuthology.orchestra.run.trial138.stdout:(5/12): libubsan-11.5.0-14.el9.x86_64.rpm 16 MB/s | 176 kB 00:00 2026-02-20T23:47:39.398 INFO:teuthology.orchestra.run.trial138.stdout:(6/12): libxslt-1.1.34-12.el9.x86_64.rpm 21 MB/s | 233 kB 00:00 2026-02-20T23:47:39.427 INFO:teuthology.orchestra.run.trial138.stdout:(7/12): protobuf-3.14.0-17.el9.x86_64.rpm 35 MB/s | 1.0 MB 00:00 2026-02-20T23:47:39.438 INFO:teuthology.orchestra.run.trial138.stdout:(8/12): socat-1.7.4.1-8.el9.x86_64.rpm 27 MB/s | 303 kB 00:00 2026-02-20T23:47:39.448 INFO:teuthology.orchestra.run.trial138.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 6.6 MB/s | 64 kB 00:00 2026-02-20T23:47:39.545 INFO:teuthology.orchestra.run.trial138.stdout:(10/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 144 kB/s | 94 kB 00:00 2026-02-20T23:47:39.831 INFO:teuthology.orchestra.run.trial138.stdout:(11/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 318 kB/s | 122 kB 00:00 2026-02-20T23:50:31.646 INFO:teuthology.orchestra.run.trial138.stdout:(12/12): ceph-test-20.3.0-5343.gbcdd429b.el9.x8 744 kB/s | 126 MB 02:53 2026-02-20T23:50:31.655 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:50:31.656 INFO:teuthology.orchestra.run.trial138.stdout:Total 770 kB/s | 131 MB 02:54 2026-02-20T23:50:31.717 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:50:31.727 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:50:31.728 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:50:32.154 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:50:32.155 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:50:32.610 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:50:32.618 INFO:teuthology.orchestra.run.trial138.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-20T23:50:32.627 INFO:teuthology.orchestra.run.trial138.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-20T23:50:32.632 INFO:teuthology.orchestra.run.trial138.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-20T23:50:32.636 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-20T23:50:32.638 INFO:teuthology.orchestra.run.trial138.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-20T23:50:32.643 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-20T23:50:32.643 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-20T23:50:32.661 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-20T23:50:32.665 INFO:teuthology.orchestra.run.trial138.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-20T23:50:32.675 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-20T23:50:32.683 INFO:teuthology.orchestra.run.trial138.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-20T23:50:32.684 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libcephsqlite-2:20.3.0-5343.gbcdd429b.el9.x86_64 11/12 2026-02-20T23:50:39.599 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5343.gbcdd429b.el9.x86_64 11/12 2026-02-20T23:50:39.617 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 12/12 2026-02-20T23:50:40.436 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 12/12 2026-02-20T23:50:40.437 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/12 2026-02-20T23:50:40.437 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libcephsqlite-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/12 2026-02-20T23:50:40.437 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-20T23:50:40.437 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-20T23:50:40.437 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-20T23:50:40.437 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-20T23:50:40.437 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-20T23:50:40.437 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-20T23:50:40.437 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-20T23:50:40.437 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-20T23:50:40.438 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-20T23:50:40.492 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-20T23:50:40.492 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:50:40.492 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-20T23:50:40.493 INFO:teuthology.orchestra.run.trial138.stdout: ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:50:40.493 INFO:teuthology.orchestra.run.trial138.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-20T23:50:40.493 INFO:teuthology.orchestra.run.trial138.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-20T23:50:40.493 INFO:teuthology.orchestra.run.trial138.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-20T23:50:40.493 INFO:teuthology.orchestra.run.trial138.stdout: libcephsqlite-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:50:40.493 INFO:teuthology.orchestra.run.trial138.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-20T23:50:40.493 INFO:teuthology.orchestra.run.trial138.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-20T23:50:40.493 INFO:teuthology.orchestra.run.trial138.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-20T23:50:40.493 INFO:teuthology.orchestra.run.trial138.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-20T23:50:40.493 INFO:teuthology.orchestra.run.trial138.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-20T23:50:40.493 INFO:teuthology.orchestra.run.trial138.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-20T23:50:40.493 INFO:teuthology.orchestra.run.trial138.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-20T23:50:40.494 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:50:40.494 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:50:40.533 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph 2026-02-20T23:50:40.844 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:03:46 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:50:40.927 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:50:40.929 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:50:40.929 INFO:teuthology.orchestra.run.trial138.stdout: Package Arch Version Repository Size 2026-02-20T23:50:40.929 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:50:40.929 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:50:40.929 INFO:teuthology.orchestra.run.trial138.stdout: ceph x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 6.5 k 2026-02-20T23:50:40.929 INFO:teuthology.orchestra.run.trial138.stdout:Installing dependencies: 2026-02-20T23:50:40.929 INFO:teuthology.orchestra.run.trial138.stdout: ceph-mds x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 2.0 M 2026-02-20T23:50:40.929 INFO:teuthology.orchestra.run.trial138.stdout: ceph-mgr x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 937 k 2026-02-20T23:50:40.930 INFO:teuthology.orchestra.run.trial138.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 297 k 2026-02-20T23:50:40.930 INFO:teuthology.orchestra.run.trial138.stdout: ceph-mon x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 4.9 M 2026-02-20T23:50:40.930 INFO:teuthology.orchestra.run.trial138.stdout: ceph-osd x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 163 k 2026-02-20T23:50:40.930 INFO:teuthology.orchestra.run.trial138.stdout: ceph-osd-classic x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 15 M 2026-02-20T23:50:40.930 INFO:teuthology.orchestra.run.trial138.stdout: ceph-osd-crimson x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 88 M 2026-02-20T23:50:40.930 INFO:teuthology.orchestra.run.trial138.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-20T23:50:40.930 INFO:teuthology.orchestra.run.trial138.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-20T23:50:40.930 INFO:teuthology.orchestra.run.trial138.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-20T23:50:40.930 INFO:teuthology.orchestra.run.trial138.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-20T23:50:40.930 INFO:teuthology.orchestra.run.trial138.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-20T23:50:40.930 INFO:teuthology.orchestra.run.trial138.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-20T23:50:40.930 INFO:teuthology.orchestra.run.trial138.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-20T23:50:40.930 INFO:teuthology.orchestra.run.trial138.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-20T23:50:40.931 INFO:teuthology.orchestra.run.trial138.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-20T23:50:40.931 INFO:teuthology.orchestra.run.trial138.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-20T23:50:40.931 INFO:teuthology.orchestra.run.trial138.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-20T23:50:40.931 INFO:teuthology.orchestra.run.trial138.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-20T23:50:40.931 INFO:teuthology.orchestra.run.trial138.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-20T23:50:40.931 INFO:teuthology.orchestra.run.trial138.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-20T23:50:40.931 INFO:teuthology.orchestra.run.trial138.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-20T23:50:40.931 INFO:teuthology.orchestra.run.trial138.stdout: python3-jaraco-collections 2026-02-20T23:50:40.931 INFO:teuthology.orchestra.run.trial138.stdout: noarch 3.0.0-8.el9 epel 23 k 2026-02-20T23:50:40.931 INFO:teuthology.orchestra.run.trial138.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-20T23:50:40.931 INFO:teuthology.orchestra.run.trial138.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-20T23:50:40.931 INFO:teuthology.orchestra.run.trial138.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-20T23:50:40.932 INFO:teuthology.orchestra.run.trial138.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-20T23:50:40.932 INFO:teuthology.orchestra.run.trial138.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-20T23:50:40.932 INFO:teuthology.orchestra.run.trial138.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-20T23:50:40.932 INFO:teuthology.orchestra.run.trial138.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-20T23:50:40.932 INFO:teuthology.orchestra.run.trial138.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-20T23:50:40.932 INFO:teuthology.orchestra.run.trial138.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-20T23:50:40.932 INFO:teuthology.orchestra.run.trial138.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-20T23:50:40.932 INFO:teuthology.orchestra.run.trial138.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-20T23:50:40.932 INFO:teuthology.orchestra.run.trial138.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-20T23:50:40.932 INFO:teuthology.orchestra.run.trial138.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-20T23:50:40.932 INFO:teuthology.orchestra.run.trial138.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-20T23:50:40.932 INFO:teuthology.orchestra.run.trial138.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-20T23:50:40.932 INFO:teuthology.orchestra.run.trial138.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-20T23:50:40.933 INFO:teuthology.orchestra.run.trial138.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-20T23:50:40.933 INFO:teuthology.orchestra.run.trial138.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-20T23:50:40.933 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:50:40.933 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:50:40.933 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:50:40.933 INFO:teuthology.orchestra.run.trial138.stdout:Install 42 Packages 2026-02-20T23:50:40.933 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:50:40.934 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 115 M 2026-02-20T23:50:40.934 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 974 M 2026-02-20T23:50:40.934 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:50:41.791 INFO:teuthology.orchestra.run.trial138.stdout:(1/42): ceph-20.3.0-5343.gbcdd429b.el9.x86_64.r 22 kB/s | 6.5 kB 00:00 2026-02-20T23:50:45.234 INFO:teuthology.orchestra.run.trial138.stdout:(2/42): ceph-mgr-20.3.0-5343.gbcdd429b.el9.x86_ 251 kB/s | 937 kB 00:03 2026-02-20T23:50:45.587 INFO:teuthology.orchestra.run.trial138.stdout:(3/42): ceph-osd-20.3.0-5343.gbcdd429b.el9.x86_ 462 kB/s | 163 kB 00:00 2026-02-20T23:50:46.325 INFO:teuthology.orchestra.run.trial138.stdout:(4/42): ceph-mds-20.3.0-5343.gbcdd429b.el9.x86_ 429 kB/s | 2.0 MB 00:04 2026-02-20T23:50:49.626 INFO:teuthology.orchestra.run.trial138.stdout:(5/42): ceph-mon-20.3.0-5343.gbcdd429b.el9.x86_ 636 kB/s | 4.9 MB 00:07 2026-02-20T23:50:50.114 INFO:teuthology.orchestra.run.trial138.stdout:(6/42): ceph-mgr-modules-core-20.3.0-5343.gbcdd 609 kB/s | 297 kB 00:00 2026-02-20T23:50:50.146 INFO:teuthology.orchestra.run.trial138.stdout:(7/42): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.2 MB/s | 40 kB 00:00 2026-02-20T23:50:50.168 INFO:teuthology.orchestra.run.trial138.stdout:(8/42): libconfig-1.7.2-9.el9.x86_64.rpm 3.2 MB/s | 72 kB 00:00 2026-02-20T23:50:50.195 INFO:teuthology.orchestra.run.trial138.stdout:(9/42): python3-cffi-1.14.5-5.el9.x86_64.rpm 9.4 MB/s | 253 kB 00:00 2026-02-20T23:50:50.235 INFO:teuthology.orchestra.run.trial138.stdout:(10/42): python3-cryptography-36.0.1-5.el9.x86_ 31 MB/s | 1.2 MB 00:00 2026-02-20T23:50:50.246 INFO:teuthology.orchestra.run.trial138.stdout:(11/42): python3-ply-3.11-14.el9.noarch.rpm 8.9 MB/s | 106 kB 00:00 2026-02-20T23:50:50.258 INFO:teuthology.orchestra.run.trial138.stdout:(12/42): python3-pycparser-2.20-6.el9.noarch.rp 11 MB/s | 135 kB 00:00 2026-02-20T23:50:50.270 INFO:teuthology.orchestra.run.trial138.stdout:(13/42): python3-pyparsing-2.4.7-9.el9.noarch.r 12 MB/s | 150 kB 00:00 2026-02-20T23:50:50.283 INFO:teuthology.orchestra.run.trial138.stdout:(14/42): python3-requests-2.25.1-10.el9.noarch. 10 MB/s | 126 kB 00:00 2026-02-20T23:50:50.295 INFO:teuthology.orchestra.run.trial138.stdout:(15/42): python3-urllib3-1.26.5-7.el9.noarch.rp 17 MB/s | 218 kB 00:00 2026-02-20T23:50:50.308 INFO:teuthology.orchestra.run.trial138.stdout:(16/42): unzip-6.0-59.el9.x86_64.rpm 14 MB/s | 182 kB 00:00 2026-02-20T23:50:50.320 INFO:teuthology.orchestra.run.trial138.stdout:(17/42): zip-3.0-35.el9.x86_64.rpm 22 MB/s | 266 kB 00:00 2026-02-20T23:50:50.455 INFO:teuthology.orchestra.run.trial138.stdout:(18/42): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 1.8 MB/s | 246 kB 00:00 2026-02-20T23:50:50.510 INFO:teuthology.orchestra.run.trial138.stdout:(19/42): lua-5.4.4-4.el9.x86_64.rpm 3.4 MB/s | 188 kB 00:00 2026-02-20T23:50:50.554 INFO:teuthology.orchestra.run.trial138.stdout:(20/42): python3-libstoragemgmt-1.10.1-1.el9.x8 3.9 MB/s | 177 kB 00:00 2026-02-20T23:50:50.597 INFO:teuthology.orchestra.run.trial138.stdout:(21/42): python3-packaging-20.9-5.el9.noarch.rp 1.8 MB/s | 77 kB 00:00 2026-02-20T23:50:50.637 INFO:teuthology.orchestra.run.trial138.stdout:(22/42): python3-toml-0.10.2-6.el9.noarch.rpm 1.0 MB/s | 42 kB 00:00 2026-02-20T23:50:50.713 INFO:teuthology.orchestra.run.trial138.stdout:(23/42): lua-devel-5.4.4-4.el9.x86_64.rpm 294 kB/s | 22 kB 00:00 2026-02-20T23:50:50.757 INFO:teuthology.orchestra.run.trial138.stdout:(24/42): luarocks-3.9.2-5.el9.noarch.rpm 3.4 MB/s | 151 kB 00:00 2026-02-20T23:50:50.765 INFO:teuthology.orchestra.run.trial138.stdout:(25/42): python3-autocommand-2.2.2-8.el9.noarch 3.5 MB/s | 29 kB 00:00 2026-02-20T23:50:50.773 INFO:teuthology.orchestra.run.trial138.stdout:(26/42): python3-backports-tarfile-1.2.0-1.el9. 7.0 MB/s | 60 kB 00:00 2026-02-20T23:50:50.781 INFO:teuthology.orchestra.run.trial138.stdout:(27/42): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 5.2 MB/s | 43 kB 00:00 2026-02-20T23:50:50.798 INFO:teuthology.orchestra.run.trial138.stdout:(28/42): python3-cheroot-10.0.1-4.el9.noarch.rp 11 MB/s | 173 kB 00:00 2026-02-20T23:50:50.818 INFO:teuthology.orchestra.run.trial138.stdout:(29/42): python3-cherrypy-18.6.1-2.el9.noarch.r 17 MB/s | 358 kB 00:00 2026-02-20T23:50:50.826 INFO:teuthology.orchestra.run.trial138.stdout:(30/42): python3-jaraco-8.2.1-3.el9.noarch.rpm 1.4 MB/s | 11 kB 00:00 2026-02-20T23:50:50.834 INFO:teuthology.orchestra.run.trial138.stdout:(31/42): python3-jaraco-classes-3.2.1-5.el9.noa 2.2 MB/s | 18 kB 00:00 2026-02-20T23:50:50.842 INFO:teuthology.orchestra.run.trial138.stdout:(32/42): python3-jaraco-collections-3.0.0-8.el9 2.9 MB/s | 23 kB 00:00 2026-02-20T23:50:50.849 INFO:teuthology.orchestra.run.trial138.stdout:(33/42): python3-jaraco-context-6.0.1-3.el9.noa 2.5 MB/s | 20 kB 00:00 2026-02-20T23:50:50.857 INFO:teuthology.orchestra.run.trial138.stdout:(34/42): python3-jaraco-functools-3.5.0-2.el9.n 2.5 MB/s | 19 kB 00:00 2026-02-20T23:50:50.866 INFO:teuthology.orchestra.run.trial138.stdout:(35/42): python3-jaraco-text-4.0.0-2.el9.noarch 2.9 MB/s | 26 kB 00:00 2026-02-20T23:50:50.875 INFO:teuthology.orchestra.run.trial138.stdout:(36/42): python3-more-itertools-8.12.0-2.el9.no 9.0 MB/s | 79 kB 00:00 2026-02-20T23:50:50.883 INFO:teuthology.orchestra.run.trial138.stdout:(37/42): python3-portend-3.1.0-2.el9.noarch.rpm 2.1 MB/s | 16 kB 00:00 2026-02-20T23:50:50.892 INFO:teuthology.orchestra.run.trial138.stdout:(38/42): python3-pyOpenSSL-21.0.0-1.el9.noarch. 10 MB/s | 90 kB 00:00 2026-02-20T23:50:50.900 INFO:teuthology.orchestra.run.trial138.stdout:(39/42): python3-tempora-5.0.0-2.el9.noarch.rpm 4.4 MB/s | 36 kB 00:00 2026-02-20T23:50:50.908 INFO:teuthology.orchestra.run.trial138.stdout:(40/42): python3-zc-lockfile-2.0-10.el9.noarch. 2.5 MB/s | 20 kB 00:00 2026-02-20T23:51:02.878 INFO:teuthology.orchestra.run.trial138.stdout:(41/42): ceph-osd-classic-20.3.0-5343.gbcdd429b 874 kB/s | 15 MB 00:17 2026-02-20T23:52:58.408 INFO:teuthology.orchestra.run.trial138.stdout:(42/42): ceph-osd-crimson-20.3.0-5343.gbcdd429b 681 kB/s | 88 MB 02:12 2026-02-20T23:52:58.410 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:52:58.410 INFO:teuthology.orchestra.run.trial138.stdout:Total 860 kB/s | 115 MB 02:17 2026-02-20T23:52:58.528 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:52:58.546 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:52:58.547 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:52:58.942 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:52:58.942 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:52:59.379 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:52:59.382 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/42 2026-02-20T23:52:59.386 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/42 2026-02-20T23:52:59.391 INFO:teuthology.orchestra.run.trial138.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/42 2026-02-20T23:52:59.393 INFO:teuthology.orchestra.run.trial138.stdout: Installing : unzip-6.0-59.el9.x86_64 4/42 2026-02-20T23:52:59.398 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 5/42 2026-02-20T23:52:59.422 INFO:teuthology.orchestra.run.trial138.stdout: Installing : zip-3.0-35.el9.x86_64 6/42 2026-02-20T23:52:59.425 INFO:teuthology.orchestra.run.trial138.stdout: Installing : luarocks-3.9.2-5.el9.noarch 7/42 2026-02-20T23:52:59.430 INFO:teuthology.orchestra.run.trial138.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 8/42 2026-02-20T23:52:59.435 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 9/42 2026-02-20T23:52:59.441 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 10/42 2026-02-20T23:52:59.445 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 11/42 2026-02-20T23:52:59.450 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 12/42 2026-02-20T23:52:59.456 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 13/42 2026-02-20T23:52:59.459 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 14/42 2026-02-20T23:52:59.465 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 15/42 2026-02-20T23:52:59.468 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 16/42 2026-02-20T23:52:59.475 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 17/42 2026-02-20T23:52:59.478 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 18/42 2026-02-20T23:52:59.501 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 19/42 2026-02-20T23:52:59.513 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 20/42 2026-02-20T23:52:59.518 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 21/42 2026-02-20T23:52:59.528 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 22/42 2026-02-20T23:52:59.534 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-20T23:52:59.548 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/42 2026-02-20T23:52:59.563 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/42 2026-02-20T23:52:59.637 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/42 2026-02-20T23:52:59.646 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 27/42 2026-02-20T23:52:59.669 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/42 2026-02-20T23:52:59.698 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 29/42 2026-02-20T23:52:59.703 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/42 2026-02-20T23:52:59.755 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/42 2026-02-20T23:52:59.795 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5343.gbcdd429b.el9. 32/42 2026-02-20T23:52:59.796 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 33/42 2026-02-20T23:52:59.819 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 33/42 2026-02-20T23:52:59.819 INFO:teuthology.orchestra.run.trial138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:52:59.819 INFO:teuthology.orchestra.run.trial138.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-20T23:52:59.819 INFO:teuthology.orchestra.run.trial138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T23:52:59.819 INFO:teuthology.orchestra.run.trial138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T23:52:59.819 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:52:59.823 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 34/42 2026-02-20T23:52:59.839 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-20T23:52:59.839 INFO:teuthology.orchestra.run.trial138.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-20T23:52:59.839 INFO:teuthology.orchestra.run.trial138.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-20T23:52:59.839 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:52:59.848 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-20T23:52:59.870 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-20T23:52:59.870 INFO:teuthology.orchestra.run.trial138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-20T23:52:59.870 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:00.469 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 36/42 2026-02-20T23:53:05.859 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_6 37/42 2026-02-20T23:53:05.869 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86_6 38/42 2026-02-20T23:53:05.870 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 39/42 2026-02-20T23:53:05.894 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 39/42 2026-02-20T23:53:05.894 INFO:teuthology.orchestra.run.trial138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:53:05.894 INFO:teuthology.orchestra.run.trial138.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-20T23:53:05.894 INFO:teuthology.orchestra.run.trial138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T23:53:05.895 INFO:teuthology.orchestra.run.trial138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T23:53:05.895 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:06.099 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-mon-2:20.3.0-5343.gbcdd429b.el9.x86_64 40/42 2026-02-20T23:53:06.119 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-mon-2:20.3.0-5343.gbcdd429b.el9.x86_64 40/42 2026-02-20T23:53:06.119 INFO:teuthology.orchestra.run.trial138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:53:06.119 INFO:teuthology.orchestra.run.trial138.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-20T23:53:06.119 INFO:teuthology.orchestra.run.trial138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T23:53:06.119 INFO:teuthology.orchestra.run.trial138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T23:53:06.119 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:06.202 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-mds-2:20.3.0-5343.gbcdd429b.el9.x86_64 41/42 2026-02-20T23:53:06.222 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-mds-2:20.3.0-5343.gbcdd429b.el9.x86_64 41/42 2026-02-20T23:53:06.222 INFO:teuthology.orchestra.run.trial138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:53:06.222 INFO:teuthology.orchestra.run.trial138.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-20T23:53:06.222 INFO:teuthology.orchestra.run.trial138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T23:53:06.222 INFO:teuthology.orchestra.run.trial138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T23:53:06.222 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:06.223 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 42/42 2026-02-20T23:53:06.229 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_6 42/42 2026-02-20T23:53:06.251 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86_6 42/42 2026-02-20T23:53:07.162 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 42/42 2026-02-20T23:53:07.163 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/42 2026-02-20T23:53:07.163 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-mds-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/42 2026-02-20T23:53:07.163 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 3/42 2026-02-20T23:53:07.163 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-mon-2:20.3.0-5343.gbcdd429b.el9.x86_64 4/42 2026-02-20T23:53:07.163 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 5/42 2026-02-20T23:53:07.163 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_6 6/42 2026-02-20T23:53:07.163 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86_6 7/42 2026-02-20T23:53:07.163 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5343.gbcdd429b.el9. 8/42 2026-02-20T23:53:07.163 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 9/42 2026-02-20T23:53:07.163 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 10/42 2026-02-20T23:53:07.164 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/42 2026-02-20T23:53:07.164 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 12/42 2026-02-20T23:53:07.164 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/42 2026-02-20T23:53:07.164 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/42 2026-02-20T23:53:07.164 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 15/42 2026-02-20T23:53:07.164 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 16/42 2026-02-20T23:53:07.164 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 17/42 2026-02-20T23:53:07.164 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : unzip-6.0-59.el9.x86_64 18/42 2026-02-20T23:53:07.164 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : zip-3.0-35.el9.x86_64 19/42 2026-02-20T23:53:07.164 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 20/42 2026-02-20T23:53:07.164 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : lua-5.4.4-4.el9.x86_64 21/42 2026-02-20T23:53:07.164 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 22/42 2026-02-20T23:53:07.165 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-20T23:53:07.165 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 24/42 2026-02-20T23:53:07.165 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 25/42 2026-02-20T23:53:07.165 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 26/42 2026-02-20T23:53:07.165 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 27/42 2026-02-20T23:53:07.165 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 28/42 2026-02-20T23:53:07.165 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 29/42 2026-02-20T23:53:07.165 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 30/42 2026-02-20T23:53:07.165 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 31/42 2026-02-20T23:53:07.165 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 32/42 2026-02-20T23:53:07.165 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 33/42 2026-02-20T23:53:07.165 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 34/42 2026-02-20T23:53:07.165 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 35/42 2026-02-20T23:53:07.165 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 36/42 2026-02-20T23:53:07.166 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 37/42 2026-02-20T23:53:07.166 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 38/42 2026-02-20T23:53:07.166 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 39/42 2026-02-20T23:53:07.166 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 40/42 2026-02-20T23:53:07.166 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 41/42 2026-02-20T23:53:07.220 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 42/42 2026-02-20T23:53:07.221 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:07.221 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-20T23:53:07.221 INFO:teuthology.orchestra.run.trial138.stdout: ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:53:07.221 INFO:teuthology.orchestra.run.trial138.stdout: ceph-mds-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:53:07.221 INFO:teuthology.orchestra.run.trial138.stdout: ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:53:07.221 INFO:teuthology.orchestra.run.trial138.stdout: ceph-mgr-modules-core-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:53:07.221 INFO:teuthology.orchestra.run.trial138.stdout: ceph-mon-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:53:07.221 INFO:teuthology.orchestra.run.trial138.stdout: ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:53:07.222 INFO:teuthology.orchestra.run.trial138.stdout: ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:53:07.222 INFO:teuthology.orchestra.run.trial138.stdout: ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:53:07.222 INFO:teuthology.orchestra.run.trial138.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-20T23:53:07.222 INFO:teuthology.orchestra.run.trial138.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-20T23:53:07.222 INFO:teuthology.orchestra.run.trial138.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T23:53:07.222 INFO:teuthology.orchestra.run.trial138.stdout: lua-5.4.4-4.el9.x86_64 2026-02-20T23:53:07.222 INFO:teuthology.orchestra.run.trial138.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-20T23:53:07.222 INFO:teuthology.orchestra.run.trial138.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-20T23:53:07.222 INFO:teuthology.orchestra.run.trial138.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-20T23:53:07.222 INFO:teuthology.orchestra.run.trial138.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-20T23:53:07.222 INFO:teuthology.orchestra.run.trial138.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-20T23:53:07.222 INFO:teuthology.orchestra.run.trial138.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-20T23:53:07.222 INFO:teuthology.orchestra.run.trial138.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-20T23:53:07.223 INFO:teuthology.orchestra.run.trial138.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-20T23:53:07.223 INFO:teuthology.orchestra.run.trial138.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-20T23:53:07.223 INFO:teuthology.orchestra.run.trial138.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-20T23:53:07.223 INFO:teuthology.orchestra.run.trial138.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-20T23:53:07.223 INFO:teuthology.orchestra.run.trial138.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-20T23:53:07.223 INFO:teuthology.orchestra.run.trial138.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-20T23:53:07.223 INFO:teuthology.orchestra.run.trial138.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-20T23:53:07.223 INFO:teuthology.orchestra.run.trial138.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-20T23:53:07.223 INFO:teuthology.orchestra.run.trial138.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T23:53:07.223 INFO:teuthology.orchestra.run.trial138.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-20T23:53:07.223 INFO:teuthology.orchestra.run.trial138.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-20T23:53:07.223 INFO:teuthology.orchestra.run.trial138.stdout: python3-ply-3.11-14.el9.noarch 2026-02-20T23:53:07.224 INFO:teuthology.orchestra.run.trial138.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-20T23:53:07.224 INFO:teuthology.orchestra.run.trial138.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-20T23:53:07.224 INFO:teuthology.orchestra.run.trial138.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-20T23:53:07.224 INFO:teuthology.orchestra.run.trial138.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-20T23:53:07.224 INFO:teuthology.orchestra.run.trial138.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-20T23:53:07.224 INFO:teuthology.orchestra.run.trial138.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-20T23:53:07.224 INFO:teuthology.orchestra.run.trial138.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-20T23:53:07.224 INFO:teuthology.orchestra.run.trial138.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-20T23:53:07.224 INFO:teuthology.orchestra.run.trial138.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-20T23:53:07.224 INFO:teuthology.orchestra.run.trial138.stdout: unzip-6.0-59.el9.x86_64 2026-02-20T23:53:07.224 INFO:teuthology.orchestra.run.trial138.stdout: zip-3.0-35.el9.x86_64 2026-02-20T23:53:07.224 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:07.224 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:53:07.263 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-base 2026-02-20T23:53:07.575 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:06:13 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:53:07.638 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:53:07.656 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:53:07.656 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-20T23:53:07.656 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:53:07.677 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install cephadm 2026-02-20T23:53:07.980 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:06:13 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:53:08.062 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:53:08.062 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:53:08.062 INFO:teuthology.orchestra.run.trial138.stdout: Package Arch Version Repository Size 2026-02-20T23:53:08.063 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:53:08.063 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:53:08.063 INFO:teuthology.orchestra.run.trial138.stdout: cephadm noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 1.0 M 2026-02-20T23:53:08.063 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:08.063 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:53:08.063 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:53:08.063 INFO:teuthology.orchestra.run.trial138.stdout:Install 1 Package 2026-02-20T23:53:08.063 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:08.063 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 1.0 M 2026-02-20T23:53:08.063 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 1.0 M 2026-02-20T23:53:08.063 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:53:11.763 INFO:teuthology.orchestra.run.trial138.stdout:cephadm-20.3.0-5343.gbcdd429b.el9.noarch.rpm 281 kB/s | 1.0 MB 00:03 2026-02-20T23:53:11.763 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:53:11.763 INFO:teuthology.orchestra.run.trial138.stdout:Total 281 kB/s | 1.0 MB 00:03 2026-02-20T23:53:11.763 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:53:11.766 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:53:11.766 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:53:11.770 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:53:11.770 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:53:11.789 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:53:11.851 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 1/1 2026-02-20T23:53:11.865 INFO:teuthology.orchestra.run.trial138.stdout: Installing : cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 1/1 2026-02-20T23:53:11.933 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 1/1 2026-02-20T23:53:11.988 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 1/1 2026-02-20T23:53:11.989 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:11.989 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-20T23:53:11.989 INFO:teuthology.orchestra.run.trial138.stdout: cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:53:11.989 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:11.989 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:53:12.016 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-immutable-object-cache 2026-02-20T23:53:12.325 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:06:18 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:53:12.405 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:53:12.405 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:53:12.405 INFO:teuthology.orchestra.run.trial138.stdout: Package Arch Version Repo Size 2026-02-20T23:53:12.405 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:53:12.405 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:53:12.405 INFO:teuthology.orchestra.run.trial138.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 149 k 2026-02-20T23:53:12.406 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:12.406 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:53:12.406 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:53:12.406 INFO:teuthology.orchestra.run.trial138.stdout:Install 1 Package 2026-02-20T23:53:12.406 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:12.406 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 149 k 2026-02-20T23:53:12.406 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 455 k 2026-02-20T23:53:12.406 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:53:13.315 INFO:teuthology.orchestra.run.trial138.stdout:ceph-immutable-object-cache-20.3.0-5343.gbcdd42 165 kB/s | 149 kB 00:00 2026-02-20T23:53:13.315 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:53:13.316 INFO:teuthology.orchestra.run.trial138.stdout:Total 164 kB/s | 149 kB 00:00 2026-02-20T23:53:13.316 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:53:13.320 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:53:13.320 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:53:13.346 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:53:13.346 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:53:13.395 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:53:13.397 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b. 1/1 2026-02-20T23:53:13.419 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b. 1/1 2026-02-20T23:53:13.419 INFO:teuthology.orchestra.run.trial138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:53:13.420 INFO:teuthology.orchestra.run.trial138.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-20T23:53:13.420 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:13.711 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b. 1/1 2026-02-20T23:53:13.711 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:13.711 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-20T23:53:13.711 INFO:teuthology.orchestra.run.trial138.stdout: ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:53:13.711 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:13.712 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:53:13.738 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-mgr 2026-02-20T23:53:14.053 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:06:20 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:53:14.116 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:53:14.134 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:53:14.134 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-20T23:53:14.135 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:53:14.157 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-mgr-dashboard 2026-02-20T23:53:14.459 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:06:20 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:53:14.541 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout: Package Arch Version Repository Size 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 51 M 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout:Installing dependencies: 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 57 k 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 17 k 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout:Install 13 Packages 2026-02-20T23:53:14.542 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:53:14.543 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 55 M 2026-02-20T23:53:14.543 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 262 M 2026-02-20T23:53:14.543 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:53:15.558 INFO:teuthology.orchestra.run.trial138.stdout:(1/13): ceph-prometheus-alerts-20.3.0-5343.gbcd 35 kB/s | 17 kB 00:00 2026-02-20T23:53:15.602 INFO:teuthology.orchestra.run.trial138.stdout:(2/13): python3-jmespath-1.0.1-1.el9.noarch.rpm 1.1 MB/s | 48 kB 00:00 2026-02-20T23:53:15.631 INFO:teuthology.orchestra.run.trial138.stdout:(3/13): ceph-grafana-dashboards-20.3.0-5343.gbc 102 kB/s | 57 kB 00:00 2026-02-20T23:53:15.637 INFO:teuthology.orchestra.run.trial138.stdout:(4/13): python3-protobuf-3.14.0-17.el9.noarch.r 7.5 MB/s | 267 kB 00:00 2026-02-20T23:53:15.757 INFO:teuthology.orchestra.run.trial138.stdout:(5/13): protobuf-compiler-3.14.0-17.el9.x86_64. 6.7 MB/s | 862 kB 00:00 2026-02-20T23:53:15.877 INFO:teuthology.orchestra.run.trial138.stdout:(6/13): grpc-data-1.46.7-10.el9.noarch.rpm 162 kB/s | 19 kB 00:00 2026-02-20T23:53:15.931 INFO:teuthology.orchestra.run.trial138.stdout:(7/13): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.8 MB/s | 551 kB 00:00 2026-02-20T23:53:15.974 INFO:teuthology.orchestra.run.trial138.stdout:(8/13): python3-grpcio-tools-1.46.7-10.el9.x86_ 3.2 MB/s | 144 kB 00:00 2026-02-20T23:53:16.016 INFO:teuthology.orchestra.run.trial138.stdout:(9/13): python3-repoze-lru-0.7-16.el9.noarch.rp 739 kB/s | 31 kB 00:00 2026-02-20T23:53:16.059 INFO:teuthology.orchestra.run.trial138.stdout:(10/13): python3-routes-2.5.1-5.el9.noarch.rpm 4.3 MB/s | 188 kB 00:00 2026-02-20T23:53:16.101 INFO:teuthology.orchestra.run.trial138.stdout:(11/13): python3-xmltodict-0.12.0-15.el9.noarch 535 kB/s | 22 kB 00:00 2026-02-20T23:53:16.121 INFO:teuthology.orchestra.run.trial138.stdout:(12/13): python3-grpcio-1.46.7-10.el9.x86_64.rp 8.4 MB/s | 2.0 MB 00:00 2026-02-20T23:54:26.752 INFO:teuthology.orchestra.run.trial138.stdout:(13/13): ceph-mgr-dashboard-20.3.0-5343.gbcdd42 724 kB/s | 51 MB 01:11 2026-02-20T23:54:26.755 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:54:26.755 INFO:teuthology.orchestra.run.trial138.stdout:Total 778 kB/s | 55 MB 01:12 2026-02-20T23:54:26.779 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:54:26.790 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:54:26.790 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:54:27.003 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:54:27.003 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:54:27.231 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:54:27.237 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 1/13 2026-02-20T23:54:27.248 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 2/13 2026-02-20T23:54:27.250 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 3/13 2026-02-20T23:54:27.277 INFO:teuthology.orchestra.run.trial138.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 4/13 2026-02-20T23:54:27.320 INFO:teuthology.orchestra.run.trial138.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 5/13 2026-02-20T23:54:27.330 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 6/13 2026-02-20T23:54:27.358 INFO:teuthology.orchestra.run.trial138.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 7/13 2026-02-20T23:54:27.368 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 8/13 2026-02-20T23:54:27.375 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/13 2026-02-20T23:54:27.378 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 10/13 2026-02-20T23:54:27.387 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5343.gbcdd429b.el9 11/13 2026-02-20T23:54:30.275 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5343.gbcdd429b.el 12/13 2026-02-20T23:54:30.281 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noa 13/13 2026-02-20T23:54:30.564 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noa 13/13 2026-02-20T23:54:30.565 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5343.gbcdd429b.el 1/13 2026-02-20T23:54:30.565 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noa 2/13 2026-02-20T23:54:30.565 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5343.gbcdd429b.el9 3/13 2026-02-20T23:54:30.565 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 4/13 2026-02-20T23:54:30.565 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 5/13 2026-02-20T23:54:30.565 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 6/13 2026-02-20T23:54:30.565 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 7/13 2026-02-20T23:54:30.565 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/13 2026-02-20T23:54:30.565 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/13 2026-02-20T23:54:30.566 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/13 2026-02-20T23:54:30.566 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/13 2026-02-20T23:54:30.566 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/13 2026-02-20T23:54:30.620 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 13/13 2026-02-20T23:54:30.621 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:30.621 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-20T23:54:30.621 INFO:teuthology.orchestra.run.trial138.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-20T23:54:30.621 INFO:teuthology.orchestra.run.trial138.stdout: ceph-grafana-dashboards-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:54:30.621 INFO:teuthology.orchestra.run.trial138.stdout: ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:54:30.621 INFO:teuthology.orchestra.run.trial138.stdout: ceph-prometheus-alerts-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:54:30.621 INFO:teuthology.orchestra.run.trial138.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-20T23:54:30.621 INFO:teuthology.orchestra.run.trial138.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-20T23:54:30.621 INFO:teuthology.orchestra.run.trial138.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-20T23:54:30.621 INFO:teuthology.orchestra.run.trial138.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-20T23:54:30.622 INFO:teuthology.orchestra.run.trial138.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-20T23:54:30.622 INFO:teuthology.orchestra.run.trial138.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-20T23:54:30.622 INFO:teuthology.orchestra.run.trial138.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-20T23:54:30.622 INFO:teuthology.orchestra.run.trial138.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-20T23:54:30.622 INFO:teuthology.orchestra.run.trial138.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-20T23:54:30.622 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:30.622 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:54:30.655 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-20T23:54:30.969 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:07:36 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:54:31.054 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:54:31.055 INFO:teuthology.orchestra.run.trial138.stdout:======================================================================================= 2026-02-20T23:54:31.055 INFO:teuthology.orchestra.run.trial138.stdout: Package Arch Version Repository Size 2026-02-20T23:54:31.056 INFO:teuthology.orchestra.run.trial138.stdout:======================================================================================= 2026-02-20T23:54:31.056 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:54:31.056 INFO:teuthology.orchestra.run.trial138.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 7.4 M 2026-02-20T23:54:31.056 INFO:teuthology.orchestra.run.trial138.stdout:Installing dependencies: 2026-02-20T23:54:31.056 INFO:teuthology.orchestra.run.trial138.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-20T23:54:31.056 INFO:teuthology.orchestra.run.trial138.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-20T23:54:31.056 INFO:teuthology.orchestra.run.trial138.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-20T23:54:31.056 INFO:teuthology.orchestra.run.trial138.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-20T23:54:31.056 INFO:teuthology.orchestra.run.trial138.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-20T23:54:31.056 INFO:teuthology.orchestra.run.trial138.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-20T23:54:31.057 INFO:teuthology.orchestra.run.trial138.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-20T23:54:31.057 INFO:teuthology.orchestra.run.trial138.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-20T23:54:31.057 INFO:teuthology.orchestra.run.trial138.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-20T23:54:31.057 INFO:teuthology.orchestra.run.trial138.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-20T23:54:31.057 INFO:teuthology.orchestra.run.trial138.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-20T23:54:31.057 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:31.057 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:54:31.057 INFO:teuthology.orchestra.run.trial138.stdout:======================================================================================= 2026-02-20T23:54:31.057 INFO:teuthology.orchestra.run.trial138.stdout:Install 12 Packages 2026-02-20T23:54:31.057 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:31.057 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 43 M 2026-02-20T23:54:31.058 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 234 M 2026-02-20T23:54:31.058 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:54:31.320 INFO:teuthology.orchestra.run.trial138.stdout:(1/12): libquadmath-11.5.0-14.el9.x86_64.rpm 1.2 MB/s | 184 kB 00:00 2026-02-20T23:54:31.370 INFO:teuthology.orchestra.run.trial138.stdout:(2/12): libgfortran-11.5.0-14.el9.x86_64.rpm 4.0 MB/s | 794 kB 00:00 2026-02-20T23:54:31.893 INFO:teuthology.orchestra.run.trial138.stdout:(3/12): flexiblas-3.0.4-9.el9.x86_64.rpm 52 kB/s | 30 kB 00:00 2026-02-20T23:54:32.051 INFO:teuthology.orchestra.run.trial138.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-9.el9.x 94 kB/s | 15 kB 00:00 2026-02-20T23:54:32.286 INFO:teuthology.orchestra.run.trial138.stdout:(5/12): openblas-0.3.29-1.el9.x86_64.rpm 179 kB/s | 42 kB 00:00 2026-02-20T23:54:35.342 INFO:teuthology.orchestra.run.trial138.stdout:(6/12): flexiblas-netlib-3.0.4-9.el9.x86_64.rpm 771 kB/s | 3.0 MB 00:03 2026-02-20T23:54:35.656 INFO:teuthology.orchestra.run.trial138.stdout:(7/12): python3-devel-3.9.25-3.el9.x86_64.rpm 778 kB/s | 244 kB 00:00 2026-02-20T23:54:35.967 INFO:teuthology.orchestra.run.trial138.stdout:(8/12): openblas-openmp-0.3.29-1.el9.x86_64.rpm 1.4 MB/s | 5.3 MB 00:03 2026-02-20T23:54:36.128 INFO:teuthology.orchestra.run.trial138.stdout:(9/12): python3-numpy-f2py-1.23.5-2.el9.x86_64. 2.7 MB/s | 442 kB 00:00 2026-02-20T23:54:36.144 INFO:teuthology.orchestra.run.trial138.stdout:(10/12): python3-numpy-1.23.5-2.el9.x86_64.rpm 13 MB/s | 6.1 MB 00:00 2026-02-20T23:54:36.943 INFO:teuthology.orchestra.run.trial138.stdout:(11/12): python3-scipy-1.9.3-2.el9.x86_64.rpm 24 MB/s | 19 MB 00:00 2026-02-20T23:54:41.984 INFO:teuthology.orchestra.run.trial138.stdout:(12/12): ceph-mgr-diskprediction-local-20.3.0-5 700 kB/s | 7.4 MB 00:10 2026-02-20T23:54:41.985 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:54:41.985 INFO:teuthology.orchestra.run.trial138.stdout:Total 3.9 MB/s | 43 MB 00:10 2026-02-20T23:54:42.144 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:54:42.157 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:54:42.157 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:54:42.360 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:54:42.360 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:54:42.579 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:54:42.582 INFO:teuthology.orchestra.run.trial138.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 1/12 2026-02-20T23:54:42.588 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 2/12 2026-02-20T23:54:42.622 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 3/12 2026-02-20T23:54:42.626 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 4/12 2026-02-20T23:54:42.686 INFO:teuthology.orchestra.run.trial138.stdout: Installing : openblas-0.3.29-1.el9.x86_64 5/12 2026-02-20T23:54:42.688 INFO:teuthology.orchestra.run.trial138.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 6/12 2026-02-20T23:54:42.709 INFO:teuthology.orchestra.run.trial138.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 7/12 2026-02-20T23:54:43.028 INFO:teuthology.orchestra.run.trial138.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 8/12 2026-02-20T23:54:43.100 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 9/12 2026-02-20T23:54:43.760 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 10/12 2026-02-20T23:54:44.214 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 11/12 2026-02-20T23:54:44.215 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd4 12/12 2026-02-20T23:54:44.504 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd4 12/12 2026-02-20T23:54:44.504 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd4 1/12 2026-02-20T23:54:44.504 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 2/12 2026-02-20T23:54:44.505 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 3/12 2026-02-20T23:54:44.505 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 4/12 2026-02-20T23:54:44.505 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 5/12 2026-02-20T23:54:44.505 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 6/12 2026-02-20T23:54:44.505 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 7/12 2026-02-20T23:54:44.505 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 8/12 2026-02-20T23:54:44.505 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 9/12 2026-02-20T23:54:44.505 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 10/12 2026-02-20T23:54:44.505 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 11/12 2026-02-20T23:54:44.562 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 12/12 2026-02-20T23:54:44.563 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:44.563 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-20T23:54:44.563 INFO:teuthology.orchestra.run.trial138.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:54:44.563 INFO:teuthology.orchestra.run.trial138.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-20T23:54:44.563 INFO:teuthology.orchestra.run.trial138.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-20T23:54:44.563 INFO:teuthology.orchestra.run.trial138.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-20T23:54:44.563 INFO:teuthology.orchestra.run.trial138.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-20T23:54:44.563 INFO:teuthology.orchestra.run.trial138.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-20T23:54:44.563 INFO:teuthology.orchestra.run.trial138.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-20T23:54:44.563 INFO:teuthology.orchestra.run.trial138.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-20T23:54:44.564 INFO:teuthology.orchestra.run.trial138.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-20T23:54:44.564 INFO:teuthology.orchestra.run.trial138.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-20T23:54:44.564 INFO:teuthology.orchestra.run.trial138.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-20T23:54:44.564 INFO:teuthology.orchestra.run.trial138.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-20T23:54:44.564 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:44.564 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:54:44.603 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-mgr-rook 2026-02-20T23:54:44.919 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:07:50 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:54:45.002 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:54:45.003 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:45.003 INFO:teuthology.orchestra.run.trial138.stdout: Package Arch Version Repository Size 2026-02-20T23:54:45.003 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:45.003 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:54:45.003 INFO:teuthology.orchestra.run.trial138.stdout: ceph-mgr-rook noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 50 k 2026-02-20T23:54:45.003 INFO:teuthology.orchestra.run.trial138.stdout:Installing dependencies: 2026-02-20T23:54:45.004 INFO:teuthology.orchestra.run.trial138.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-20T23:54:45.004 INFO:teuthology.orchestra.run.trial138.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-20T23:54:45.004 INFO:teuthology.orchestra.run.trial138.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-20T23:54:45.004 INFO:teuthology.orchestra.run.trial138.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-20T23:54:45.004 INFO:teuthology.orchestra.run.trial138.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-20T23:54:45.004 INFO:teuthology.orchestra.run.trial138.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-20T23:54:45.004 INFO:teuthology.orchestra.run.trial138.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-20T23:54:45.004 INFO:teuthology.orchestra.run.trial138.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-20T23:54:45.004 INFO:teuthology.orchestra.run.trial138.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-20T23:54:45.004 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:45.004 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:54:45.005 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:45.005 INFO:teuthology.orchestra.run.trial138.stdout:Install 10 Packages 2026-02-20T23:54:45.005 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:45.005 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 2.0 M 2026-02-20T23:54:45.006 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 25 M 2026-02-20T23:54:45.006 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:54:45.766 INFO:teuthology.orchestra.run.trial138.stdout:(1/10): ceph-mgr-rook-20.3.0-5343.gbcdd429b.el9 119 kB/s | 50 kB 00:00 2026-02-20T23:54:45.790 INFO:teuthology.orchestra.run.trial138.stdout:(2/10): python3-pyasn1-0.4.8-6.el9.noarch.rpm 354 kB/s | 159 kB 00:00 2026-02-20T23:54:45.851 INFO:teuthology.orchestra.run.trial138.stdout:(3/10): python3-pyasn1-modules-0.4.8-6.el9.noar 547 kB/s | 279 kB 00:00 2026-02-20T23:54:46.094 INFO:teuthology.orchestra.run.trial138.stdout:(4/10): python3-requests-oauthlib-1.3.0-12.el9. 163 kB/s | 54 kB 00:00 2026-02-20T23:54:46.102 INFO:teuthology.orchestra.run.trial138.stdout:(5/10): python3-cachetools-4.2.4-1.el9.noarch.r 103 kB/s | 32 kB 00:00 2026-02-20T23:54:46.415 INFO:teuthology.orchestra.run.trial138.stdout:(6/10): python3-certifi-2023.05.07-4.el9.noarch 25 kB/s | 14 kB 00:00 2026-02-20T23:54:46.497 INFO:teuthology.orchestra.run.trial138.stdout:(7/10): python3-kubernetes-26.1.0-3.el9.noarch. 2.6 MB/s | 1.0 MB 00:00 2026-02-20T23:54:46.568 INFO:teuthology.orchestra.run.trial138.stdout:(8/10): python3-websocket-client-1.2.3-2.el9.no 1.2 MB/s | 90 kB 00:00 2026-02-20T23:54:46.582 INFO:teuthology.orchestra.run.trial138.stdout:(9/10): python3-google-auth-2.45.0-1.el9.noarch 521 kB/s | 254 kB 00:00 2026-02-20T23:54:46.582 INFO:teuthology.orchestra.run.trial138.stdout:(10/10): python3-rsa-4.9-2.el9.noarch.rpm 352 kB/s | 59 kB 00:00 2026-02-20T23:54:46.584 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:54:46.584 INFO:teuthology.orchestra.run.trial138.stdout:Total 1.3 MB/s | 2.0 MB 00:01 2026-02-20T23:54:46.610 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:54:46.617 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:54:46.617 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:54:46.656 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:54:46.656 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:54:46.730 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:54:46.781 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 1/10 2026-02-20T23:54:46.794 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 2/10 2026-02-20T23:54:46.808 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/10 2026-02-20T23:54:46.813 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/10 2026-02-20T23:54:46.821 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/10 2026-02-20T23:54:46.860 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/10 2026-02-20T23:54:46.871 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-20T23:54:47.189 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 8/10 2026-02-20T23:54:47.204 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 9/10 2026-02-20T23:54:47.205 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch 10/10 2026-02-20T23:54:47.310 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch 10/10 2026-02-20T23:54:47.310 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch 1/10 2026-02-20T23:54:47.310 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 2/10 2026-02-20T23:54:47.310 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 3/10 2026-02-20T23:54:47.310 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 4/10 2026-02-20T23:54:47.310 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 5/10 2026-02-20T23:54:47.311 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 6/10 2026-02-20T23:54:47.311 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-20T23:54:47.311 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 8/10 2026-02-20T23:54:47.311 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 9/10 2026-02-20T23:54:47.369 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 10/10 2026-02-20T23:54:47.369 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:47.369 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-20T23:54:47.369 INFO:teuthology.orchestra.run.trial138.stdout: ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:54:47.369 INFO:teuthology.orchestra.run.trial138.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-20T23:54:47.369 INFO:teuthology.orchestra.run.trial138.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-20T23:54:47.369 INFO:teuthology.orchestra.run.trial138.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-20T23:54:47.369 INFO:teuthology.orchestra.run.trial138.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-20T23:54:47.369 INFO:teuthology.orchestra.run.trial138.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-20T23:54:47.369 INFO:teuthology.orchestra.run.trial138.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-20T23:54:47.369 INFO:teuthology.orchestra.run.trial138.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-20T23:54:47.369 INFO:teuthology.orchestra.run.trial138.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-20T23:54:47.369 INFO:teuthology.orchestra.run.trial138.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-20T23:54:47.369 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:47.369 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:54:47.398 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-mgr-cephadm 2026-02-20T23:54:47.705 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:07:53 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:54:47.787 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:54:47.788 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:47.788 INFO:teuthology.orchestra.run.trial138.stdout: Package Arch Version Repository Size 2026-02-20T23:54:47.789 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:47.789 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:54:47.789 INFO:teuthology.orchestra.run.trial138.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 184 k 2026-02-20T23:54:47.789 INFO:teuthology.orchestra.run.trial138.stdout:Installing dependencies: 2026-02-20T23:54:47.789 INFO:teuthology.orchestra.run.trial138.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-20T23:54:47.789 INFO:teuthology.orchestra.run.trial138.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-20T23:54:47.789 INFO:teuthology.orchestra.run.trial138.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-20T23:54:47.789 INFO:teuthology.orchestra.run.trial138.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-20T23:54:47.789 INFO:teuthology.orchestra.run.trial138.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-20T23:54:47.789 INFO:teuthology.orchestra.run.trial138.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-20T23:54:47.789 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:47.790 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:54:47.790 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:47.790 INFO:teuthology.orchestra.run.trial138.stdout:Install 7 Packages 2026-02-20T23:54:47.790 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:47.790 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 7.1 M 2026-02-20T23:54:47.790 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 34 M 2026-02-20T23:54:47.790 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:54:48.692 INFO:teuthology.orchestra.run.trial138.stdout:(1/7): python3-jinja2-2.11.3-8.el9.noarch.rpm 368 kB/s | 249 kB 00:00 2026-02-20T23:54:48.798 INFO:teuthology.orchestra.run.trial138.stdout:(2/7): python3-markupsafe-1.1.1-12.el9.x86_64.r 327 kB/s | 35 kB 00:00 2026-02-20T23:54:48.856 INFO:teuthology.orchestra.run.trial138.stdout:(3/7): ceph-mgr-cephadm-20.3.0-5343.gbcdd429b.e 219 kB/s | 184 kB 00:00 2026-02-20T23:54:49.025 INFO:teuthology.orchestra.run.trial138.stdout:(4/7): python3-natsort-7.1.1-5.el9.noarch.rpm 343 kB/s | 58 kB 00:00 2026-02-20T23:54:49.076 INFO:teuthology.orchestra.run.trial138.stdout:(5/7): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.9 MB/s | 548 kB 00:00 2026-02-20T23:54:49.103 INFO:teuthology.orchestra.run.trial138.stdout:(6/7): python3-typing-extensions-4.15.0-1.el9.n 1.1 MB/s | 86 kB 00:00 2026-02-20T23:54:49.111 INFO:teuthology.orchestra.run.trial138.stdout:(7/7): python3-babel-2.9.1-2.el9.noarch.rpm 5.5 MB/s | 6.0 MB 00:01 2026-02-20T23:54:49.113 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:54:49.114 INFO:teuthology.orchestra.run.trial138.stdout:Total 5.4 MB/s | 7.1 MB 00:01 2026-02-20T23:54:49.150 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:54:49.156 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:54:49.157 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:54:49.228 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:54:49.228 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:54:49.316 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:54:49.365 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 1/7 2026-02-20T23:54:49.374 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/7 2026-02-20T23:54:49.380 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/7 2026-02-20T23:54:49.602 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-20T23:54:49.625 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 5/7 2026-02-20T23:54:49.650 INFO:teuthology.orchestra.run.trial138.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 6/7 2026-02-20T23:54:49.651 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 7/7 2026-02-20T23:54:49.758 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 7/7 2026-02-20T23:54:49.758 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 1/7 2026-02-20T23:54:49.758 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/7 2026-02-20T23:54:49.758 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 3/7 2026-02-20T23:54:49.758 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-20T23:54:49.758 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 5/7 2026-02-20T23:54:49.758 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 6/7 2026-02-20T23:54:49.816 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 7/7 2026-02-20T23:54:49.816 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:49.816 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-20T23:54:49.816 INFO:teuthology.orchestra.run.trial138.stdout: ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch 2026-02-20T23:54:49.816 INFO:teuthology.orchestra.run.trial138.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-20T23:54:49.816 INFO:teuthology.orchestra.run.trial138.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-20T23:54:49.816 INFO:teuthology.orchestra.run.trial138.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-20T23:54:49.817 INFO:teuthology.orchestra.run.trial138.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-20T23:54:49.817 INFO:teuthology.orchestra.run.trial138.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-20T23:54:49.817 INFO:teuthology.orchestra.run.trial138.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-20T23:54:49.817 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:49.817 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:54:49.844 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-osd 2026-02-20T23:54:50.151 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:07:56 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:54:50.215 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:54:50.233 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:54:50.234 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-20T23:54:50.234 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:54:50.255 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-osd-classic 2026-02-20T23:54:50.563 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:07:56 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:54:50.629 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:54:50.647 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:54:50.648 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-20T23:54:50.648 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:54:50.668 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-fuse 2026-02-20T23:54:50.980 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:07:56 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:54:51.066 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:54:51.066 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:51.066 INFO:teuthology.orchestra.run.trial138.stdout: Package Arch Version Repo Size 2026-02-20T23:54:51.066 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:51.066 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:54:51.066 INFO:teuthology.orchestra.run.trial138.stdout: ceph-fuse x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 914 k 2026-02-20T23:54:51.066 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:51.066 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:54:51.067 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:51.067 INFO:teuthology.orchestra.run.trial138.stdout:Install 1 Package 2026-02-20T23:54:51.067 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:51.067 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 914 k 2026-02-20T23:54:51.067 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 2.9 M 2026-02-20T23:54:51.067 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:54:53.038 INFO:teuthology.orchestra.run.trial138.stdout:ceph-fuse-20.3.0-5343.gbcdd429b.el9.x86_64.rpm 464 kB/s | 914 kB 00:01 2026-02-20T23:54:53.038 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:54:53.039 INFO:teuthology.orchestra.run.trial138.stdout:Total 463 kB/s | 914 kB 00:01 2026-02-20T23:54:53.039 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:54:53.043 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:54:53.043 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:54:53.076 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:54:53.076 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:54:53.166 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:54:53.182 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:54:53.397 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:54:53.454 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:54:53.455 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:53.455 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-20T23:54:53.455 INFO:teuthology.orchestra.run.trial138.stdout: ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:54:53.455 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:53.455 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:54:53.482 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-volume 2026-02-20T23:54:53.784 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:07:59 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:54:53.868 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:54:53.868 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:53.868 INFO:teuthology.orchestra.run.trial138.stdout: Package Arch Version Repository Size 2026-02-20T23:54:53.868 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:53.868 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:54:53.869 INFO:teuthology.orchestra.run.trial138.stdout: ceph-volume noarch 2:20.3.0-5343.gbcdd429b.el9 ceph-noarch 301 k 2026-02-20T23:54:53.869 INFO:teuthology.orchestra.run.trial138.stdout:Installing dependencies: 2026-02-20T23:54:53.869 INFO:teuthology.orchestra.run.trial138.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-20T23:54:53.869 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:53.869 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:54:53.869 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:53.869 INFO:teuthology.orchestra.run.trial138.stdout:Install 2 Packages 2026-02-20T23:54:53.869 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:53.869 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 652 k 2026-02-20T23:54:53.869 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 2.2 M 2026-02-20T23:54:53.869 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:54:53.992 INFO:teuthology.orchestra.run.trial138.stdout:(1/2): cryptsetup-2.8.1-3.el9.x86_64.rpm 5.6 MB/s | 351 kB 00:00 2026-02-20T23:54:54.848 INFO:teuthology.orchestra.run.trial138.stdout:(2/2): ceph-volume-20.3.0-5343.gbcdd429b.el9.no 329 kB/s | 301 kB 00:00 2026-02-20T23:54:54.852 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:54:54.852 INFO:teuthology.orchestra.run.trial138.stdout:Total 663 kB/s | 652 kB 00:00 2026-02-20T23:54:54.862 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:54:54.868 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:54:54.868 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:54:54.911 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:54:54.911 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:54:54.991 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:54:55.054 INFO:teuthology.orchestra.run.trial138.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 1/2 2026-02-20T23:54:55.056 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch 2/2 2026-02-20T23:54:55.071 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch 2/2 2026-02-20T23:54:55.071 INFO:teuthology.orchestra.run.trial138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:54:55.072 INFO:teuthology.orchestra.run.trial138.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-20T23:54:55.072 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:55.312 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch 1/2 2026-02-20T23:54:55.369 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 2/2 2026-02-20T23:54:55.369 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:55.370 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-20T23:54:55.370 INFO:teuthology.orchestra.run.trial138.stdout: ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch cryptsetup-2.8.1-3.el9.x86_64 2026-02-20T23:54:55.370 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:55.370 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:54:55.397 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install librados-devel 2026-02-20T23:54:55.710 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:08:01 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:54:55.793 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:54:55.794 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:55.794 INFO:teuthology.orchestra.run.trial138.stdout: Package Arch Version Repo Size 2026-02-20T23:54:55.794 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:55.794 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:54:55.794 INFO:teuthology.orchestra.run.trial138.stdout: librados-devel x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 124 k 2026-02-20T23:54:55.795 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:55.795 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:54:55.795 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:55.795 INFO:teuthology.orchestra.run.trial138.stdout:Install 1 Package 2026-02-20T23:54:55.795 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:55.795 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 124 k 2026-02-20T23:54:55.795 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 434 k 2026-02-20T23:54:55.795 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:54:56.361 INFO:teuthology.orchestra.run.trial138.stdout:librados-devel-20.3.0-5343.gbcdd429b.el9.x86_64 218 kB/s | 124 kB 00:00 2026-02-20T23:54:56.361 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:54:56.361 INFO:teuthology.orchestra.run.trial138.stdout:Total 218 kB/s | 124 kB 00:00 2026-02-20T23:54:56.362 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:54:56.364 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:54:56.365 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:54:56.396 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:54:56.396 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:54:56.449 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:54:56.466 INFO:teuthology.orchestra.run.trial138.stdout: Installing : librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:54:56.551 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:54:56.609 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:54:56.609 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:56.609 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-20T23:54:56.610 INFO:teuthology.orchestra.run.trial138.stdout: librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:54:56.610 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:56.610 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:54:56.638 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install libcephfs2 2026-02-20T23:54:56.946 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:08:02 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:54:57.012 INFO:teuthology.orchestra.run.trial138.stdout:Package libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:54:57.030 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:54:57.031 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-20T23:54:57.031 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:54:57.051 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install libcephfs-devel 2026-02-20T23:54:57.361 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:08:03 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:54:57.444 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:54:57.445 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:57.445 INFO:teuthology.orchestra.run.trial138.stdout: Package Arch Version Repo Size 2026-02-20T23:54:57.445 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:57.445 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:54:57.445 INFO:teuthology.orchestra.run.trial138.stdout: libcephfs-devel x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 37 k 2026-02-20T23:54:57.446 INFO:teuthology.orchestra.run.trial138.stdout:Installing dependencies: 2026-02-20T23:54:57.446 INFO:teuthology.orchestra.run.trial138.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 26 k 2026-02-20T23:54:57.446 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:57.446 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:54:57.446 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:54:57.446 INFO:teuthology.orchestra.run.trial138.stdout:Install 2 Packages 2026-02-20T23:54:57.446 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:57.446 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 63 k 2026-02-20T23:54:57.446 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 227 k 2026-02-20T23:54:57.446 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:54:57.801 INFO:teuthology.orchestra.run.trial138.stdout:(1/2): libcephfs-proxy2-20.3.0-5343.gbcdd429b.e 72 kB/s | 26 kB 00:00 2026-02-20T23:54:57.809 INFO:teuthology.orchestra.run.trial138.stdout:(2/2): libcephfs-devel-20.3.0-5343.gbcdd429b.el 103 kB/s | 37 kB 00:00 2026-02-20T23:54:57.809 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:54:57.810 INFO:teuthology.orchestra.run.trial138.stdout:Total 172 kB/s | 63 kB 00:00 2026-02-20T23:54:57.810 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:54:57.813 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:54:57.813 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:54:57.846 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:54:57.846 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:54:57.894 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:54:57.895 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libcephfs-proxy2-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/2 2026-02-20T23:54:57.910 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/2 2026-02-20T23:54:57.927 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/2 2026-02-20T23:54:57.984 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/2 2026-02-20T23:54:57.985 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/2 2026-02-20T23:54:58.042 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/2 2026-02-20T23:54:58.043 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:58.043 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-20T23:54:58.043 INFO:teuthology.orchestra.run.trial138.stdout: libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:54:58.043 INFO:teuthology.orchestra.run.trial138.stdout: libcephfs-proxy2-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:54:58.043 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:54:58.043 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:54:58.072 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install librados2 2026-02-20T23:54:58.381 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:08:04 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:54:58.445 INFO:teuthology.orchestra.run.trial138.stdout:Package librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:54:58.463 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:54:58.464 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-20T23:54:58.464 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:54:58.485 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install librbd1 2026-02-20T23:54:58.799 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:08:04 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:54:58.865 INFO:teuthology.orchestra.run.trial138.stdout:Package librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:54:58.884 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:54:58.884 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-20T23:54:58.884 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:54:58.907 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install python3-rados 2026-02-20T23:54:59.218 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:08:05 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:54:59.283 INFO:teuthology.orchestra.run.trial138.stdout:Package python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:54:59.301 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:54:59.302 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-20T23:54:59.302 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:54:59.324 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install python3-rgw 2026-02-20T23:54:59.630 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:08:05 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:54:59.697 INFO:teuthology.orchestra.run.trial138.stdout:Package python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:54:59.716 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:54:59.716 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-20T23:54:59.716 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:54:59.738 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install python3-cephfs 2026-02-20T23:55:00.042 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:08:06 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:55:00.107 INFO:teuthology.orchestra.run.trial138.stdout:Package python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:55:00.125 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:55:00.125 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-20T23:55:00.126 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:55:00.147 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install python3-rbd 2026-02-20T23:55:00.456 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:08:06 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:55:00.522 INFO:teuthology.orchestra.run.trial138.stdout:Package python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:55:00.540 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:55:00.540 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-20T23:55:00.541 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:55:00.562 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install rbd-fuse 2026-02-20T23:55:00.867 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:08:06 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:55:00.950 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:55:00.951 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:55:00.951 INFO:teuthology.orchestra.run.trial138.stdout: Package Architecture Version Repository Size 2026-02-20T23:55:00.952 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:55:00.952 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:55:00.952 INFO:teuthology.orchestra.run.trial138.stdout: rbd-fuse x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 86 k 2026-02-20T23:55:00.952 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:55:00.952 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:55:00.952 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:55:00.952 INFO:teuthology.orchestra.run.trial138.stdout:Install 1 Package 2026-02-20T23:55:00.952 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:55:00.952 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 86 k 2026-02-20T23:55:00.952 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 236 k 2026-02-20T23:55:00.952 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:55:01.550 INFO:teuthology.orchestra.run.trial138.stdout:rbd-fuse-20.3.0-5343.gbcdd429b.el9.x86_64.rpm 145 kB/s | 86 kB 00:00 2026-02-20T23:55:01.550 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:55:01.550 INFO:teuthology.orchestra.run.trial138.stdout:Total 144 kB/s | 86 kB 00:00 2026-02-20T23:55:01.550 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:55:01.554 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:55:01.554 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:55:01.585 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:55:01.585 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:55:01.635 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:55:01.652 INFO:teuthology.orchestra.run.trial138.stdout: Installing : rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:55:01.731 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:55:01.789 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:55:01.789 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:55:01.790 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-20T23:55:01.790 INFO:teuthology.orchestra.run.trial138.stdout: rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:55:01.790 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:55:01.790 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:55:01.817 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install rbd-mirror 2026-02-20T23:55:02.126 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:08:08 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:55:02.210 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:55:02.210 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:55:02.210 INFO:teuthology.orchestra.run.trial138.stdout: Package Arch Version Repo Size 2026-02-20T23:55:02.210 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:55:02.210 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:55:02.211 INFO:teuthology.orchestra.run.trial138.stdout: rbd-mirror x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 2.6 M 2026-02-20T23:55:02.211 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:55:02.211 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:55:02.211 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:55:02.211 INFO:teuthology.orchestra.run.trial138.stdout:Install 1 Package 2026-02-20T23:55:02.211 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:55:02.211 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 2.6 M 2026-02-20T23:55:02.211 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 11 M 2026-02-20T23:55:02.211 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:55:10.612 INFO:teuthology.orchestra.run.trial138.stdout:rbd-mirror-20.3.0-5343.gbcdd429b.el9.x86_64.rpm 316 kB/s | 2.6 MB 00:08 2026-02-20T23:55:10.613 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:55:10.613 INFO:teuthology.orchestra.run.trial138.stdout:Total 316 kB/s | 2.6 MB 00:08 2026-02-20T23:55:10.613 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:55:10.619 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:55:10.619 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:55:10.657 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:55:10.657 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:55:10.826 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:55:10.828 INFO:teuthology.orchestra.run.trial138.stdout: Installing : rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:55:10.851 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:55:10.851 INFO:teuthology.orchestra.run.trial138.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T23:55:10.851 INFO:teuthology.orchestra.run.trial138.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-20T23:55:10.852 INFO:teuthology.orchestra.run.trial138.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T23:55:10.852 INFO:teuthology.orchestra.run.trial138.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T23:55:10.852 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:55:11.146 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:55:11.146 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:55:11.146 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-20T23:55:11.146 INFO:teuthology.orchestra.run.trial138.stdout: rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:55:11.146 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:55:11.146 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:55:11.173 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install rbd-nbd 2026-02-20T23:55:11.489 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:08:17 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:55:11.571 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:55:11.571 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:55:11.572 INFO:teuthology.orchestra.run.trial138.stdout: Package Architecture Version Repository Size 2026-02-20T23:55:11.572 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:55:11.572 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:55:11.572 INFO:teuthology.orchestra.run.trial138.stdout: rbd-nbd x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 166 k 2026-02-20T23:55:11.572 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:55:11.572 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:55:11.572 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:55:11.572 INFO:teuthology.orchestra.run.trial138.stdout:Install 1 Package 2026-02-20T23:55:11.572 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:55:11.572 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 166 k 2026-02-20T23:55:11.572 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 496 k 2026-02-20T23:55:11.572 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-20T23:55:12.206 INFO:teuthology.orchestra.run.trial138.stdout:rbd-nbd-20.3.0-5343.gbcdd429b.el9.x86_64.rpm 262 kB/s | 166 kB 00:00 2026-02-20T23:55:12.208 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-20T23:55:12.208 INFO:teuthology.orchestra.run.trial138.stdout:Total 261 kB/s | 166 kB 00:00 2026-02-20T23:55:12.208 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-20T23:55:12.213 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-20T23:55:12.213 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-20T23:55:12.244 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-20T23:55:12.244 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-20T23:55:12.299 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-20T23:55:12.316 INFO:teuthology.orchestra.run.trial138.stdout: Installing : rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:55:12.401 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:55:12.460 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/1 2026-02-20T23:55:12.460 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:55:12.460 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-20T23:55:12.460 INFO:teuthology.orchestra.run.trial138.stdout: rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-20T23:55:12.460 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:55:12.460 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:55:12.490 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-osd-crimson 2026-02-20T23:55:12.802 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:08:18 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:55:12.870 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-osd-crimson-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-20T23:55:12.889 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:55:12.889 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-20T23:55:12.890 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-20T23:55:12.911 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-20T23:55:13.219 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:08:19 ago on Fri 20 Feb 2026 11:46:54 PM UTC. 2026-02-20T23:55:13.303 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-20T23:55:13.303 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:55:13.303 INFO:teuthology.orchestra.run.trial138.stdout: Package Arch Version Repo Size 2026-02-20T23:55:13.303 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:55:13.303 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-20T23:55:13.303 INFO:teuthology.orchestra.run.trial138.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 433 M 2026-02-20T23:55:13.304 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:55:13.304 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-20T23:55:13.304 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-20T23:55:13.304 INFO:teuthology.orchestra.run.trial138.stdout:Install 1 Package 2026-02-20T23:55:13.304 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-20T23:55:13.304 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 433 M 2026-02-20T23:55:13.304 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 2.6 G 2026-02-20T23:55:13.304 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-21T00:04:17.832 INFO:teuthology.orchestra.run.trial138.stdout:ceph-osd-crimson-debuginfo-20.3.0-5343.gbcdd429 815 kB/s | 433 MB 09:04 2026-02-21T00:04:17.833 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:04:17.833 INFO:teuthology.orchestra.run.trial138.stdout:Total 815 kB/s | 433 MB 09:04 2026-02-21T00:04:17.833 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-21T00:04:17.835 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-21T00:04:17.836 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-21T00:04:19.129 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-21T00:04:19.129 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-21T00:04:42.700 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-21T00:04:42.717 INFO:teuthology.orchestra.run.trial138.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5343.gbcdd429b.e 1/1 2026-02-21T00:04:44.562 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5343.gbcdd429b.e 1/1 2026-02-21T00:04:44.619 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5343.gbcdd429b.e 1/1 2026-02-21T00:04:44.619 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:04:44.619 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-21T00:04:44.619 INFO:teuthology.orchestra.run.trial138.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:04:44.619 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:04:44.619 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:04:44.696 DEBUG:teuthology.parallel:result is None 2026-02-21T00:04:44.696 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-21T00:04:44.869 DEBUG:teuthology.orchestra.run.trial138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-21T00:04:44.891 INFO:teuthology.orchestra.run.trial138.stdout:20.3.0-5343.gbcdd429b.el9 2026-02-21T00:04:44.891 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5343.gbcdd429b.el9 2026-02-21T00:04:44.892 INFO:teuthology.task.install:The correct ceph version 20.3.0-5343.gbcdd429b is installed. 2026-02-21T00:04:44.894 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-21T00:04:44.894 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:04:44.894 DEBUG:teuthology.orchestra.run.trial138:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-21T00:04:44.957 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-21T00:04:44.958 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:04:44.958 DEBUG:teuthology.orchestra.run.trial138:> sudo dd of=/usr/bin/daemon-helper 2026-02-21T00:04:45.021 DEBUG:teuthology.orchestra.run.trial138:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-21T00:04:45.085 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-21T00:04:45.085 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:04:45.086 DEBUG:teuthology.orchestra.run.trial138:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-21T00:04:45.151 DEBUG:teuthology.orchestra.run.trial138:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-21T00:04:45.214 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-21T00:04:45.214 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:04:45.214 DEBUG:teuthology.orchestra.run.trial138:> sudo dd of=/usr/bin/stdin-killer 2026-02-21T00:04:45.280 DEBUG:teuthology.orchestra.run.trial138:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-21T00:04:45.344 INFO:teuthology.run_tasks:Running task pexec... 2026-02-21T00:04:45.356 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-21T00:04:45.356 DEBUG:teuthology.orchestra.run.trial138:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-21T00:04:45.387 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial138.front.sepia.ceph.com 2026-02-21T00:04:45.387 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-21T00:04:45.437 DEBUG:teuthology.parallel:result is None 2026-02-21T00:04:45.437 INFO:teuthology.run_tasks:Running task install... 2026-02-21T00:04:45.451 DEBUG:teuthology.task.install:project ceph 2026-02-21T00:04:45.452 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}} 2026-02-21T00:04:45.452 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'} 2026-02-21T00:04:45.452 INFO:teuthology.task.install:Using flavor: debug 2026-02-21T00:04:45.460 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-21T00:04:45.460 INFO:teuthology.task.install:extra packages: [] 2026-02-21T00:04:45.461 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'tag': None, 'wait_for_package': False} 2026-02-21T00:04:45.461 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-21T00:04:45.625 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/ 2026-02-21T00:04:45.625 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5343.gbcdd429b 2026-02-21T00:04:46.010 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/wip-liucm-osd-shard-redesign-crimson-only/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-21T00:04:46.010 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:04:46.010 DEBUG:teuthology.orchestra.run.trial138:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-21T00:04:46.036 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-21T00:04:46.036 DEBUG:teuthology.orchestra.run.trial138:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-21T00:04:46.103 DEBUG:teuthology.orchestra.run.trial138:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-21T00:04:46.166 DEBUG:teuthology.orchestra.run.trial138:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-21T00:04:46.222 INFO:teuthology.orchestra.run.trial138.stdout:check_obsoletes = 1 2026-02-21T00:04:46.232 DEBUG:teuthology.orchestra.run.trial138:> sudo yum clean all 2026-02-21T00:04:46.391 INFO:teuthology.orchestra.run.trial138.stdout:56 files removed 2026-02-21T00:04:46.405 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-radosgw 2026-02-21T00:04:47.245 INFO:teuthology.orchestra.run.trial138.stdout:ceph packages for x86_64 133 kB/s | 92 kB 00:00 2026-02-21T00:04:47.912 INFO:teuthology.orchestra.run.trial138.stdout:ceph noarch packages 37 kB/s | 24 kB 00:00 2026-02-21T00:04:48.519 INFO:teuthology.orchestra.run.trial138.stdout:ceph source packages 3.5 kB/s | 2.1 kB 00:00 2026-02-21T00:04:50.180 INFO:teuthology.orchestra.run.trial138.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.9 MB 00:01 2026-02-21T00:04:54.682 INFO:teuthology.orchestra.run.trial138.stdout:CentOS Stream 9 - AppStream 6.7 MB/s | 27 MB 00:03 2026-02-21T00:04:58.170 INFO:teuthology.orchestra.run.trial138.stdout:CentOS Stream 9 - CRB 6.9 MB/s | 7.8 MB 00:01 2026-02-21T00:04:59.074 INFO:teuthology.orchestra.run.trial138.stdout:CentOS Stream 9 - Extras packages 117 kB/s | 20 kB 00:00 2026-02-21T00:05:01.188 INFO:teuthology.orchestra.run.trial138.stdout:Extra Packages for Enterprise Linux 9.9 MB/s | 20 MB 00:02 2026-02-21T00:05:04.401 INFO:teuthology.orchestra.run.trial138.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-21T00:05:05.578 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-radosgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:05.596 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:05.597 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:05.597 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:05.622 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-test 2026-02-21T00:05:05.933 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:01 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:05.997 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-test-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:06.016 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:06.016 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:06.017 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:06.037 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph 2026-02-21T00:05:06.348 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:02 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:06.414 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:06.432 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:06.433 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:06.433 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:06.455 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-base 2026-02-21T00:05:06.769 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:02 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:06.833 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-base-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:06.851 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:06.852 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:06.852 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:06.873 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install cephadm 2026-02-21T00:05:07.180 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:03 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:07.245 INFO:teuthology.orchestra.run.trial138.stdout:Package cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:05:07.263 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:07.264 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:07.264 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:07.287 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-immutable-object-cache 2026-02-21T00:05:07.600 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:03 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:07.665 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-immutable-object-cache-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:07.683 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:07.684 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:07.684 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:07.705 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-mgr 2026-02-21T00:05:08.013 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:04 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:08.077 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-mgr-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:08.095 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:08.096 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:08.096 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:08.116 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-mgr-dashboard 2026-02-21T00:05:08.427 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:04 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:08.493 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-mgr-dashboard-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:05:08.512 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:08.512 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:08.512 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:08.534 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-21T00:05:08.848 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:04 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:08.914 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:05:08.932 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:08.933 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:08.933 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:08.954 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-mgr-rook 2026-02-21T00:05:09.259 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:05 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:09.324 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-mgr-rook-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:05:09.343 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:09.343 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:09.343 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:09.364 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-mgr-cephadm 2026-02-21T00:05:09.671 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:05 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:09.737 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-mgr-cephadm-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:05:09.755 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:09.755 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:09.756 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:09.776 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-osd 2026-02-21T00:05:10.084 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:06 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:10.148 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-osd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:10.166 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:10.167 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:10.167 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:10.188 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-osd-classic 2026-02-21T00:05:10.495 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:06 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:10.561 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-osd-classic-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:10.579 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:10.579 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:10.579 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:10.600 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-fuse 2026-02-21T00:05:10.907 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:06 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:10.971 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:10.989 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:10.990 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:10.990 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:11.010 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install ceph-volume 2026-02-21T00:05:11.318 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:07 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:11.384 INFO:teuthology.orchestra.run.trial138.stdout:Package ceph-volume-2:20.3.0-5343.gbcdd429b.el9.noarch is already installed. 2026-02-21T00:05:11.402 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:11.403 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:11.403 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:11.424 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install librados-devel 2026-02-21T00:05:11.735 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:07 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:11.801 INFO:teuthology.orchestra.run.trial138.stdout:Package librados-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:11.819 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:11.820 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:11.820 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:11.841 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install libcephfs2 2026-02-21T00:05:12.150 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:08 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:12.216 INFO:teuthology.orchestra.run.trial138.stdout:Package libcephfs2-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:12.234 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:12.235 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:12.235 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:12.255 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install libcephfs-devel 2026-02-21T00:05:12.565 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:08 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:12.630 INFO:teuthology.orchestra.run.trial138.stdout:Package libcephfs-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:12.648 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:12.649 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:12.649 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:12.670 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install librados2 2026-02-21T00:05:12.986 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:08 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:13.050 INFO:teuthology.orchestra.run.trial138.stdout:Package librados2-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:13.068 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:13.069 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:13.069 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:13.091 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install librbd1 2026-02-21T00:05:13.403 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:09 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:13.468 INFO:teuthology.orchestra.run.trial138.stdout:Package librbd1-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:13.487 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:13.488 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:13.488 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:13.508 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install python3-rados 2026-02-21T00:05:13.813 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:09 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:13.878 INFO:teuthology.orchestra.run.trial138.stdout:Package python3-rados-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:13.896 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:13.897 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:13.897 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:13.918 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install python3-rgw 2026-02-21T00:05:14.228 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:10 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:14.292 INFO:teuthology.orchestra.run.trial138.stdout:Package python3-rgw-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:14.312 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:14.312 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:14.312 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:14.333 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install python3-cephfs 2026-02-21T00:05:14.639 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:10 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:14.705 INFO:teuthology.orchestra.run.trial138.stdout:Package python3-cephfs-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:14.724 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:14.724 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:14.724 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:14.745 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install python3-rbd 2026-02-21T00:05:15.058 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:11 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:15.125 INFO:teuthology.orchestra.run.trial138.stdout:Package python3-rbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:15.143 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:15.144 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:15.144 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:15.167 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install rbd-fuse 2026-02-21T00:05:15.481 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:11 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:15.545 INFO:teuthology.orchestra.run.trial138.stdout:Package rbd-fuse-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:15.564 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:15.565 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:15.565 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:15.586 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install rbd-mirror 2026-02-21T00:05:15.894 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:11 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:15.960 INFO:teuthology.orchestra.run.trial138.stdout:Package rbd-mirror-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:15.979 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:15.979 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:15.979 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:16.001 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install rbd-nbd 2026-02-21T00:05:16.314 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:12 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:05:16.378 INFO:teuthology.orchestra.run.trial138.stdout:Package rbd-nbd-2:20.3.0-5343.gbcdd429b.el9.x86_64 is already installed. 2026-02-21T00:05:16.396 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:05:16.397 INFO:teuthology.orchestra.run.trial138.stdout:Nothing to do. 2026-02-21T00:05:16.397 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:05:16.421 DEBUG:teuthology.parallel:result is None 2026-02-21T00:05:16.422 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f 2026-02-21T00:05:16.589 DEBUG:teuthology.orchestra.run.trial138:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-21T00:05:16.610 INFO:teuthology.orchestra.run.trial138.stdout:20.3.0-5343.gbcdd429b.el9 2026-02-21T00:05:16.610 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5343.gbcdd429b.el9 2026-02-21T00:05:16.610 INFO:teuthology.task.install:The correct ceph version 20.3.0-5343.gbcdd429b is installed. 2026-02-21T00:05:16.612 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-21T00:05:16.612 INFO:teuthology.run_tasks:Running task ceph... 2026-02-21T00:05:16.696 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-21T00:05:16.697 DEBUG:teuthology.orchestra.run.trial138:> sudo chmod 777 /var/log/ceph 2026-02-21T00:05:16.721 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-21T00:05:16.721 DEBUG:teuthology.orchestra.run.trial138:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-21T00:05:16.785 INFO:tasks.ceph:Creating extra log directories... 2026-02-21T00:05:16.785 DEBUG:teuthology.orchestra.run.trial138:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-21T00:05:16.860 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-21T00:05:16.860 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-21T00:05:16.861 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61909', 'branch': 'wip-liucm-osd-shard-redesign-crimson-only', 'description': 'crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} perf_ignorelist settings/optimized workloads/radosbench_4K_seq_read}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '61909', 'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-liucm-osd-shard-redesign-crimson-only'}, 'ceph': {'conf': {'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}}, 'workunit': {'branch': 'wip-liucm-osd-shard-redesign-crimson-only', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}}, 'owner': 'scheduled_liucm@soko04.front.sepia.ceph.com', 'priority': 120, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 6283, 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'wip-liucm-osd-shard-redesign-crimson-only', 'suite_path': '/home/teuthworker/src/git.ceph.com_ceph-c_bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://git.ceph.com/ceph-ci.git', 'suite_sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'targets': {'trial138.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBL1i29wZ1CwcdTV0uC5Pdul/Z4fMZCQq2Y+tS5d5EwzVMMKM/kACnTOtzztUSUDtJ73GUXQ6FlH6I3mf5vXRCKQ='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': True, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'wait-for-scrub': False, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': 'bcdd429b2cc82df735d9dec4bb0cc600cdb61a8f', 'cluster': 'ceph'}}, {'ssh_keys': None}, {'cbt': {'benchmarks': {'radosbench': {'concurrent_ops': 4, 'concurrent_procs': 2, 'op_size': [4096], 'pool_monitoring_list': ['collectl'], 'pool_profile': 'replicated', 'run_monitoring_list': ['collectl'], 'time': 60, 'write_only': False}}, 'cluster': {'iterations': 1, 'osds_per_node': 3, 'pool_profiles': {'replicated': {'pg_size': 256, 'pgp_size': 256, 'replication': 'replicated'}}, 'user': 'ubuntu'}, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-20_22:16:48', 'tube': 'trial', 'user': 'liucm', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699'} 2026-02-21T00:05:16.861 DEBUG:teuthology.orchestra.run.trial138:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-21T00:05:16.909 DEBUG:teuthology.orchestra.run.trial138:> sudo install -d -m0777 -- /var/run/ceph 2026-02-21T00:05:16.973 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:16.973 DEBUG:teuthology.orchestra.run.trial138:> dd if=/scratch_devs of=/dev/stdout 2026-02-21T00:05:17.032 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-21T00:05:17.032 DEBUG:teuthology.orchestra.run.trial138:> stat /dev/vg_nvme/lv_1 2026-02-21T00:05:17.090 INFO:teuthology.orchestra.run.trial138.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-21T00:05:17.090 INFO:teuthology.orchestra.run.trial138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T00:05:17.090 INFO:teuthology.orchestra.run.trial138.stdout:Device: 6h/6d Inode: 911 Links: 1 2026-02-21T00:05:17.090 INFO:teuthology.orchestra.run.trial138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T00:05:17.090 INFO:teuthology.orchestra.run.trial138.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T00:05:17.091 INFO:teuthology.orchestra.run.trial138.stdout:Access: 2026-02-20 23:55:10.994869374 +0000 2026-02-21T00:05:17.091 INFO:teuthology.orchestra.run.trial138.stdout:Modify: 2026-02-20 23:46:24.754474271 +0000 2026-02-21T00:05:17.091 INFO:teuthology.orchestra.run.trial138.stdout:Change: 2026-02-20 23:46:24.754474271 +0000 2026-02-21T00:05:17.091 INFO:teuthology.orchestra.run.trial138.stdout: Birth: 2026-02-20 23:46:24.754474271 +0000 2026-02-21T00:05:17.091 DEBUG:teuthology.orchestra.run.trial138:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-21T00:05:17.153 INFO:teuthology.orchestra.run.trial138.stderr:1+0 records in 2026-02-21T00:05:17.153 INFO:teuthology.orchestra.run.trial138.stderr:1+0 records out 2026-02-21T00:05:17.154 INFO:teuthology.orchestra.run.trial138.stderr:512 bytes copied, 0.000174493 s, 2.9 MB/s 2026-02-21T00:05:17.155 DEBUG:teuthology.orchestra.run.trial138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-21T00:05:17.212 DEBUG:teuthology.orchestra.run.trial138:> stat /dev/vg_nvme/lv_2 2026-02-21T00:05:17.269 INFO:teuthology.orchestra.run.trial138.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-21T00:05:17.269 INFO:teuthology.orchestra.run.trial138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T00:05:17.269 INFO:teuthology.orchestra.run.trial138.stdout:Device: 6h/6d Inode: 939 Links: 1 2026-02-21T00:05:17.269 INFO:teuthology.orchestra.run.trial138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T00:05:17.269 INFO:teuthology.orchestra.run.trial138.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T00:05:17.269 INFO:teuthology.orchestra.run.trial138.stdout:Access: 2026-02-20 23:55:10.994869374 +0000 2026-02-21T00:05:17.270 INFO:teuthology.orchestra.run.trial138.stdout:Modify: 2026-02-20 23:46:24.993464965 +0000 2026-02-21T00:05:17.270 INFO:teuthology.orchestra.run.trial138.stdout:Change: 2026-02-20 23:46:24.993464965 +0000 2026-02-21T00:05:17.270 INFO:teuthology.orchestra.run.trial138.stdout: Birth: 2026-02-20 23:46:24.993464965 +0000 2026-02-21T00:05:17.270 DEBUG:teuthology.orchestra.run.trial138:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-21T00:05:17.332 INFO:teuthology.orchestra.run.trial138.stderr:1+0 records in 2026-02-21T00:05:17.332 INFO:teuthology.orchestra.run.trial138.stderr:1+0 records out 2026-02-21T00:05:17.333 INFO:teuthology.orchestra.run.trial138.stderr:512 bytes copied, 0.000150078 s, 3.4 MB/s 2026-02-21T00:05:17.334 DEBUG:teuthology.orchestra.run.trial138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-21T00:05:17.392 DEBUG:teuthology.orchestra.run.trial138:> stat /dev/vg_nvme/lv_3 2026-02-21T00:05:17.448 INFO:teuthology.orchestra.run.trial138.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-21T00:05:17.449 INFO:teuthology.orchestra.run.trial138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T00:05:17.449 INFO:teuthology.orchestra.run.trial138.stdout:Device: 6h/6d Inode: 967 Links: 1 2026-02-21T00:05:17.449 INFO:teuthology.orchestra.run.trial138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T00:05:17.449 INFO:teuthology.orchestra.run.trial138.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T00:05:17.449 INFO:teuthology.orchestra.run.trial138.stdout:Access: 2026-02-20 23:55:10.994869374 +0000 2026-02-21T00:05:17.449 INFO:teuthology.orchestra.run.trial138.stdout:Modify: 2026-02-20 23:46:25.246455114 +0000 2026-02-21T00:05:17.449 INFO:teuthology.orchestra.run.trial138.stdout:Change: 2026-02-20 23:46:25.246455114 +0000 2026-02-21T00:05:17.449 INFO:teuthology.orchestra.run.trial138.stdout: Birth: 2026-02-20 23:46:25.246455114 +0000 2026-02-21T00:05:17.450 DEBUG:teuthology.orchestra.run.trial138:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-21T00:05:17.512 INFO:teuthology.orchestra.run.trial138.stderr:1+0 records in 2026-02-21T00:05:17.512 INFO:teuthology.orchestra.run.trial138.stderr:1+0 records out 2026-02-21T00:05:17.512 INFO:teuthology.orchestra.run.trial138.stderr:512 bytes copied, 0.000146731 s, 3.5 MB/s 2026-02-21T00:05:17.513 DEBUG:teuthology.orchestra.run.trial138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-21T00:05:17.571 DEBUG:teuthology.orchestra.run.trial138:> stat /dev/vg_nvme/lv_4 2026-02-21T00:05:17.627 INFO:teuthology.orchestra.run.trial138.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-21T00:05:17.627 INFO:teuthology.orchestra.run.trial138.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-21T00:05:17.627 INFO:teuthology.orchestra.run.trial138.stdout:Device: 6h/6d Inode: 994 Links: 1 2026-02-21T00:05:17.627 INFO:teuthology.orchestra.run.trial138.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-21T00:05:17.627 INFO:teuthology.orchestra.run.trial138.stdout:Context: system_u:object_r:device_t:s0 2026-02-21T00:05:17.627 INFO:teuthology.orchestra.run.trial138.stdout:Access: 2026-02-20 23:55:10.995869344 +0000 2026-02-21T00:05:17.628 INFO:teuthology.orchestra.run.trial138.stdout:Modify: 2026-02-20 23:46:25.496445379 +0000 2026-02-21T00:05:17.628 INFO:teuthology.orchestra.run.trial138.stdout:Change: 2026-02-20 23:46:25.496445379 +0000 2026-02-21T00:05:17.628 INFO:teuthology.orchestra.run.trial138.stdout: Birth: 2026-02-20 23:46:25.496445379 +0000 2026-02-21T00:05:17.628 DEBUG:teuthology.orchestra.run.trial138:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-21T00:05:17.689 INFO:teuthology.orchestra.run.trial138.stderr:1+0 records in 2026-02-21T00:05:17.690 INFO:teuthology.orchestra.run.trial138.stderr:1+0 records out 2026-02-21T00:05:17.690 INFO:teuthology.orchestra.run.trial138.stderr:512 bytes copied, 0.000128148 s, 4.0 MB/s 2026-02-21T00:05:17.691 DEBUG:teuthology.orchestra.run.trial138:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-21T00:05:17.748 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T00:05:17.748 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial138.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-21T00:05:17.748 INFO:tasks.ceph:Generating config... 2026-02-21T00:05:17.751 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-21T00:05:17.751 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-21T00:05:17.751 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-21T00:05:17.751 INFO:tasks.ceph:[osd] debug filestore = 0/0 2026-02-21T00:05:17.752 INFO:tasks.ceph:[osd] debug journal = 0/0 2026-02-21T00:05:17.752 INFO:tasks.ceph:[osd] debug ms = 0/0 2026-02-21T00:05:17.752 INFO:tasks.ceph:[osd] debug osd = 0/0 2026-02-21T00:05:17.752 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-21T00:05:17.752 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-21T00:05:17.752 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-21T00:05:17.752 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-21T00:05:17.752 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-21T00:05:17.752 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-21T00:05:17.752 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-21T00:05:17.752 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-21T00:05:17.752 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-21T00:05:17.753 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-21T00:05:17.753 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-21T00:05:17.753 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-21T00:05:17.753 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-21T00:05:17.753 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-21T00:05:17.753 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-21T00:05:17.753 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-21T00:05:17.753 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-21T00:05:17.753 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-21T00:05:17.753 INFO:tasks.ceph:[osd] seastore main device type = RANDOM_BLOCK_SSD 2026-02-21T00:05:17.753 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-21T00:05:17.753 INFO:tasks.ceph:[global] auth client required = none 2026-02-21T00:05:17.754 INFO:tasks.ceph:[global] auth cluster required = none 2026-02-21T00:05:17.754 INFO:tasks.ceph:[global] auth service required = none 2026-02-21T00:05:17.754 INFO:tasks.ceph:[global] auth supported = none 2026-02-21T00:05:17.754 INFO:tasks.ceph:[global] debug asok = 0/0 2026-02-21T00:05:17.754 INFO:tasks.ceph:[global] debug auth = 0/0 2026-02-21T00:05:17.754 INFO:tasks.ceph:[global] debug bdev = 0/0 2026-02-21T00:05:17.754 INFO:tasks.ceph:[global] debug bluefs = 0/0 2026-02-21T00:05:17.754 INFO:tasks.ceph:[global] debug bluestore = 0/0 2026-02-21T00:05:17.754 INFO:tasks.ceph:[global] debug buffer = 0/0 2026-02-21T00:05:17.754 INFO:tasks.ceph:[global] debug civetweb = 0/0 2026-02-21T00:05:17.754 INFO:tasks.ceph:[global] debug client = 0/0 2026-02-21T00:05:17.754 INFO:tasks.ceph:[global] debug compressor = 0/0 2026-02-21T00:05:17.754 INFO:tasks.ceph:[global] debug context = 0/0 2026-02-21T00:05:17.755 INFO:tasks.ceph:[global] debug crush = 0/0 2026-02-21T00:05:17.755 INFO:tasks.ceph:[global] debug crypto = 0/0 2026-02-21T00:05:17.755 INFO:tasks.ceph:[global] debug dpdk = 0/0 2026-02-21T00:05:17.755 INFO:tasks.ceph:[global] debug eventtrace = 0/0 2026-02-21T00:05:17.755 INFO:tasks.ceph:[global] debug filer = 0/0 2026-02-21T00:05:17.755 INFO:tasks.ceph:[global] debug filestore = 0/0 2026-02-21T00:05:17.755 INFO:tasks.ceph:[global] debug finisher = 0/0 2026-02-21T00:05:17.755 INFO:tasks.ceph:[global] debug fuse = 0/0 2026-02-21T00:05:17.755 INFO:tasks.ceph:[global] debug heartbeatmap = 0/0 2026-02-21T00:05:17.755 INFO:tasks.ceph:[global] debug javaclient = 0/0 2026-02-21T00:05:17.755 INFO:tasks.ceph:[global] debug journal = 0/0 2026-02-21T00:05:17.755 INFO:tasks.ceph:[global] debug journaler = 0/0 2026-02-21T00:05:17.755 INFO:tasks.ceph:[global] debug kstore = 0/0 2026-02-21T00:05:17.756 INFO:tasks.ceph:[global] debug leveldb = 0/0 2026-02-21T00:05:17.756 INFO:tasks.ceph:[global] debug lockdep = 0/0 2026-02-21T00:05:17.756 INFO:tasks.ceph:[global] debug mds = 0/0 2026-02-21T00:05:17.756 INFO:tasks.ceph:[global] debug mds balancer = 0/0 2026-02-21T00:05:17.756 INFO:tasks.ceph:[global] debug mds locker = 0/0 2026-02-21T00:05:17.756 INFO:tasks.ceph:[global] debug mds log = 0/0 2026-02-21T00:05:17.756 INFO:tasks.ceph:[global] debug mds log expire = 0/0 2026-02-21T00:05:17.756 INFO:tasks.ceph:[global] debug mds migrator = 0/0 2026-02-21T00:05:17.756 INFO:tasks.ceph:[global] debug memdb = 0/0 2026-02-21T00:05:17.756 INFO:tasks.ceph:[global] debug mgr = 0/0 2026-02-21T00:05:17.756 INFO:tasks.ceph:[global] debug mgrc = 0/0 2026-02-21T00:05:17.756 INFO:tasks.ceph:[global] debug mon = 0/0 2026-02-21T00:05:17.757 INFO:tasks.ceph:[global] debug monc = 0/0 2026-02-21T00:05:17.757 INFO:tasks.ceph:[global] debug ms = 0/0 2026-02-21T00:05:17.757 INFO:tasks.ceph:[global] debug objclass = 0/0 2026-02-21T00:05:17.757 INFO:tasks.ceph:[global] debug objectcacher = 0/0 2026-02-21T00:05:17.757 INFO:tasks.ceph:[global] debug objecter = 0/0 2026-02-21T00:05:17.757 INFO:tasks.ceph:[global] debug optracker = 0/0 2026-02-21T00:05:17.757 INFO:tasks.ceph:[global] debug osd = 0/0 2026-02-21T00:05:17.757 INFO:tasks.ceph:[global] debug paxos = 0/0 2026-02-21T00:05:17.757 INFO:tasks.ceph:[global] debug perfcounter = 0/0 2026-02-21T00:05:17.757 INFO:tasks.ceph:[global] debug rados = 0/0 2026-02-21T00:05:17.757 INFO:tasks.ceph:[global] debug rbd = 0/0 2026-02-21T00:05:17.757 INFO:tasks.ceph:[global] debug rbd mirror = 0/0 2026-02-21T00:05:17.758 INFO:tasks.ceph:[global] debug rbd replay = 0/0 2026-02-21T00:05:17.758 INFO:tasks.ceph:[global] debug refs = 0/0 2026-02-21T00:05:17.758 INFO:tasks.ceph:[global] debug rgw = 0/0 2026-02-21T00:05:17.758 INFO:tasks.ceph:[global] debug rgw lifecycle = 0/0 2026-02-21T00:05:17.758 INFO:tasks.ceph:[global] debug rgw notification = 0/0 2026-02-21T00:05:17.758 INFO:tasks.ceph:[global] debug rgw sync = 0/0 2026-02-21T00:05:17.758 INFO:tasks.ceph:[global] debug rocksdb = 0/0 2026-02-21T00:05:17.758 INFO:tasks.ceph:[global] debug striper = 0/0 2026-02-21T00:05:17.758 INFO:tasks.ceph:[global] debug throttle = 0/0 2026-02-21T00:05:17.758 INFO:tasks.ceph:[global] debug timer = 0/0 2026-02-21T00:05:17.758 INFO:tasks.ceph:[global] debug tp = 0/0 2026-02-21T00:05:17.758 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-21T00:05:17.758 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-21T00:05:17.759 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-21T00:05:17.759 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-21T00:05:17.759 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-21T00:05:17.759 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-21T00:05:17.759 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-21T00:05:17.759 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-21T00:05:17.759 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-21T00:05:17.759 INFO:tasks.ceph:[mon] debug mon = 0/0 2026-02-21T00:05:17.759 INFO:tasks.ceph:[mon] debug ms = 0/0 2026-02-21T00:05:17.759 INFO:tasks.ceph:[mon] debug paxos = 0/0 2026-02-21T00:05:17.759 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-21T00:05:17.760 INFO:tasks.ceph:Setting up mon.a... 2026-02-21T00:05:17.760 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-21T00:05:17.828 INFO:teuthology.orchestra.run.trial138.stdout:creating /etc/ceph/ceph.keyring 2026-02-21T00:05:17.831 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-21T00:05:17.917 DEBUG:teuthology.orchestra.run.trial138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T00:05:17.942 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.138')] 2026-02-21T00:05:17.943 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'none', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.138', 'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': '0/0', 'debug mon': '0/0', 'debug paxos': '0/0', 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-21T00:05:17.944 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:17.945 DEBUG:teuthology.orchestra.run.trial138:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-21T00:05:18.000 DEBUG:teuthology.orchestra.run.trial138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.138 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:05:18.081 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:18.080+0000 7fe5fac85f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:18.081 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:18.080+0000 7fe5fac85f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:18.081 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:18.080+0000 7fe5fac85f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:18.082 INFO:teuthology.orchestra.run.trial138.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:05:18.083 INFO:teuthology.orchestra.run.trial138.stdout:monmaptool: generated fsid 20c293de-c5ff-4972-9b51-1667a3ab6ae9 2026-02-21T00:05:18.083 INFO:teuthology.orchestra.run.trial138.stdout:setting min_mon_release = tentacle 2026-02-21T00:05:18.083 INFO:teuthology.orchestra.run.trial138.stdout:epoch 0 2026-02-21T00:05:18.083 INFO:teuthology.orchestra.run.trial138.stdout:fsid 20c293de-c5ff-4972-9b51-1667a3ab6ae9 2026-02-21T00:05:18.083 INFO:teuthology.orchestra.run.trial138.stdout:last_changed 2026-02-21T00:05:18.080727+0000 2026-02-21T00:05:18.083 INFO:teuthology.orchestra.run.trial138.stdout:created 2026-02-21T00:05:18.080727+0000 2026-02-21T00:05:18.083 INFO:teuthology.orchestra.run.trial138.stdout:min_mon_release 20 (tentacle) 2026-02-21T00:05:18.083 INFO:teuthology.orchestra.run.trial138.stdout:election_strategy: 1 2026-02-21T00:05:18.083 INFO:teuthology.orchestra.run.trial138.stdout:0: [v2:10.20.193.138:3300/0,v1:10.20.193.138:6789/0] mon.a 2026-02-21T00:05:18.083 INFO:teuthology.orchestra.run.trial138.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-21T00:05:18.084 DEBUG:teuthology.orchestra.run.trial138:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-21T00:05:18.139 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 20c293de-c5ff-4972-9b51-1667a3ab6ae9... 2026-02-21T00:05:18.141 DEBUG:teuthology.orchestra.run.trial138:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-21T00:05:18.218 INFO:teuthology.orchestra.run.trial138.stdout:[global] 2026-02-21T00:05:18.218 INFO:teuthology.orchestra.run.trial138.stdout: chdir = "" 2026-02-21T00:05:18.218 INFO:teuthology.orchestra.run.trial138.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-21T00:05:18.218 INFO:teuthology.orchestra.run.trial138.stdout: auth supported = none 2026-02-21T00:05:18.219 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.219 INFO:teuthology.orchestra.run.trial138.stdout: filestore xattr use omap = true 2026-02-21T00:05:18.219 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.219 INFO:teuthology.orchestra.run.trial138.stdout: mon clock drift allowed = 1.000 2026-02-21T00:05:18.219 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.219 INFO:teuthology.orchestra.run.trial138.stdout: osd crush chooseleaf type = 0 2026-02-21T00:05:18.219 INFO:teuthology.orchestra.run.trial138.stdout: auth debug = true 2026-02-21T00:05:18.219 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.219 INFO:teuthology.orchestra.run.trial138.stdout: ms die on old message = true 2026-02-21T00:05:18.219 INFO:teuthology.orchestra.run.trial138.stdout: ms die on bug = true 2026-02-21T00:05:18.219 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.219 INFO:teuthology.orchestra.run.trial138.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-21T00:05:18.220 INFO:teuthology.orchestra.run.trial138.stdout: mon pg warn max object skew = 0 2026-02-21T00:05:18.220 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.220 INFO:teuthology.orchestra.run.trial138.stdout: # disable pg_autoscaler by default for new pools 2026-02-21T00:05:18.220 INFO:teuthology.orchestra.run.trial138.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-21T00:05:18.220 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.220 INFO:teuthology.orchestra.run.trial138.stdout: osd pool default size = 2 2026-02-21T00:05:18.220 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.220 INFO:teuthology.orchestra.run.trial138.stdout: mon osd allow primary affinity = true 2026-02-21T00:05:18.220 INFO:teuthology.orchestra.run.trial138.stdout: mon osd allow pg remap = true 2026-02-21T00:05:18.220 INFO:teuthology.orchestra.run.trial138.stdout: mon warn on legacy crush tunables = false 2026-02-21T00:05:18.220 INFO:teuthology.orchestra.run.trial138.stdout: mon warn on crush straw calc version zero = false 2026-02-21T00:05:18.220 INFO:teuthology.orchestra.run.trial138.stdout: mon warn on no sortbitwise = false 2026-02-21T00:05:18.221 INFO:teuthology.orchestra.run.trial138.stdout: mon warn on osd down out interval zero = false 2026-02-21T00:05:18.221 INFO:teuthology.orchestra.run.trial138.stdout: mon warn on too few osds = false 2026-02-21T00:05:18.221 INFO:teuthology.orchestra.run.trial138.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-21T00:05:18.221 INFO:teuthology.orchestra.run.trial138.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-21T00:05:18.221 INFO:teuthology.orchestra.run.trial138.stdout: mon_allow_pool_size_one = true 2026-02-21T00:05:18.221 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.221 INFO:teuthology.orchestra.run.trial138.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-21T00:05:18.221 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.221 INFO:teuthology.orchestra.run.trial138.stdout: osd default data pool replay window = 5 2026-02-21T00:05:18.221 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.221 INFO:teuthology.orchestra.run.trial138.stdout: mon allow pool delete = true 2026-02-21T00:05:18.221 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.222 INFO:teuthology.orchestra.run.trial138.stdout: mon cluster log file level = debug 2026-02-21T00:05:18.222 INFO:teuthology.orchestra.run.trial138.stdout: debug asserts on shutdown = true 2026-02-21T00:05:18.222 INFO:teuthology.orchestra.run.trial138.stdout: mon health detail to clog = false 2026-02-21T00:05:18.222 INFO:teuthology.orchestra.run.trial138.stdout: mon host = 10.20.193.138 2026-02-21T00:05:18.222 INFO:teuthology.orchestra.run.trial138.stdout: auth client required = none 2026-02-21T00:05:18.222 INFO:teuthology.orchestra.run.trial138.stdout: auth cluster required = none 2026-02-21T00:05:18.222 INFO:teuthology.orchestra.run.trial138.stdout: auth service required = none 2026-02-21T00:05:18.222 INFO:teuthology.orchestra.run.trial138.stdout: debug asok = 0/0 2026-02-21T00:05:18.222 INFO:teuthology.orchestra.run.trial138.stdout: debug auth = 0/0 2026-02-21T00:05:18.222 INFO:teuthology.orchestra.run.trial138.stdout: debug bdev = 0/0 2026-02-21T00:05:18.222 INFO:teuthology.orchestra.run.trial138.stdout: debug bluefs = 0/0 2026-02-21T00:05:18.222 INFO:teuthology.orchestra.run.trial138.stdout: debug bluestore = 0/0 2026-02-21T00:05:18.223 INFO:teuthology.orchestra.run.trial138.stdout: debug buffer = 0/0 2026-02-21T00:05:18.223 INFO:teuthology.orchestra.run.trial138.stdout: debug civetweb = 0/0 2026-02-21T00:05:18.223 INFO:teuthology.orchestra.run.trial138.stdout: debug client = 0/0 2026-02-21T00:05:18.223 INFO:teuthology.orchestra.run.trial138.stdout: debug compressor = 0/0 2026-02-21T00:05:18.223 INFO:teuthology.orchestra.run.trial138.stdout: debug context = 0/0 2026-02-21T00:05:18.223 INFO:teuthology.orchestra.run.trial138.stdout: debug crush = 0/0 2026-02-21T00:05:18.223 INFO:teuthology.orchestra.run.trial138.stdout: debug crypto = 0/0 2026-02-21T00:05:18.223 INFO:teuthology.orchestra.run.trial138.stdout: debug dpdk = 0/0 2026-02-21T00:05:18.223 INFO:teuthology.orchestra.run.trial138.stdout: debug eventtrace = 0/0 2026-02-21T00:05:18.223 INFO:teuthology.orchestra.run.trial138.stdout: debug filer = 0/0 2026-02-21T00:05:18.223 INFO:teuthology.orchestra.run.trial138.stdout: debug filestore = 0/0 2026-02-21T00:05:18.223 INFO:teuthology.orchestra.run.trial138.stdout: debug finisher = 0/0 2026-02-21T00:05:18.223 INFO:teuthology.orchestra.run.trial138.stdout: debug fuse = 0/0 2026-02-21T00:05:18.224 INFO:teuthology.orchestra.run.trial138.stdout: debug heartbeatmap = 0/0 2026-02-21T00:05:18.224 INFO:teuthology.orchestra.run.trial138.stdout: debug javaclient = 0/0 2026-02-21T00:05:18.224 INFO:teuthology.orchestra.run.trial138.stdout: debug journal = 0/0 2026-02-21T00:05:18.224 INFO:teuthology.orchestra.run.trial138.stdout: debug journaler = 0/0 2026-02-21T00:05:18.224 INFO:teuthology.orchestra.run.trial138.stdout: debug kstore = 0/0 2026-02-21T00:05:18.224 INFO:teuthology.orchestra.run.trial138.stdout: debug leveldb = 0/0 2026-02-21T00:05:18.224 INFO:teuthology.orchestra.run.trial138.stdout: debug lockdep = 0/0 2026-02-21T00:05:18.224 INFO:teuthology.orchestra.run.trial138.stdout: debug mds = 0/0 2026-02-21T00:05:18.224 INFO:teuthology.orchestra.run.trial138.stdout: debug mds balancer = 0/0 2026-02-21T00:05:18.224 INFO:teuthology.orchestra.run.trial138.stdout: debug mds locker = 0/0 2026-02-21T00:05:18.224 INFO:teuthology.orchestra.run.trial138.stdout: debug mds log = 0/0 2026-02-21T00:05:18.224 INFO:teuthology.orchestra.run.trial138.stdout: debug mds log expire = 0/0 2026-02-21T00:05:18.225 INFO:teuthology.orchestra.run.trial138.stdout: debug mds migrator = 0/0 2026-02-21T00:05:18.225 INFO:teuthology.orchestra.run.trial138.stdout: debug memdb = 0/0 2026-02-21T00:05:18.225 INFO:teuthology.orchestra.run.trial138.stdout: debug mgr = 0/0 2026-02-21T00:05:18.225 INFO:teuthology.orchestra.run.trial138.stdout: debug mgrc = 0/0 2026-02-21T00:05:18.225 INFO:teuthology.orchestra.run.trial138.stdout: debug mon = 0/0 2026-02-21T00:05:18.225 INFO:teuthology.orchestra.run.trial138.stdout: debug monc = 0/0 2026-02-21T00:05:18.225 INFO:teuthology.orchestra.run.trial138.stdout: debug ms = 0/0 2026-02-21T00:05:18.225 INFO:teuthology.orchestra.run.trial138.stdout: debug objclass = 0/0 2026-02-21T00:05:18.225 INFO:teuthology.orchestra.run.trial138.stdout: debug objectcacher = 0/0 2026-02-21T00:05:18.225 INFO:teuthology.orchestra.run.trial138.stdout: debug objecter = 0/0 2026-02-21T00:05:18.225 INFO:teuthology.orchestra.run.trial138.stdout: debug optracker = 0/0 2026-02-21T00:05:18.225 INFO:teuthology.orchestra.run.trial138.stdout: debug osd = 0/0 2026-02-21T00:05:18.225 INFO:teuthology.orchestra.run.trial138.stdout: debug paxos = 0/0 2026-02-21T00:05:18.226 INFO:teuthology.orchestra.run.trial138.stdout: debug perfcounter = 0/0 2026-02-21T00:05:18.226 INFO:teuthology.orchestra.run.trial138.stdout: debug rados = 0/0 2026-02-21T00:05:18.226 INFO:teuthology.orchestra.run.trial138.stdout: debug rbd = 0/0 2026-02-21T00:05:18.226 INFO:teuthology.orchestra.run.trial138.stdout: debug rbd mirror = 0/0 2026-02-21T00:05:18.226 INFO:teuthology.orchestra.run.trial138.stdout: debug rbd replay = 0/0 2026-02-21T00:05:18.226 INFO:teuthology.orchestra.run.trial138.stdout: debug refs = 0/0 2026-02-21T00:05:18.226 INFO:teuthology.orchestra.run.trial138.stdout: debug rgw = 0/0 2026-02-21T00:05:18.226 INFO:teuthology.orchestra.run.trial138.stdout: debug rgw lifecycle = 0/0 2026-02-21T00:05:18.226 INFO:teuthology.orchestra.run.trial138.stdout: debug rgw notification = 0/0 2026-02-21T00:05:18.226 INFO:teuthology.orchestra.run.trial138.stdout: debug rgw sync = 0/0 2026-02-21T00:05:18.226 INFO:teuthology.orchestra.run.trial138.stdout: debug rocksdb = 0/0 2026-02-21T00:05:18.226 INFO:teuthology.orchestra.run.trial138.stdout: debug striper = 0/0 2026-02-21T00:05:18.226 INFO:teuthology.orchestra.run.trial138.stdout: debug throttle = 0/0 2026-02-21T00:05:18.226 INFO:teuthology.orchestra.run.trial138.stdout: debug timer = 0/0 2026-02-21T00:05:18.227 INFO:teuthology.orchestra.run.trial138.stdout: debug tp = 0/0 2026-02-21T00:05:18.227 INFO:teuthology.orchestra.run.trial138.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-21T00:05:18.227 INFO:teuthology.orchestra.run.trial138.stdout: ms client mode = crc 2026-02-21T00:05:18.227 INFO:teuthology.orchestra.run.trial138.stdout: ms cluster mode = crc 2026-02-21T00:05:18.227 INFO:teuthology.orchestra.run.trial138.stdout: ms mon client mode = crc 2026-02-21T00:05:18.227 INFO:teuthology.orchestra.run.trial138.stdout: ms mon cluster mode = crc 2026-02-21T00:05:18.227 INFO:teuthology.orchestra.run.trial138.stdout: ms mon service mode = crc 2026-02-21T00:05:18.227 INFO:teuthology.orchestra.run.trial138.stdout: ms service mode = crc 2026-02-21T00:05:18.227 INFO:teuthology.orchestra.run.trial138.stdout: fsid = 20c293de-c5ff-4972-9b51-1667a3ab6ae9 2026-02-21T00:05:18.227 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.227 INFO:teuthology.orchestra.run.trial138.stdout:[osd] 2026-02-21T00:05:18.227 INFO:teuthology.orchestra.run.trial138.stdout: osd journal size = 100 2026-02-21T00:05:18.228 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.228 INFO:teuthology.orchestra.run.trial138.stdout: osd scrub load threshold = 5.0 2026-02-21T00:05:18.228 INFO:teuthology.orchestra.run.trial138.stdout: osd scrub max interval = 600 2026-02-21T00:05:18.228 INFO:teuthology.orchestra.run.trial138.stdout: osd mclock profile = high_recovery_ops 2026-02-21T00:05:18.228 INFO:teuthology.orchestra.run.trial138.stdout: osd mclock skip benchmark = true 2026-02-21T00:05:18.228 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.228 INFO:teuthology.orchestra.run.trial138.stdout: osd recover clone overlap = true 2026-02-21T00:05:18.228 INFO:teuthology.orchestra.run.trial138.stdout: osd recovery max chunk = 1048576 2026-02-21T00:05:18.228 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.228 INFO:teuthology.orchestra.run.trial138.stdout: osd debug shutdown = true 2026-02-21T00:05:18.228 INFO:teuthology.orchestra.run.trial138.stdout: osd debug op order = true 2026-02-21T00:05:18.228 INFO:teuthology.orchestra.run.trial138.stdout: osd debug verify stray on activate = true 2026-02-21T00:05:18.229 INFO:teuthology.orchestra.run.trial138.stdout: osd debug trim objects = true 2026-02-21T00:05:18.229 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.229 INFO:teuthology.orchestra.run.trial138.stdout: osd open classes on start = true 2026-02-21T00:05:18.229 INFO:teuthology.orchestra.run.trial138.stdout: osd debug pg log writeout = true 2026-02-21T00:05:18.229 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.229 INFO:teuthology.orchestra.run.trial138.stdout: osd deep scrub update digest min age = 30 2026-02-21T00:05:18.229 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.229 INFO:teuthology.orchestra.run.trial138.stdout: osd map max advance = 10 2026-02-21T00:05:18.229 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.229 INFO:teuthology.orchestra.run.trial138.stdout: journal zero on create = true 2026-02-21T00:05:18.229 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.229 INFO:teuthology.orchestra.run.trial138.stdout: filestore ondisk finisher threads = 3 2026-02-21T00:05:18.229 INFO:teuthology.orchestra.run.trial138.stdout: filestore apply finisher threads = 3 2026-02-21T00:05:18.230 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.230 INFO:teuthology.orchestra.run.trial138.stdout: bdev debug aio = true 2026-02-21T00:05:18.230 INFO:teuthology.orchestra.run.trial138.stdout: osd debug misdirected ops = true 2026-02-21T00:05:18.230 INFO:teuthology.orchestra.run.trial138.stdout: debug monc = 20 2026-02-21T00:05:18.230 INFO:teuthology.orchestra.run.trial138.stdout: crimson cpu num = 4 2026-02-21T00:05:18.230 INFO:teuthology.orchestra.run.trial138.stdout: crimson osd obc lru size = 10 2026-02-21T00:05:18.230 INFO:teuthology.orchestra.run.trial138.stdout: debug filestore = 0/0 2026-02-21T00:05:18.230 INFO:teuthology.orchestra.run.trial138.stdout: debug journal = 0/0 2026-02-21T00:05:18.230 INFO:teuthology.orchestra.run.trial138.stdout: debug ms = 0/0 2026-02-21T00:05:18.230 INFO:teuthology.orchestra.run.trial138.stdout: debug osd = 0/0 2026-02-21T00:05:18.230 INFO:teuthology.orchestra.run.trial138.stdout: debug seastore = 20 2026-02-21T00:05:18.230 INFO:teuthology.orchestra.run.trial138.stdout: debug seastore backref = 20 2026-02-21T00:05:18.231 INFO:teuthology.orchestra.run.trial138.stdout: debug seastore cache = 20 2026-02-21T00:05:18.231 INFO:teuthology.orchestra.run.trial138.stdout: debug seastore cleaner = 20 2026-02-21T00:05:18.231 INFO:teuthology.orchestra.run.trial138.stdout: debug seastore device = 20 2026-02-21T00:05:18.231 INFO:teuthology.orchestra.run.trial138.stdout: debug seastore epm = 20 2026-02-21T00:05:18.231 INFO:teuthology.orchestra.run.trial138.stdout: debug seastore fixedkv tree = 20 2026-02-21T00:05:18.231 INFO:teuthology.orchestra.run.trial138.stdout: debug seastore journal = 20 2026-02-21T00:05:18.231 INFO:teuthology.orchestra.run.trial138.stdout: debug seastore lba = 20 2026-02-21T00:05:18.232 INFO:teuthology.orchestra.run.trial138.stdout: debug seastore odata = 20 2026-02-21T00:05:18.232 INFO:teuthology.orchestra.run.trial138.stdout: debug seastore omap = 20 2026-02-21T00:05:18.232 INFO:teuthology.orchestra.run.trial138.stdout: debug seastore onode = 20 2026-02-21T00:05:18.232 INFO:teuthology.orchestra.run.trial138.stdout: debug seastore t = 20 2026-02-21T00:05:18.232 INFO:teuthology.orchestra.run.trial138.stdout: debug seastore tm = 20 2026-02-21T00:05:18.232 INFO:teuthology.orchestra.run.trial138.stdout: osd objectstore = seastore 2026-02-21T00:05:18.232 INFO:teuthology.orchestra.run.trial138.stdout: osd shutdown pgref assert = True 2026-02-21T00:05:18.232 INFO:teuthology.orchestra.run.trial138.stdout: seastore cachepin size pershard = 64M 2026-02-21T00:05:18.232 INFO:teuthology.orchestra.run.trial138.stdout: seastore cachepin type = 2Q 2026-02-21T00:05:18.232 INFO:teuthology.orchestra.run.trial138.stdout: seastore main device type = RANDOM_BLOCK_SSD 2026-02-21T00:05:18.232 INFO:teuthology.orchestra.run.trial138.stdout: seastore max concurrent transactions = 8 2026-02-21T00:05:18.232 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.233 INFO:teuthology.orchestra.run.trial138.stdout:[mgr] 2026-02-21T00:05:18.233 INFO:teuthology.orchestra.run.trial138.stdout: debug ms = 1 2026-02-21T00:05:18.233 INFO:teuthology.orchestra.run.trial138.stdout: debug mgr = 20 2026-02-21T00:05:18.233 INFO:teuthology.orchestra.run.trial138.stdout: debug mon = 20 2026-02-21T00:05:18.233 INFO:teuthology.orchestra.run.trial138.stdout: debug auth = 20 2026-02-21T00:05:18.233 INFO:teuthology.orchestra.run.trial138.stdout: mon reweight min pgs per osd = 4 2026-02-21T00:05:18.233 INFO:teuthology.orchestra.run.trial138.stdout: mon reweight min bytes per osd = 10 2026-02-21T00:05:18.233 INFO:teuthology.orchestra.run.trial138.stdout: mgr/telemetry/nag = false 2026-02-21T00:05:18.233 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.233 INFO:teuthology.orchestra.run.trial138.stdout:[mon] 2026-02-21T00:05:18.233 INFO:teuthology.orchestra.run.trial138.stdout: debug ms = 0/0 2026-02-21T00:05:18.234 INFO:teuthology.orchestra.run.trial138.stdout: debug mon = 0/0 2026-02-21T00:05:18.234 INFO:teuthology.orchestra.run.trial138.stdout: debug paxos = 0/0 2026-02-21T00:05:18.234 INFO:teuthology.orchestra.run.trial138.stdout: debug auth = 20 2026-02-21T00:05:18.234 INFO:teuthology.orchestra.run.trial138.stdout: mon data avail warn = 5 2026-02-21T00:05:18.234 INFO:teuthology.orchestra.run.trial138.stdout: mon mgr mkfs grace = 240 2026-02-21T00:05:18.234 INFO:teuthology.orchestra.run.trial138.stdout: mon reweight min pgs per osd = 4 2026-02-21T00:05:18.234 INFO:teuthology.orchestra.run.trial138.stdout: mon osd reporter subtree level = osd 2026-02-21T00:05:18.234 INFO:teuthology.orchestra.run.trial138.stdout: mon osd prime pg temp = true 2026-02-21T00:05:18.234 INFO:teuthology.orchestra.run.trial138.stdout: mon reweight min bytes per osd = 10 2026-02-21T00:05:18.234 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.234 INFO:teuthology.orchestra.run.trial138.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-21T00:05:18.234 INFO:teuthology.orchestra.run.trial138.stdout: auth mon ticket ttl = 660 # 11m 2026-02-21T00:05:18.234 INFO:teuthology.orchestra.run.trial138.stdout: auth service ticket ttl = 240 # 4m 2026-02-21T00:05:18.235 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.235 INFO:teuthology.orchestra.run.trial138.stdout: # don't complain about insecure global_id in the test suite 2026-02-21T00:05:18.235 INFO:teuthology.orchestra.run.trial138.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-21T00:05:18.235 INFO:teuthology.orchestra.run.trial138.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-21T00:05:18.235 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.235 INFO:teuthology.orchestra.run.trial138.stdout: # 1m isn't quite enough 2026-02-21T00:05:18.235 INFO:teuthology.orchestra.run.trial138.stdout: mon_down_mkfs_grace = 2m 2026-02-21T00:05:18.235 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.235 INFO:teuthology.orchestra.run.trial138.stdout: mon_warn_on_filestore_osds = false 2026-02-21T00:05:18.235 INFO:teuthology.orchestra.run.trial138.stdout: osd pool default crimson = True 2026-02-21T00:05:18.235 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:18.235 INFO:teuthology.orchestra.run.trial138.stdout:[client] 2026-02-21T00:05:18.236 INFO:teuthology.orchestra.run.trial138.stdout: rgw cache enabled = true 2026-02-21T00:05:18.236 INFO:teuthology.orchestra.run.trial138.stdout: rgw enable ops log = true 2026-02-21T00:05:18.236 INFO:teuthology.orchestra.run.trial138.stdout: rgw enable usage log = true 2026-02-21T00:05:18.236 INFO:teuthology.orchestra.run.trial138.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-21T00:05:18.236 INFO:teuthology.orchestra.run.trial138.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-21T00:05:18.236 INFO:teuthology.orchestra.run.trial138.stdout:[mon.a] 2026-02-21T00:05:18.237 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-21T00:05:18.238 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-21T00:05:18.328 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-21T00:05:18.329 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:18.329 DEBUG:teuthology.orchestra.run.trial138:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-21T00:05:18.345 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:18.345 DEBUG:teuthology.orchestra.run.trial138:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-21T00:05:18.402 INFO:tasks.ceph:Sending monmap to node ubuntu@trial138.front.sepia.ceph.com 2026-02-21T00:05:18.402 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:18.402 DEBUG:teuthology.orchestra.run.trial138:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-21T00:05:18.402 DEBUG:teuthology.orchestra.run.trial138:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-21T00:05:18.475 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:18.475 DEBUG:teuthology.orchestra.run.trial138:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:05:18.531 INFO:tasks.ceph:Setting up mon nodes... 2026-02-21T00:05:18.531 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-21T00:05:18.531 DEBUG:teuthology.orchestra.run.trial138:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-21T00:05:18.619 INFO:teuthology.orchestra.run.trial138.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-21T00:05:18.622 INFO:tasks.ceph:Setting up mds nodes... 2026-02-21T00:05:18.622 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-21T00:05:18.623 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-21T00:05:18.664 INFO:teuthology.orchestra.run.trial138.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-21T00:05:18.676 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-21T00:05:18.676 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial138.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-21T00:05:18.676 DEBUG:teuthology.orchestra.run.trial138:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-21T00:05:18.740 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T00:05:18.740 INFO:tasks.ceph:role: osd.0 2026-02-21T00:05:18.740 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial138.front.sepia.ceph.com 2026-02-21T00:05:18.740 DEBUG:teuthology.orchestra.run.trial138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-21T00:05:18.805 INFO:teuthology.orchestra.run.trial138.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T00:05:18.805 INFO:teuthology.orchestra.run.trial138.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T00:05:18.805 INFO:teuthology.orchestra.run.trial138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T00:05:18.805 INFO:teuthology.orchestra.run.trial138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T00:05:18.805 INFO:teuthology.orchestra.run.trial138.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T00:05:18.805 INFO:teuthology.orchestra.run.trial138.stdout: = sunit=0 swidth=0 blks 2026-02-21T00:05:18.805 INFO:teuthology.orchestra.run.trial138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T00:05:18.805 INFO:teuthology.orchestra.run.trial138.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T00:05:18.805 INFO:teuthology.orchestra.run.trial138.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T00:05:18.806 INFO:teuthology.orchestra.run.trial138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T00:05:19.381 INFO:teuthology.orchestra.run.trial138.stdout:Discarding blocks...Done. 2026-02-21T00:05:19.382 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial138.front.sepia.ceph.com -o noatime 2026-02-21T00:05:19.382 DEBUG:teuthology.orchestra.run.trial138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-21T00:05:19.451 DEBUG:teuthology.orchestra.run.trial138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-21T00:05:19.519 DEBUG:teuthology.orchestra.run.trial138:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-21T00:05:19.584 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T00:05:19.585 INFO:tasks.ceph:role: osd.1 2026-02-21T00:05:19.585 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial138.front.sepia.ceph.com 2026-02-21T00:05:19.585 DEBUG:teuthology.orchestra.run.trial138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-21T00:05:19.650 INFO:teuthology.orchestra.run.trial138.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T00:05:19.651 INFO:teuthology.orchestra.run.trial138.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T00:05:19.651 INFO:teuthology.orchestra.run.trial138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T00:05:19.651 INFO:teuthology.orchestra.run.trial138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T00:05:19.651 INFO:teuthology.orchestra.run.trial138.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T00:05:19.651 INFO:teuthology.orchestra.run.trial138.stdout: = sunit=0 swidth=0 blks 2026-02-21T00:05:19.651 INFO:teuthology.orchestra.run.trial138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T00:05:19.651 INFO:teuthology.orchestra.run.trial138.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T00:05:19.651 INFO:teuthology.orchestra.run.trial138.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T00:05:19.651 INFO:teuthology.orchestra.run.trial138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T00:05:20.212 INFO:teuthology.orchestra.run.trial138.stdout:Discarding blocks...Done. 2026-02-21T00:05:20.214 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial138.front.sepia.ceph.com -o noatime 2026-02-21T00:05:20.215 DEBUG:teuthology.orchestra.run.trial138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-21T00:05:20.283 DEBUG:teuthology.orchestra.run.trial138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-21T00:05:20.349 DEBUG:teuthology.orchestra.run.trial138:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-21T00:05:20.415 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-21T00:05:20.415 INFO:tasks.ceph:role: osd.2 2026-02-21T00:05:20.415 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial138.front.sepia.ceph.com 2026-02-21T00:05:20.415 DEBUG:teuthology.orchestra.run.trial138:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-21T00:05:20.479 INFO:teuthology.orchestra.run.trial138.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-21T00:05:20.479 INFO:teuthology.orchestra.run.trial138.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-21T00:05:20.479 INFO:teuthology.orchestra.run.trial138.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-21T00:05:20.479 INFO:teuthology.orchestra.run.trial138.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-21T00:05:20.480 INFO:teuthology.orchestra.run.trial138.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-21T00:05:20.480 INFO:teuthology.orchestra.run.trial138.stdout: = sunit=0 swidth=0 blks 2026-02-21T00:05:20.480 INFO:teuthology.orchestra.run.trial138.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-21T00:05:20.480 INFO:teuthology.orchestra.run.trial138.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-21T00:05:20.480 INFO:teuthology.orchestra.run.trial138.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-21T00:05:20.480 INFO:teuthology.orchestra.run.trial138.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-21T00:05:21.049 INFO:teuthology.orchestra.run.trial138.stdout:Discarding blocks...Done. 2026-02-21T00:05:21.051 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial138.front.sepia.ceph.com -o noatime 2026-02-21T00:05:21.052 DEBUG:teuthology.orchestra.run.trial138:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-21T00:05:21.121 DEBUG:teuthology.orchestra.run.trial138:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-21T00:05:21.189 DEBUG:teuthology.orchestra.run.trial138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:05:21.338 INFO:teuthology.orchestra.run.trial138.stderr:INFO  osd - OSD::main: parsing early config 2026-02-21T00:05:21.394 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:21,393 seastar - Reactor backend: linux-aio 2026-02-21T00:05:21.403 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:21,402 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:05:21.403 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:21,403 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:05:21.405 INFO:teuthology.orchestra.run.trial138.stderr:==39079==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:05:21.465 INFO:teuthology.orchestra.run.trial138.stderr:DEBUG 2026-02-21 00:05:21,464 [shard 0:main] seastar - reactor::drain 2026-02-21T00:05:23.521 INFO:teuthology.orchestra.run.trial138.stderr:----------------------------------------------------- 2026-02-21T00:05:23.522 INFO:teuthology.orchestra.run.trial138.stderr:Suppressions used: 2026-02-21T00:05:23.522 INFO:teuthology.orchestra.run.trial138.stderr: count bytes template 2026-02-21T00:05:23.522 INFO:teuthology.orchestra.run.trial138.stderr: 1 8 MallocExtension::Register 2026-02-21T00:05:23.522 INFO:teuthology.orchestra.run.trial138.stderr:----------------------------------------------------- 2026-02-21T00:05:23.522 INFO:teuthology.orchestra.run.trial138.stderr: 2026-02-21T00:05:23.541 INFO:teuthology.orchestra.run.trial138.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-21T00:05:23.541 INFO:teuthology.orchestra.run.trial138.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-21T00:05:23.582 INFO:teuthology.orchestra.run.trial138.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-21T00:05:23.582 INFO:teuthology.orchestra.run.trial138.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-21T00:05:23.589 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:23,588 seastar - Reactor backend: linux-aio 2026-02-21T00:05:23.604 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:23,604 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:05:23.605 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:23,604 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:05:23.608 INFO:teuthology.orchestra.run.trial138.stderr:==39077==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:05:23.609 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:23,608 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-21T00:05:23.769 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x9ec9639 0xa0a7ce0 0xa0a8052 0xa0a80fe 0xa0d163d 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:23.769 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0xcee22b 0x11680e4 0xf741c5 0x9fe4c31 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5a18 0x9fec227 0x9fec907 0xa0d213c 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:23.770 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0xa0d4d6e 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:23.802 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x11930b /lib64/libasan.so.8+0xebbf /lib64/libasan.so.8+0xec776 0xcee0e1 0x7242c44 0x7eff20c 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:05:23.860 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x157c7 0xcd2ea4 0xce4ede 0xa0d2c7b 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:23.861 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc3d1 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe6ca27 0xa0b3f0a 0xa0b40c6 0xa0b422f 0xa0b435a 0xa0b4884 0xa0d1aff 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:23.861 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x157b4 0x9fe51f5 0x9fe6ec7 0x9fe7414 0x9fe7613 0xa0c7985 0xa0c80e0 0xa0d236e 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:23.973 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191ca6 /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x776e6e7 0x776e786 0x755bbfb 0x763dfc6 0x7efedc0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:23.973 INFO:teuthology.orchestra.run.trial138.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffae9d3e87 0xffffffffae6910d2 0xffffffffaf34d451 0xffffffffaf400ba2 2026-02-21T00:05:23.974 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x157c7 0x10d1d95 0xf741d0 0x9fe4a64 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5f05 0x9fe6403 0x9fe6a33 0x9fe6b9d 0xa0c1f5d 0xa0d4d60 0xa096d59 0xa095d59 0x7efff32 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:23.974 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16b35d /lib64/libasan.so.8+0xb0541 /lib64/libasan.so.8+0xb1437 0xba66a7 0x10d1d1e 0xf741d0 0x9fe4c31 0x9fe4f2e 0x9fe51a1 0x9fe6ec7 0x9fe7414 0x9fe7613 0xa0c7985 0xa0c80e0 0xa0d236e 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:26.178 INFO:teuthology.orchestra.run.trial138.stderr:----------------------------------------------------- 2026-02-21T00:05:26.178 INFO:teuthology.orchestra.run.trial138.stderr:Suppressions used: 2026-02-21T00:05:26.178 INFO:teuthology.orchestra.run.trial138.stderr: count bytes template 2026-02-21T00:05:26.178 INFO:teuthology.orchestra.run.trial138.stderr: 1 8 MallocExtension::Register 2026-02-21T00:05:26.179 INFO:teuthology.orchestra.run.trial138.stderr:----------------------------------------------------- 2026-02-21T00:05:26.179 INFO:teuthology.orchestra.run.trial138.stderr: 2026-02-21T00:05:26.212 DEBUG:teuthology.orchestra.run.trial138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-21T00:05:26.237 DEBUG:teuthology.orchestra.run.trial138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:05:26.389 INFO:teuthology.orchestra.run.trial138.stderr:INFO  osd - OSD::main: parsing early config 2026-02-21T00:05:26.443 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:26,442 seastar - Reactor backend: linux-aio 2026-02-21T00:05:26.455 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:26,455 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:05:26.455 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:26,455 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:05:26.457 INFO:teuthology.orchestra.run.trial138.stderr:==39149==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:05:26.516 INFO:teuthology.orchestra.run.trial138.stderr:DEBUG 2026-02-21 00:05:26,516 [shard 0:main] seastar - reactor::drain 2026-02-21T00:05:28.572 INFO:teuthology.orchestra.run.trial138.stderr:----------------------------------------------------- 2026-02-21T00:05:28.573 INFO:teuthology.orchestra.run.trial138.stderr:Suppressions used: 2026-02-21T00:05:28.573 INFO:teuthology.orchestra.run.trial138.stderr: count bytes template 2026-02-21T00:05:28.573 INFO:teuthology.orchestra.run.trial138.stderr: 1 8 MallocExtension::Register 2026-02-21T00:05:28.573 INFO:teuthology.orchestra.run.trial138.stderr:----------------------------------------------------- 2026-02-21T00:05:28.573 INFO:teuthology.orchestra.run.trial138.stderr: 2026-02-21T00:05:28.592 INFO:teuthology.orchestra.run.trial138.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-21T00:05:28.592 INFO:teuthology.orchestra.run.trial138.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-21T00:05:28.641 INFO:teuthology.orchestra.run.trial138.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-21T00:05:28.641 INFO:teuthology.orchestra.run.trial138.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-21T00:05:28.648 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:28,647 seastar - Reactor backend: linux-aio 2026-02-21T00:05:28.659 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:28,659 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:05:28.659 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:28,659 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:05:28.663 INFO:teuthology.orchestra.run.trial138.stderr:==39147==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:05:28.663 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:28,663 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-21T00:05:28.822 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15865 0x10d1d95 0xf741d0 0xa0c28bd 0xa0c2eaa 0xa0c322d 0xa0d3384 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:28.822 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x10b6c6 0xe6c75a 0xe6cb15 0xe6cd08 0xa0d87e9 0x7bc71f4 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:28.823 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 33 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe6ca27 0x7bc7117 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:28.855 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119134 /lib64/libasan.so.8+0xebbf /lib64/libasan.so.8+0xec776 0xcee0e1 0x7242c44 0x7eff20c 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:05:28.915 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe6ca27 0xa0d86a2 0x7bc71f4 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:28.915 INFO:teuthology.orchestra.run.trial138.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffaed27ef7 0xffffffffae9f7f16 0xffffffffae9fb8d0 0xffffffffae9fcc31 0xffffffffaea24893 0xffffffffaea24a55 0xffffffffae9c97c8 0xffffffffae9d3aeb 0xffffffffae9d3f4b 0xffffffffae6910d2 0xffffffffaf34d451 0xffffffffaf400ba2 2026-02-21T00:05:28.915 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119513 /lib64/libasan.so.8+0xec7bb 0xcee0e1 0xa0d8788 0x7bc71f4 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:28.915 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x10d1d3f 0xf741d0 0xa0c28bd 0xa0c2eaa 0xa0c322d 0xa0d3384 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:29.031 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x116816a 0xf741c5 0x9fe948a 0x9fe9dc9 0xa0d0de9 0xa0d1d09 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:29.031 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc4c5 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe6ca27 0xe6cd7e 0xa0d87e9 0x7bc71f4 0x7effe85 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:29.031 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x10b6c3 0xe6c75a 0xe6cb15 0xa0d4fdc 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:31.221 INFO:teuthology.orchestra.run.trial138.stderr:----------------------------------------------------- 2026-02-21T00:05:31.221 INFO:teuthology.orchestra.run.trial138.stderr:Suppressions used: 2026-02-21T00:05:31.221 INFO:teuthology.orchestra.run.trial138.stderr: count bytes template 2026-02-21T00:05:31.221 INFO:teuthology.orchestra.run.trial138.stderr: 1 8 MallocExtension::Register 2026-02-21T00:05:31.221 INFO:teuthology.orchestra.run.trial138.stderr:----------------------------------------------------- 2026-02-21T00:05:31.222 INFO:teuthology.orchestra.run.trial138.stderr: 2026-02-21T00:05:31.256 DEBUG:teuthology.orchestra.run.trial138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-21T00:05:31.281 DEBUG:teuthology.orchestra.run.trial138:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:05:31.438 INFO:teuthology.orchestra.run.trial138.stderr:INFO  osd - OSD::main: parsing early config 2026-02-21T00:05:31.488 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:31,488 seastar - Reactor backend: linux-aio 2026-02-21T00:05:31.499 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:31,498 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:05:31.499 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:31,498 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:05:31.501 INFO:teuthology.orchestra.run.trial138.stderr:==39216==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:05:31.561 INFO:teuthology.orchestra.run.trial138.stderr:DEBUG 2026-02-21 00:05:31,560 [shard 0:main] seastar - reactor::drain 2026-02-21T00:05:33.616 INFO:teuthology.orchestra.run.trial138.stderr:----------------------------------------------------- 2026-02-21T00:05:33.616 INFO:teuthology.orchestra.run.trial138.stderr:Suppressions used: 2026-02-21T00:05:33.616 INFO:teuthology.orchestra.run.trial138.stderr: count bytes template 2026-02-21T00:05:33.616 INFO:teuthology.orchestra.run.trial138.stderr: 1 8 MallocExtension::Register 2026-02-21T00:05:33.616 INFO:teuthology.orchestra.run.trial138.stderr:----------------------------------------------------- 2026-02-21T00:05:33.617 INFO:teuthology.orchestra.run.trial138.stderr: 2026-02-21T00:05:33.636 INFO:teuthology.orchestra.run.trial138.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-21T00:05:33.636 INFO:teuthology.orchestra.run.trial138.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-21T00:05:33.677 INFO:teuthology.orchestra.run.trial138.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-21T00:05:33.677 INFO:teuthology.orchestra.run.trial138.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-21T00:05:33.684 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:33,683 seastar - Reactor backend: linux-aio 2026-02-21T00:05:33.697 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:33,697 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:05:33.698 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:33,697 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-21T00:05:33.702 INFO:teuthology.orchestra.run.trial138.stderr:==39214==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-21T00:05:33.702 INFO:teuthology.orchestra.run.trial138.stderr:INFO 2026-02-21 00:05:33,701 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-21T00:05:33.861 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x10d1c31 0xf741d0 0x9fe948a 0x9fe9dc9 0xa0d0de9 0xa0d1d09 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:33.861 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1006a8 /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x116816a 0xf741c5 0x9fe4a64 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5f05 0x9fe6403 0x9fe6a33 0x9fe6b9d 0xa0c1f5d 0xa0d4d60 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:33.861 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x9fe51a1 0x9fe5457 0x9fe5f05 0x9fe6403 0x9fe6a33 0x9fe6b9d 0xa0c1f5d 0xa0d4d60 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7272ece 0x7276a51 0x72f0759 0x7747e38 0x755fe18 0x75856b6 0x752dba4 0x72cd3c6 0x7632df7 0x7534172 0x72cd45c 0x76d5b4f 0x7747ec1 0x755ff1a 0x7560012 0x7560098 0x756011d 0x758586e 0x7747f42 0x756019a 0x756021f 0x7585a75 0x76e807e 0x76e8103 0x76e81e1 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:33.893 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x12976b /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x116816a 0xf741c5 0x9fe4c31 0x9fe4f2e 0x9fe51a1 0x9fe5457 0x9fe5a18 0x9fec227 0x9fec907 0xa0d213c 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fbb7e 0x9f82ca3 0x9f834de 0xf2905b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb83e94 2026-02-21T00:05:33.952 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xa28f /lib64/libasan.so.8+0xe8e0 /lib64/libasan.so.8+0xef905 0x3422ab8 0x3422c92 0x3422f49 0x342303e 0x7242ac8 0xa0b73f2 0xa0b760a 0xa0b78dc 0x7efff82 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:33.953 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f 0x11681cc 0xf741c5 0x9fe874e 0x9fe908d 0xa0ce025 0xa0d870b 0x7bc71f4 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:33.953 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1006a8 /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x157c7 0xa0c80ec 0xa0d236e 0xa0d67e7 0xa096d59 0xa095d59 0x7eff0c0 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x7273ba9 0x781c9ac 0x781cb00 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:34.064 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc9dc /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe6ca27 0xe6cd08 0x77b14b5 0x77b15b6 0x763e19c 0x7bc7158 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:34.064 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119106 /lib64/libasan.so.8+0xebbf /lib64/libasan.so.8+0xec776 0xcee0e1 0x773f88f 0x75cad8c 0x74260d4 0x7bc7231 0x7efeff5 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:34.064 INFO:teuthology.orchestra.run.trial138.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa212fcb 0xa0225e9 0xa06f80d 0xa246bc6 0xa246e34 0xa246f7c 0xa24743b /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119114 /lib64/libasan.so.8+0xebbf /lib64/libasan.so.8+0xec776 0xcee0e1 0x773f8b4 0x75cad8c 0x74260d4 0x7bc7231 0x7efed15 0x7f0acff 0x7d0de34 0x7d43927 0x7491f41 0x722bdbf 0x722c803 0x77423ff 0x755df98 0x757efb3 0x752d98e 0x71c93b4 0x76314bf 0x7533731 0x71c944a 0x76d517e 0x7742601 0x755e11e 0x755e216 0x755e29c 0x755e321 0x757f16a 0x7742682 0x755e39e 0x755e423 0x757f371 0x76e5ee1 0x76e5f66 0x76e6044 0x7742802 0x7699988 0x7843fbe 0x787be46 0x76e6130 0x77b2d02 0x78d6d32 0xa25f029 0xa1c540c 0xa28b913 0xa2fa361 0xa2fdc2a 0xa2fe2a6 0x29671c5 0xa91b6b0 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-21T00:05:36.286 INFO:teuthology.orchestra.run.trial138.stderr:----------------------------------------------------- 2026-02-21T00:05:36.286 INFO:teuthology.orchestra.run.trial138.stderr:Suppressions used: 2026-02-21T00:05:36.286 INFO:teuthology.orchestra.run.trial138.stderr: count bytes template 2026-02-21T00:05:36.286 INFO:teuthology.orchestra.run.trial138.stderr: 1 8 MallocExtension::Register 2026-02-21T00:05:36.287 INFO:teuthology.orchestra.run.trial138.stderr:----------------------------------------------------- 2026-02-21T00:05:36.287 INFO:teuthology.orchestra.run.trial138.stderr: 2026-02-21T00:05:36.320 DEBUG:teuthology.orchestra.run.trial138:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-21T00:05:36.345 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-21T00:05:36.345 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:36.345 DEBUG:teuthology.orchestra.run.trial138:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-21T00:05:36.408 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:36.408 DEBUG:teuthology.orchestra.run.trial138:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-21T00:05:36.473 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:36.473 DEBUG:teuthology.orchestra.run.trial138:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-21T00:05:36.536 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:36.536 DEBUG:teuthology.orchestra.run.trial138:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-21T00:05:36.600 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:36.600 DEBUG:teuthology.orchestra.run.trial138:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-21T00:05:36.657 INFO:tasks.ceph:Adding keys to all mons... 2026-02-21T00:05:36.657 DEBUG:teuthology.orchestra.run.trial138:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-21T00:05:36.718 INFO:teuthology.orchestra.run.trial138.stdout:[mgr.x] 2026-02-21T00:05:36.718 INFO:teuthology.orchestra.run.trial138.stdout: key = AQC+9phpE6HqJBAAUqLPwF388EnwBKwhcfY+dA== 2026-02-21T00:05:36.718 INFO:teuthology.orchestra.run.trial138.stdout:[osd.0] 2026-02-21T00:05:36.719 INFO:teuthology.orchestra.run.trial138.stdout: key = AQDD9phpzYtNKBAAnxdIEUdeZCm3CkFTzYQ0Bw== 2026-02-21T00:05:36.719 INFO:teuthology.orchestra.run.trial138.stdout:[osd.1] 2026-02-21T00:05:36.719 INFO:teuthology.orchestra.run.trial138.stdout: key = AQDI9phpTFV3KxAAFaJLHx9KW9QHIU8YTAmV0Q== 2026-02-21T00:05:36.719 INFO:teuthology.orchestra.run.trial138.stdout:[osd.2] 2026-02-21T00:05:36.719 INFO:teuthology.orchestra.run.trial138.stdout: key = AQDN9php/0THLRAArLgk9EqbSsXOqydJsbPNgQ== 2026-02-21T00:05:36.719 INFO:teuthology.orchestra.run.trial138.stdout:[client.0] 2026-02-21T00:05:36.719 INFO:teuthology.orchestra.run.trial138.stdout: key = AQC+9php3gmQJxAAI/432A5aET449m+JM81OMA== 2026-02-21T00:05:36.720 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-21T00:05:36.810 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T00:05:36.858 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T00:05:36.906 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-21T00:05:36.955 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-21T00:05:37.007 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-21T00:05:37.007 DEBUG:teuthology.orchestra.run.trial138:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-21T00:05:37.032 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-21T00:05:37.120 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:37.118+0000 7f833a284c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:37.120 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:37.118+0000 7f833a284c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:37.127 DEBUG:teuthology.orchestra.run.trial138:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-21T00:05:37.152 DEBUG:teuthology.orchestra.run.trial138:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-21T00:05:37.209 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-21T00:05:37.210 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-21T00:05:37.210 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-21T00:05:37.252 INFO:tasks.ceph.mon.a:Started 2026-02-21T00:05:37.252 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-21T00:05:37.253 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-21T00:05:37.253 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-21T00:05:37.255 INFO:tasks.ceph.mgr.x:Started 2026-02-21T00:05:37.255 DEBUG:tasks.ceph:set 0 configs 2026-02-21T00:05:37.255 DEBUG:teuthology.orchestra.run.trial138:> sudo ceph --cluster ceph config dump 2026-02-21T00:05:37.314 INFO:tasks.ceph.mon.a.trial138.stderr:2026-02-21T00:05:37.311+0000 7fa08028ac00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:37.314 INFO:tasks.ceph.mon.a.trial138.stderr:2026-02-21T00:05:37.312+0000 7fa08028ac00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:37.314 INFO:tasks.ceph.mon.a.trial138.stderr:2026-02-21T00:05:37.312+0000 7fa08028ac00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:37.320 INFO:tasks.ceph.mgr.x.trial138.stderr:2026-02-21T00:05:37.318+0000 7f807540bfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:37.330 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:37.326+0000 7f5e1e39c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:37.331 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:37.330+0000 7f5e1e39c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:37.362 INFO:teuthology.orchestra.run.trial138.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-21T00:05:37.370 INFO:tasks.ceph:Setting crush tunables to default 2026-02-21T00:05:37.370 DEBUG:teuthology.orchestra.run.trial138:> sudo ceph --cluster ceph osd crush tunables default 2026-02-21T00:05:37.443 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:37.438+0000 7f48a77b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:37.444 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:37.443+0000 7f48a77b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:37.478 INFO:teuthology.orchestra.run.trial138.stderr:adjusted tunables profile to default 2026-02-21T00:05:37.485 INFO:tasks.ceph:check_enable_crimson: True 2026-02-21T00:05:37.486 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-21T00:05:37.486 DEBUG:teuthology.orchestra.run.trial138:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-21T00:05:37.523 INFO:tasks.ceph.mgr.x.trial138.stderr:2026-02-21T00:05:37.521+0000 7f807540bfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:37.524 INFO:tasks.ceph.mgr.x.trial138.stderr:2026-02-21T00:05:37.522+0000 7f807540bfc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:37.560 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:37.556+0000 7f59f7dfd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:37.561 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:37.560+0000 7f59f7dfd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:37.599 INFO:tasks.ceph.mon.a.trial138.stderr:2026-02-21T00:05:37.597+0000 7fa0779f1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-21T00:05:37.599 INFO:tasks.ceph.mon.a.trial138.stderr:Please be aware that this feature is experimental, untested, 2026-02-21T00:05:37.599 INFO:tasks.ceph.mon.a.trial138.stderr:unsupported, and may result in data corruption, data loss, 2026-02-21T00:05:37.599 INFO:tasks.ceph.mon.a.trial138.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-21T00:05:37.599 INFO:tasks.ceph.mon.a.trial138.stderr:feature with important data. 2026-02-21T00:05:37.599 INFO:tasks.ceph.mon.a.trial138.stderr: 2026-02-21T00:05:38.356 INFO:tasks.ceph.mgr.x.trial138.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-21T00:05:38.357 INFO:tasks.ceph.mgr.x.trial138.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-21T00:05:38.357 INFO:tasks.ceph.mgr.x.trial138.stderr: from numpy import show_config as show_numpy_config 2026-02-21T00:05:38.489 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-21T00:05:38.489 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:38.490 DEBUG:teuthology.orchestra.run.trial138:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-21T00:05:38.554 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:38.555 DEBUG:teuthology.orchestra.run.trial138:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-21T00:05:38.620 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:38.620 DEBUG:teuthology.orchestra.run.trial138:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-21T00:05:38.683 DEBUG:teuthology.orchestra.run.trial138:> sudo ceph --cluster ceph osd new 3ad3ce1b-cfcb-43d1-9fe0-79d84536b8d6 0 2026-02-21T00:05:38.799 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:38.794+0000 7f89e31ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:38.800 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:38.798+0000 7f89e31ad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:38.837 INFO:teuthology.orchestra.run.trial138.stdout:0 2026-02-21T00:05:38.846 DEBUG:teuthology.orchestra.run.trial138:> sudo ceph --cluster ceph osd new 8479ec64-5af4-40dd-a915-48ec331dafe4 1 2026-02-21T00:05:38.921 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:38.916+0000 7facee59c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:38.922 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:38.920+0000 7facee59c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:38.960 INFO:teuthology.orchestra.run.trial138.stdout:1 2026-02-21T00:05:38.969 DEBUG:teuthology.orchestra.run.trial138:> sudo ceph --cluster ceph osd new a05ea9e1-19de-4c46-a865-cf66644c5813 2 2026-02-21T00:05:39.043 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:39.038+0000 7f188a4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:39.045 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:39.043+0000 7f188a4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:39.081 INFO:teuthology.orchestra.run.trial138.stdout:2 2026-02-21T00:05:39.090 DEBUG:teuthology.orchestra.run.trial138:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-21T00:05:39.119 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-21T00:05:39.119 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-21T00:05:39.161 INFO:tasks.ceph.osd.0:Started 2026-02-21T00:05:39.161 DEBUG:teuthology.orchestra.run.trial138:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-21T00:05:39.190 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-21T00:05:39.190 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-21T00:05:39.233 INFO:tasks.ceph.osd.1:Started 2026-02-21T00:05:39.233 DEBUG:teuthology.orchestra.run.trial138:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-21T00:05:39.262 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-21T00:05:39.262 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-21T00:05:39.304 INFO:tasks.ceph.osd.2:Started 2026-02-21T00:05:39.304 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T00:05:39.381 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:39.377+0000 7f57f4e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:39.383 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:39.381+0000 7f57f4e5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:39.416 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:39.417 INFO:teuthology.orchestra.run.trial138.stdout:{"epoch":6,"fsid":"20c293de-c5ff-4972-9b51-1667a3ab6ae9","created":"2026-02-21T00:05:37.323444+0000","modified":"2026-02-21T00:05:39.080264+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T00:05:39.080264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"3ad3ce1b-cfcb-43d1-9fe0-79d84536b8d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8479ec64-5af4-40dd-a915-48ec331dafe4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a05ea9e1-19de-4c46-a865-cf66644c5813","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:05:39.427 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-21T00:05:39.427 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-21T00:05:39.729 DEBUG:teuthology.orchestra.run.trial138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T00:05:39.798 INFO:teuthology.misc.health.trial138.stderr:2026-02-21T00:05:39.794+0000 7f94cdfa4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:39.799 INFO:teuthology.misc.health.trial138.stderr:2026-02-21T00:05:39.797+0000 7f94cdfa4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:39.830 INFO:teuthology.misc.health.trial138.stdout: 2026-02-21T00:05:39.830 INFO:teuthology.misc.health.trial138.stdout:{"epoch":6,"fsid":"20c293de-c5ff-4972-9b51-1667a3ab6ae9","created":"2026-02-21T00:05:37.323444+0000","modified":"2026-02-21T00:05:39.080264+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T00:05:39.080264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"3ad3ce1b-cfcb-43d1-9fe0-79d84536b8d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8479ec64-5af4-40dd-a915-48ec331dafe4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a05ea9e1-19de-4c46-a865-cf66644c5813","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:05:39.836 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-21T00:05:40.581 INFO:tasks.ceph.mgr.x.trial138.stderr:2026-02-21T00:05:40.579+0000 7f7f92a29640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-21T00:05:46.139 DEBUG:teuthology.orchestra.run.trial138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T00:05:46.209 INFO:teuthology.misc.health.trial138.stderr:2026-02-21T00:05:46.206+0000 7f8a364cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:46.211 INFO:teuthology.misc.health.trial138.stderr:2026-02-21T00:05:46.209+0000 7f8a364cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:46.321 INFO:teuthology.misc.health.trial138.stdout: 2026-02-21T00:05:46.321 INFO:teuthology.misc.health.trial138.stdout:{"epoch":7,"fsid":"20c293de-c5ff-4972-9b51-1667a3ab6ae9","created":"2026-02-21T00:05:37.323444+0000","modified":"2026-02-21T00:05:45.592501+0000","last_up_change":"0.000000","last_in_change":"2026-02-21T00:05:39.080264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"3ad3ce1b-cfcb-43d1-9fe0-79d84536b8d6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"8479ec64-5af4-40dd-a915-48ec331dafe4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"a05ea9e1-19de-4c46-a865-cf66644c5813","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:05:46.329 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-21T00:05:52.632 DEBUG:teuthology.orchestra.run.trial138:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-21T00:05:52.700 INFO:teuthology.misc.health.trial138.stderr:2026-02-21T00:05:52.697+0000 7f189ed74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:52.701 INFO:teuthology.misc.health.trial138.stderr:2026-02-21T00:05:52.701+0000 7f189ed74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:52.811 INFO:teuthology.misc.health.trial138.stdout: 2026-02-21T00:05:52.811 INFO:teuthology.misc.health.trial138.stdout:{"epoch":13,"fsid":"20c293de-c5ff-4972-9b51-1667a3ab6ae9","created":"2026-02-21T00:05:37.323444+0000","modified":"2026-02-21T00:05:51.608502+0000","last_up_change":"2026-02-21T00:05:48.599688+0000","last_in_change":"2026-02-21T00:05:39.080264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T00:05:49.591544+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ad3ce1b-cfcb-43d1-9fe0-79d84536b8d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6805","nonce":2049133107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6804","nonce":2049133107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6813","nonce":2049133107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6807","nonce":2049133107}]},"public_addr":"10.20.193.138:6805/2049133107","cluster_addr":"10.20.193.138:6804/2049133107","heartbeat_back_addr":"10.20.193.138:6813/2049133107","heartbeat_front_addr":"10.20.193.138:6807/2049133107","state":["exists","up"]},{"osd":1,"uuid":"8479ec64-5af4-40dd-a915-48ec331dafe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6808","nonce":1064251231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6809","nonce":1064251231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6810","nonce":1064251231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6806","nonce":1064251231}]},"public_addr":"10.20.193.138:6808/1064251231","cluster_addr":"10.20.193.138:6809/1064251231","heartbeat_back_addr":"10.20.193.138:6810/1064251231","heartbeat_front_addr":"10.20.193.138:6806/1064251231","state":["exists","up"]},{"osd":2,"uuid":"a05ea9e1-19de-4c46-a865-cf66644c5813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6803","nonce":2828738185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6802","nonce":2828738185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6812","nonce":2828738185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6811","nonce":2828738185}]},"public_addr":"10.20.193.138:6803/2828738185","cluster_addr":"10.20.193.138:6802/2828738185","heartbeat_back_addr":"10.20.193.138:6812/2828738185","heartbeat_front_addr":"10.20.193.138:6811/2828738185","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:05:52.819 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-21T00:05:52.819 INFO:tasks.ceph:Creating RBD pool 2026-02-21T00:05:52.819 DEBUG:teuthology.orchestra.run.trial138:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-21T00:05:52.894 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:52.888+0000 7f60f4383640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:52.895 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:52.894+0000 7f60f4383640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:53.614 INFO:teuthology.orchestra.run.trial138.stderr:pool 'rbd' created 2026-02-21T00:05:53.627 DEBUG:teuthology.orchestra.run.trial138:> rbd --cluster ceph pool init rbd 2026-02-21T00:05:53.669 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:53.667+0000 7fb50093e240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:53.670 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:53.667+0000 7fb50093e240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:53.671 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:53.669+0000 7fb50093e240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:56.649 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-21T00:05:56.650 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-21T00:05:56.651 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-21T00:05:56.767 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:56.763+0000 7fd2a239c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:56.768 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:56.767+0000 7fd2a239c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:56.878 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:56.887 INFO:teuthology.orchestra.run.trial138.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-21T00:05:56.887 INFO:tasks.ceph_manager:config epoch is 1 2026-02-21T00:05:56.887 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-21T00:05:56.887 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-21T00:05:56.887 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-21T00:05:56.962 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:56.959+0000 7fd9e51fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:56.963 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:56.962+0000 7fd9e51fd640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.106 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:57.114 INFO:teuthology.orchestra.run.trial138.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6800","nonce":921699926},{"type":"v1","addr":"10.20.193.138:6801","nonce":921699926}]},"active_addr":"10.20.193.138:6801/921699926","active_change":"2026-02-21T00:05:39.575309+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.138:0","nonce":2157486666}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.138:0","nonce":2559280993}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.138:0","nonce":866887702}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.138:0","nonce":3807385719}]}]} 2026-02-21T00:05:57.118 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-21T00:05:57.119 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-21T00:05:57.119 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T00:05:57.194 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.190+0000 7f620bb8c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.195 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.193+0000 7f620bb8c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.303 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:57.303 INFO:teuthology.orchestra.run.trial138.stdout:{"epoch":17,"fsid":"20c293de-c5ff-4972-9b51-1667a3ab6ae9","created":"2026-02-21T00:05:37.323444+0000","modified":"2026-02-21T00:05:56.618793+0000","last_up_change":"2026-02-21T00:05:48.599688+0000","last_in_change":"2026-02-21T00:05:39.080264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T00:05:49.591544+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-21T00:05:53.007798+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ad3ce1b-cfcb-43d1-9fe0-79d84536b8d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6805","nonce":2049133107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6804","nonce":2049133107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6813","nonce":2049133107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6807","nonce":2049133107}]},"public_addr":"10.20.193.138:6805/2049133107","cluster_addr":"10.20.193.138:6804/2049133107","heartbeat_back_addr":"10.20.193.138:6813/2049133107","heartbeat_front_addr":"10.20.193.138:6807/2049133107","state":["exists","up"]},{"osd":1,"uuid":"8479ec64-5af4-40dd-a915-48ec331dafe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6808","nonce":1064251231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6809","nonce":1064251231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6810","nonce":1064251231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6806","nonce":1064251231}]},"public_addr":"10.20.193.138:6808/1064251231","cluster_addr":"10.20.193.138:6809/1064251231","heartbeat_back_addr":"10.20.193.138:6810/1064251231","heartbeat_front_addr":"10.20.193.138:6806/1064251231","state":["exists","up"]},{"osd":2,"uuid":"a05ea9e1-19de-4c46-a865-cf66644c5813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6803","nonce":2828738185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6802","nonce":2828738185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6812","nonce":2828738185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6811","nonce":2828738185}]},"public_addr":"10.20.193.138:6803/2828738185","cluster_addr":"10.20.193.138:6802/2828738185","heartbeat_back_addr":"10.20.193.138:6812/2828738185","heartbeat_front_addr":"10.20.193.138:6811/2828738185","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:05:57.312 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-21T00:05:57.313 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-21T00:05:57.390 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.386+0000 7fe722d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.391 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.389+0000 7fe722d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.501 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:57.501 INFO:teuthology.orchestra.run.trial138.stdout:{"epoch":17,"fsid":"20c293de-c5ff-4972-9b51-1667a3ab6ae9","created":"2026-02-21T00:05:37.323444+0000","modified":"2026-02-21T00:05:56.618793+0000","last_up_change":"2026-02-21T00:05:48.599688+0000","last_in_change":"2026-02-21T00:05:39.080264+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-21T00:05:49.591544+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-21T00:05:53.007798+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"3ad3ce1b-cfcb-43d1-9fe0-79d84536b8d6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6805","nonce":2049133107}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6804","nonce":2049133107}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6813","nonce":2049133107}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6807","nonce":2049133107}]},"public_addr":"10.20.193.138:6805/2049133107","cluster_addr":"10.20.193.138:6804/2049133107","heartbeat_back_addr":"10.20.193.138:6813/2049133107","heartbeat_front_addr":"10.20.193.138:6807/2049133107","state":["exists","up"]},{"osd":1,"uuid":"8479ec64-5af4-40dd-a915-48ec331dafe4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6808","nonce":1064251231}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6809","nonce":1064251231}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6810","nonce":1064251231}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6806","nonce":1064251231}]},"public_addr":"10.20.193.138:6808/1064251231","cluster_addr":"10.20.193.138:6809/1064251231","heartbeat_back_addr":"10.20.193.138:6810/1064251231","heartbeat_front_addr":"10.20.193.138:6806/1064251231","state":["exists","up"]},{"osd":2,"uuid":"a05ea9e1-19de-4c46-a865-cf66644c5813","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6803","nonce":2828738185}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6802","nonce":2828738185}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6812","nonce":2828738185}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.138:6811","nonce":2828738185}]},"public_addr":"10.20.193.138:6803/2828738185","cluster_addr":"10.20.193.138:6802/2828738185","heartbeat_back_addr":"10.20.193.138:6812/2828738185","heartbeat_front_addr":"10.20.193.138:6811/2828738185","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-21T00:05:57.511 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-21T00:05:57.511 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-21T00:05:57.512 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-21T00:05:57.593 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.589+0000 7f8d2821a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.594 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.589+0000 7f5b7a50b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.594 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.590+0000 7fa20865a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.595 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.593+0000 7f8d2821a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.595 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.593+0000 7f5b7a50b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.596 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.593+0000 7fa20865a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.609 INFO:teuthology.orchestra.run.trial138.stdout:42949672968 2026-02-21T00:05:57.609 INFO:teuthology.orchestra.run.trial138.stdout:38654705673 2026-02-21T00:05:57.610 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-21T00:05:57.611 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-21T00:05:57.612 INFO:teuthology.orchestra.run.trial138.stdout:38654705673 2026-02-21T00:05:57.612 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-21T00:05:57.732 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.727+0000 7fb7c101a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.732 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.727+0000 7f9b2283d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.733 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.728+0000 7fb21ca06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.733 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.731+0000 7f9b2283d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.733 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.731+0000 7fb7c101a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.734 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.732+0000 7fb21ca06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.848 INFO:teuthology.orchestra.run.trial138.stdout:38654705673 2026-02-21T00:05:57.849 INFO:teuthology.orchestra.run.trial138.stdout:38654705673 2026-02-21T00:05:57.851 INFO:teuthology.orchestra.run.trial138.stdout:42949672968 2026-02-21T00:05:57.860 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705673 got 38654705673 for osd.1 2026-02-21T00:05:57.860 DEBUG:teuthology.parallel:result is None 2026-02-21T00:05:57.861 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705673 got 38654705673 for osd.2 2026-02-21T00:05:57.862 DEBUG:teuthology.parallel:result is None 2026-02-21T00:05:57.862 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672968 got 42949672968 for osd.0 2026-02-21T00:05:57.862 DEBUG:teuthology.parallel:result is None 2026-02-21T00:05:57.863 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-21T00:05:57.863 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T00:05:57.977 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.973+0000 7f52cea3d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:57.978 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:57.976+0000 7f52cea3d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:58.086 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:58.086 INFO:teuthology.orchestra.run.trial138.stderr:dumped all 2026-02-21T00:05:58.094 INFO:teuthology.orchestra.run.trial138.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2026-02-21T00:05:57.602712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62917004,"kb_used_data":62917004,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94369348,"statfs":{"total":161061224448,"available":96634212352,"internally_reserved":0,"allocated":64427012096,"data_stored":64427012096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.633281+0000","last_change":"2026-02-21T00:05:56.638065+0000","last_active":"2026-02-21T00:05:56.633281+0000","last_peered":"2026-02-21T00:05:56.633281+0000","last_clean":"2026-02-21T00:05:56.633281+0000","last_became_active":"2026-02-21T00:05:54.650636+0000","last_became_peered":"2026-02-21T00:05:54.650636+0000","last_unstale":"2026-02-21T00:05:56.633281+0000","last_undegraded":"2026-02-21T00:05:56.633281+0000","last_fullsized":"2026-02-21T00:05:56.633281+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_clean_scrub_stamp":"2026-02-21T00:05:53.611366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.635792+0000","last_change":"2026-02-21T00:05:56.638866+0000","last_active":"2026-02-21T00:05:56.635792+0000","last_peered":"2026-02-21T00:05:56.635792+0000","last_clean":"2026-02-21T00:05:56.635792+0000","last_became_active":"2026-02-21T00:05:54.657567+0000","last_became_peered":"2026-02-21T00:05:54.657567+0000","last_unstale":"2026-02-21T00:05:56.635792+0000","last_undegraded":"2026-02-21T00:05:56.635792+0000","last_fullsized":"2026-02-21T00:05:56.635792+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_clean_scrub_stamp":"2026-02-21T00:05:53.611366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.634945+0000","last_change":"2026-02-21T00:05:56.640208+0000","last_active":"2026-02-21T00:05:56.634945+0000","last_peered":"2026-02-21T00:05:56.634945+0000","last_clean":"2026-02-21T00:05:56.634945+0000","last_became_active":"2026-02-21T00:05:54.654453+0000","last_became_peered":"2026-02-21T00:05:54.654453+0000","last_unstale":"2026-02-21T00:05:56.634945+0000","last_undegraded":"2026-02-21T00:05:56.634945+0000","last_fullsized":"2026-02-21T00:05:56.634945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_clean_scrub_stamp":"2026-02-21T00:05:53.611366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.635887+0000","last_change":"2026-02-21T00:05:56.640229+0000","last_active":"2026-02-21T00:05:56.635887+0000","last_peered":"2026-02-21T00:05:56.635887+0000","last_clean":"2026-02-21T00:05:56.635887+0000","last_became_active":"2026-02-21T00:05:54.662092+0000","last_became_peered":"2026-02-21T00:05:54.662092+0000","last_unstale":"2026-02-21T00:05:56.635887+0000","last_undegraded":"2026-02-21T00:05:56.635887+0000","last_fullsized":"2026-02-21T00:05:56.635887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_clean_scrub_stamp":"2026-02-21T00:05:53.611366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.636131+0000","last_change":"2026-02-21T00:05:56.636131+0000","last_active":"2026-02-21T00:05:56.636131+0000","last_peered":"2026-02-21T00:05:56.636131+0000","last_clean":"2026-02-21T00:05:56.636131+0000","last_became_active":"2026-02-21T00:05:54.655103+0000","last_became_peered":"2026-02-21T00:05:54.655103+0000","last_unstale":"2026-02-21T00:05:56.636131+0000","last_undegraded":"2026-02-21T00:05:56.636131+0000","last_fullsized":"2026-02-21T00:05:56.636131+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_clean_scrub_stamp":"2026-02-21T00:05:53.611366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.633966+0000","last_change":"2026-02-21T00:05:56.636272+0000","last_active":"2026-02-21T00:05:56.633966+0000","last_peered":"2026-02-21T00:05:56.633966+0000","last_clean":"2026-02-21T00:05:56.633966+0000","last_became_active":"2026-02-21T00:05:54.654539+0000","last_became_peered":"2026-02-21T00:05:54.654539+0000","last_unstale":"2026-02-21T00:05:56.633966+0000","last_undegraded":"2026-02-21T00:05:56.633966+0000","last_fullsized":"2026-02-21T00:05:56.633966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_clean_scrub_stamp":"2026-02-21T00:05:53.611366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.631759+0000","last_change":"2026-02-21T00:05:56.634958+0000","last_active":"2026-02-21T00:05:56.631759+0000","last_peered":"2026-02-21T00:05:56.631759+0000","last_clean":"2026-02-21T00:05:56.631759+0000","last_became_active":"2026-02-21T00:05:54.666425+0000","last_became_peered":"2026-02-21T00:05:54.666425+0000","last_unstale":"2026-02-21T00:05:56.631759+0000","last_undegraded":"2026-02-21T00:05:56.631759+0000","last_fullsized":"2026-02-21T00:05:56.631759+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_clean_scrub_stamp":"2026-02-21T00:05:53.611366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.634895+0000","last_change":"2026-02-21T00:05:56.638196+0000","last_active":"2026-02-21T00:05:56.634895+0000","last_peered":"2026-02-21T00:05:56.634895+0000","last_clean":"2026-02-21T00:05:56.634895+0000","last_became_active":"2026-02-21T00:05:54.654954+0000","last_became_peered":"2026-02-21T00:05:54.654954+0000","last_unstale":"2026-02-21T00:05:56.634895+0000","last_undegraded":"2026-02-21T00:05:56.634895+0000","last_fullsized":"2026-02-21T00:05:56.634895+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_clean_scrub_stamp":"2026-02-21T00:05:53.611366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.634083+0000","last_change":"2026-02-21T00:05:50.638379+0000","last_active":"2026-02-21T00:05:56.634083+0000","last_peered":"2026-02-21T00:05:56.634083+0000","last_clean":"2026-02-21T00:05:56.634083+0000","last_became_active":"2026-02-21T00:05:50.631326+0000","last_became_peered":"2026-02-21T00:05:50.631326+0000","last_unstale":"2026-02-21T00:05:56.634083+0000","last_undegraded":"2026-02-21T00:05:56.634083+0000","last_fullsized":"2026-02-21T00:05:56.634083+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:49.600455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:49.600455+0000","last_clean_scrub_stamp":"2026-02-21T00:05:49.600455+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971812,"kb_used_data":20971812,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456972,"statfs":{"total":53687074816,"available":32211939328,"internally_reserved":0,"allocated":21475135488,"data_stored":21475135488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972620,"kb_used_data":20972620,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456164,"statfs":{"total":53687074816,"available":32211111936,"internally_reserved":0,"allocated":21475962880,"data_stored":21475962880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972572,"kb_used_data":20972572,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456212,"statfs":{"total":53687074816,"available":32211161088,"internally_reserved":0,"allocated":21475913728,"data_stored":21475913728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T00:05:58.096 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-21T00:05:58.172 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:58.168+0000 7fa0fd39c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:58.173 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:58.172+0000 7fa0fd39c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:58.281 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:58.282 INFO:teuthology.orchestra.run.trial138.stderr:dumped all 2026-02-21T00:05:58.290 INFO:teuthology.orchestra.run.trial138.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2026-02-21T00:05:57.602712+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62917004,"kb_used_data":62917004,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94369348,"statfs":{"total":161061224448,"available":96634212352,"internally_reserved":0,"allocated":64427012096,"data_stored":64427012096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.633281+0000","last_change":"2026-02-21T00:05:56.638065+0000","last_active":"2026-02-21T00:05:56.633281+0000","last_peered":"2026-02-21T00:05:56.633281+0000","last_clean":"2026-02-21T00:05:56.633281+0000","last_became_active":"2026-02-21T00:05:54.650636+0000","last_became_peered":"2026-02-21T00:05:54.650636+0000","last_unstale":"2026-02-21T00:05:56.633281+0000","last_undegraded":"2026-02-21T00:05:56.633281+0000","last_fullsized":"2026-02-21T00:05:56.633281+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_clean_scrub_stamp":"2026-02-21T00:05:53.611366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.635792+0000","last_change":"2026-02-21T00:05:56.638866+0000","last_active":"2026-02-21T00:05:56.635792+0000","last_peered":"2026-02-21T00:05:56.635792+0000","last_clean":"2026-02-21T00:05:56.635792+0000","last_became_active":"2026-02-21T00:05:54.657567+0000","last_became_peered":"2026-02-21T00:05:54.657567+0000","last_unstale":"2026-02-21T00:05:56.635792+0000","last_undegraded":"2026-02-21T00:05:56.635792+0000","last_fullsized":"2026-02-21T00:05:56.635792+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_clean_scrub_stamp":"2026-02-21T00:05:53.611366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.634945+0000","last_change":"2026-02-21T00:05:56.640208+0000","last_active":"2026-02-21T00:05:56.634945+0000","last_peered":"2026-02-21T00:05:56.634945+0000","last_clean":"2026-02-21T00:05:56.634945+0000","last_became_active":"2026-02-21T00:05:54.654453+0000","last_became_peered":"2026-02-21T00:05:54.654453+0000","last_unstale":"2026-02-21T00:05:56.634945+0000","last_undegraded":"2026-02-21T00:05:56.634945+0000","last_fullsized":"2026-02-21T00:05:56.634945+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_clean_scrub_stamp":"2026-02-21T00:05:53.611366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.635887+0000","last_change":"2026-02-21T00:05:56.640229+0000","last_active":"2026-02-21T00:05:56.635887+0000","last_peered":"2026-02-21T00:05:56.635887+0000","last_clean":"2026-02-21T00:05:56.635887+0000","last_became_active":"2026-02-21T00:05:54.662092+0000","last_became_peered":"2026-02-21T00:05:54.662092+0000","last_unstale":"2026-02-21T00:05:56.635887+0000","last_undegraded":"2026-02-21T00:05:56.635887+0000","last_fullsized":"2026-02-21T00:05:56.635887+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_clean_scrub_stamp":"2026-02-21T00:05:53.611366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"16'1","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.636131+0000","last_change":"2026-02-21T00:05:56.636131+0000","last_active":"2026-02-21T00:05:56.636131+0000","last_peered":"2026-02-21T00:05:56.636131+0000","last_clean":"2026-02-21T00:05:56.636131+0000","last_became_active":"2026-02-21T00:05:54.655103+0000","last_became_peered":"2026-02-21T00:05:54.655103+0000","last_unstale":"2026-02-21T00:05:56.636131+0000","last_undegraded":"2026-02-21T00:05:56.636131+0000","last_fullsized":"2026-02-21T00:05:56.636131+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_clean_scrub_stamp":"2026-02-21T00:05:53.611366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.633966+0000","last_change":"2026-02-21T00:05:56.636272+0000","last_active":"2026-02-21T00:05:56.633966+0000","last_peered":"2026-02-21T00:05:56.633966+0000","last_clean":"2026-02-21T00:05:56.633966+0000","last_became_active":"2026-02-21T00:05:54.654539+0000","last_became_peered":"2026-02-21T00:05:54.654539+0000","last_unstale":"2026-02-21T00:05:56.633966+0000","last_undegraded":"2026-02-21T00:05:56.633966+0000","last_fullsized":"2026-02-21T00:05:56.633966+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_clean_scrub_stamp":"2026-02-21T00:05:53.611366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.631759+0000","last_change":"2026-02-21T00:05:56.634958+0000","last_active":"2026-02-21T00:05:56.631759+0000","last_peered":"2026-02-21T00:05:56.631759+0000","last_clean":"2026-02-21T00:05:56.631759+0000","last_became_active":"2026-02-21T00:05:54.666425+0000","last_became_peered":"2026-02-21T00:05:54.666425+0000","last_unstale":"2026-02-21T00:05:56.631759+0000","last_undegraded":"2026-02-21T00:05:56.631759+0000","last_fullsized":"2026-02-21T00:05:56.631759+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_clean_scrub_stamp":"2026-02-21T00:05:53.611366+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.634895+0000","last_change":"2026-02-21T00:05:56.638196+0000","last_active":"2026-02-21T00:05:56.634895+0000","last_peered":"2026-02-21T00:05:56.634895+0000","last_clean":"2026-02-21T00:05:56.634895+0000","last_became_active":"2026-02-21T00:05:54.654954+0000","last_became_peered":"2026-02-21T00:05:54.654954+0000","last_unstale":"2026-02-21T00:05:56.634895+0000","last_undegraded":"2026-02-21T00:05:56.634895+0000","last_fullsized":"2026-02-21T00:05:56.634895+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:53.611366+0000","last_clean_scrub_stamp":"2026-02-21T00:05:53.611366+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-21T00:05:56.634083+0000","last_change":"2026-02-21T00:05:50.638379+0000","last_active":"2026-02-21T00:05:56.634083+0000","last_peered":"2026-02-21T00:05:56.634083+0000","last_clean":"2026-02-21T00:05:56.634083+0000","last_became_active":"2026-02-21T00:05:50.631326+0000","last_became_peered":"2026-02-21T00:05:50.631326+0000","last_unstale":"2026-02-21T00:05:56.634083+0000","last_undegraded":"2026-02-21T00:05:56.634083+0000","last_fullsized":"2026-02-21T00:05:56.634083+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-21T00:05:49.600455+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-21T00:05:49.600455+0000","last_clean_scrub_stamp":"2026-02-21T00:05:49.600455+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971812,"kb_used_data":20971812,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456972,"statfs":{"total":53687074816,"available":32211939328,"internally_reserved":0,"allocated":21475135488,"data_stored":21475135488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705673,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972620,"kb_used_data":20972620,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456164,"statfs":{"total":53687074816,"available":32211111936,"internally_reserved":0,"allocated":21475962880,"data_stored":21475962880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672968,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972572,"kb_used_data":20972572,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456212,"statfs":{"total":53687074816,"available":32211161088,"internally_reserved":0,"allocated":21475913728,"data_stored":21475913728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-21T00:05:58.292 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-21T00:05:58.292 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-21T00:05:58.292 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-21T00:05:58.292 DEBUG:teuthology.orchestra.run.trial138:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-21T00:05:58.368 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:58.365+0000 7fe0396cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:58.369 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:05:58.367+0000 7fe0396cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:05:58.507 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:05:58.507 INFO:teuthology.orchestra.run.trial138.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-21T00:05:58.516 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-21T00:05:58.516 INFO:teuthology.run_tasks:Running task ssh_keys... 2026-02-21T00:05:58.609 DEBUG:teuthology.orchestra.run.trial138:> echo 'StrictHostKeyChecking no 2026-02-21T00:05:58.609 DEBUG:teuthology.orchestra.run.trial138:> ' > /home/ubuntu/.ssh/config && echo 'UserKnownHostsFile ' /dev/null >> /home/ubuntu/.ssh/config && chmod 600 /home/ubuntu/.ssh/config 2026-02-21T00:05:58.625 INFO:teuthology.task.ssh_keys:generated public key AAAAB3NzaC1yc2EAAAADAQABAAABAQCZFddkngvBYFgVnG9nlkGGC1+3a784mmgyJKG5SOgQRcq5Ioe3JqQjBmh9w2ce0dzviXI0vlC8ayUQBhokrb2p2yZWEiNOPkGucKYsoSoF+PjIzIeKDHxkVY3Jf/uTlEtx4cOuuezrH9q52QuRNWsBiXIj6A47GModdm6Y+l6wfqBv3TPh6IQnTVR9QuHDUI7ysDUl88HhRIAs2QVVUPiQpaABJshS1eOkIXhbSX/e1OyFCOYsG4gFCgcuMLRUcXBzAMX6jmItzHBzx00EldQQRa/HvyKlJxGeSco/EMJuFQrT/V7S65x/eGwzzgLMPjqhSiCR3M1KHAQJ6P0Sjh8B 2026-02-21T00:05:58.625 INFO:teuthology.task.ssh_keys:pushing keys to trial138.front.sepia.ceph.com for ubuntu 2026-02-21T00:05:58.625 DEBUG:teuthology.orchestra.run.trial138:> rm -f -- /home/ubuntu/.ssh/id_rsa 2026-02-21T00:05:58.683 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:58.683 DEBUG:teuthology.orchestra.run.trial138:> dd of=/home/ubuntu/.ssh/id_rsa 2026-02-21T00:05:58.684 DEBUG:teuthology.orchestra.run.trial138:> chmod 0500 /home/ubuntu/.ssh/id_rsa 2026-02-21T00:05:58.740 DEBUG:teuthology.orchestra.run.trial138:> rm -f -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-21T00:05:58.797 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:58.797 DEBUG:teuthology.orchestra.run.trial138:> dd of=/home/ubuntu/.ssh/id_rsa.pub 2026-02-21T00:05:58.854 DEBUG:teuthology.orchestra.run.trial138:> cp -v -a /home/ubuntu/.ssh/authorized_keys /home/ubuntu/.ssh/authorized_keys_2026-02-21_00:05:58:854213 2026-02-21T00:05:58.911 INFO:teuthology.orchestra.run.trial138.stdout:'/home/ubuntu/.ssh/authorized_keys' -> '/home/ubuntu/.ssh/authorized_keys_2026-02-21_00:05:58:854213' 2026-02-21T00:05:58.911 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:58.911 DEBUG:teuthology.orchestra.run.trial138:> dd of=/home/ubuntu/.ssh/authorized_keys conv=notrunc oflag=append 2026-02-21T00:05:58.966 INFO:teuthology.run_tasks:Running task cbt... 2026-02-21T00:05:58.990 INFO:tasks.cbt:cbt configuration is {'cluster': {'user': 'ubuntu', 'head': 'ubuntu@trial138.front.sepia.ceph.com', 'osds': ['ubuntu@trial138.front.sepia.ceph.com'], 'mons': ['ubuntu@trial138.front.sepia.ceph.com'], 'clients': ['ubuntu@trial138.front.sepia.ceph.com'], 'rgws': {'ubuntu@trial138.front.sepia.ceph.com': None}, 'osds_per_node': 3, 'rebuild_every_test': False, 'use_existing': True, 'is_teuthology': True, 'iterations': 1, 'tmp_dir': '/tmp/cbt', 'pool_profiles': {'replicated': {'pg_size': 256, 'pgp_size': 256, 'replication': 'replicated'}}, 'pid_dir': '/var/run/ceph'}, 'benchmarks': {'radosbench': {'concurrent_ops': 4, 'concurrent_procs': 2, 'op_size': [4096], 'pool_monitoring_list': ['collectl'], 'pool_profile': 'replicated', 'run_monitoring_list': ['collectl'], 'time': 60, 'write_only': False}}, 'client_endpoints': None, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}} 2026-02-21T00:05:58.991 DEBUG:teuthology.orchestra.run.trial138:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/cbt 2026-02-21T00:05:59.027 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:05:59.027 DEBUG:teuthology.orchestra.run.trial138:> dd of=/home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2026-02-21T00:05:59.079 DEBUG:teuthology.orchestra.run.trial138:> git clone --depth 1 -b master https://github.com/ceph/cbt.git /home/ubuntu/cephtest/cbt 2026-02-21T00:05:59.141 INFO:teuthology.orchestra.run.trial138.stderr:Cloning into '/home/ubuntu/cephtest/cbt'... 2026-02-21T00:05:59.393 INFO:tasks.cbt:Installing collectl 2026-02-21T00:05:59.393 DEBUG:teuthology.orchestra.run.trial138:> sudo mkdir collectl && cd collectl && sudo wget https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download -O collectl.tar.gz && sudo tar -xvf collectl.tar.gz && cd collectl-4.3.1 && sudo ./INSTALL 2026-02-21T00:05:59.467 INFO:teuthology.orchestra.run.trial138.stderr:--2026-02-21 00:05:59-- https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download 2026-02-21T00:05:59.528 INFO:teuthology.orchestra.run.trial138.stderr:Resolving sourceforge.net (sourceforge.net)... 104.18.12.149, 104.18.13.149, 2606:4700::6812:c95, ... 2026-02-21T00:05:59.531 INFO:teuthology.orchestra.run.trial138.stderr:Connecting to sourceforge.net (sourceforge.net)|104.18.12.149|:443... connected. 2026-02-21T00:05:59.721 INFO:teuthology.orchestra.run.trial138.stderr:HTTP request sent, awaiting response... 302 Found 2026-02-21T00:05:59.721 INFO:teuthology.orchestra.run.trial138.stderr:Location: https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABpmPbn_uTSudAcL1uUtIebK6fSte1I-Uye-2qSeLWTGS4JQhqjzVTjdIcsQefjD6SfDShXkeCkYXM9ZHd6AA7W12u_Jw%3D%3D&use_mirror=cytranet-dal&r= [following] 2026-02-21T00:05:59.722 INFO:teuthology.orchestra.run.trial138.stderr:--2026-02-21 00:05:59-- https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABpmPbn_uTSudAcL1uUtIebK6fSte1I-Uye-2qSeLWTGS4JQhqjzVTjdIcsQefjD6SfDShXkeCkYXM9ZHd6AA7W12u_Jw%3D%3D&use_mirror=cytranet-dal&r= 2026-02-21T00:05:59.726 INFO:teuthology.orchestra.run.trial138.stderr:Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 104.18.12.149, 104.18.13.149, 2606:4700::6812:d95, ... 2026-02-21T00:05:59.729 INFO:teuthology.orchestra.run.trial138.stderr:Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|104.18.12.149|:443... connected. 2026-02-21T00:05:59.906 INFO:teuthology.orchestra.run.trial138.stderr:HTTP request sent, awaiting response... 302 Found 2026-02-21T00:05:59.906 INFO:teuthology.orchestra.run.trial138.stderr:Location: https://cytranet-dal.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 [following] 2026-02-21T00:05:59.906 INFO:teuthology.orchestra.run.trial138.stderr:--2026-02-21 00:05:59-- https://cytranet-dal.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 2026-02-21T00:05:59.912 INFO:teuthology.orchestra.run.trial138.stderr:Resolving cytranet-dal.dl.sourceforge.net (cytranet-dal.dl.sourceforge.net)... 162.226.127.129 2026-02-21T00:05:59.954 INFO:teuthology.orchestra.run.trial138.stderr:Connecting to cytranet-dal.dl.sourceforge.net (cytranet-dal.dl.sourceforge.net)|162.226.127.129|:443... connected. 2026-02-21T00:06:00.165 INFO:teuthology.orchestra.run.trial138.stderr:HTTP request sent, awaiting response... 200 OK 2026-02-21T00:06:00.165 INFO:teuthology.orchestra.run.trial138.stderr:Length: 620693 (606K) [application/x-gzip] 2026-02-21T00:06:00.165 INFO:teuthology.orchestra.run.trial138.stderr:Saving to: ‘collectl.tar.gz’ 2026-02-21T00:06:00.165 INFO:teuthology.orchestra.run.trial138.stderr: 2026-02-21T00:06:00.207 INFO:teuthology.orchestra.run.trial138.stderr: 0K .......... .......... .......... .......... .......... 8% 1.16M 0s 2026-02-21T00:06:00.247 INFO:teuthology.orchestra.run.trial138.stderr: 50K .......... .......... .......... .......... .......... 16% 1.21M 0s 2026-02-21T00:06:00.249 INFO:teuthology.orchestra.run.trial138.stderr: 100K .......... .......... .......... .......... .......... 24% 25.9M 0s 2026-02-21T00:06:00.288 INFO:teuthology.orchestra.run.trial138.stderr: 150K .......... .......... .......... .......... .......... 32% 1.28M 0s 2026-02-21T00:06:00.292 INFO:teuthology.orchestra.run.trial138.stderr: 200K .......... .......... .......... .......... .......... 41% 38.7M 0s 2026-02-21T00:06:00.292 INFO:teuthology.orchestra.run.trial138.stderr: 250K .......... .......... .......... .......... .......... 49% 48.8M 0s 2026-02-21T00:06:00.292 INFO:teuthology.orchestra.run.trial138.stderr: 300K .......... .......... .......... .......... .......... 57% 69.6M 0s 2026-02-21T00:06:00.332 INFO:teuthology.orchestra.run.trial138.stderr: 350K .......... .......... .......... .......... .......... 65% 1.30M 0s 2026-02-21T00:06:00.332 INFO:teuthology.orchestra.run.trial138.stderr: 400K .......... .......... .......... .......... .......... 74% 103M 0s 2026-02-21T00:06:00.332 INFO:teuthology.orchestra.run.trial138.stderr: 450K .......... .......... .......... .......... .......... 82% 64.5M 0s 2026-02-21T00:06:00.333 INFO:teuthology.orchestra.run.trial138.stderr: 500K .......... .......... .......... .......... .......... 90% 63.6M 0s 2026-02-21T00:06:00.333 INFO:teuthology.orchestra.run.trial138.stderr: 550K .......... .......... .......... .......... .......... 98% 49.6M 0s 2026-02-21T00:06:00.333 INFO:teuthology.orchestra.run.trial138.stderr: 600K ...... 100% 11.4T=0.2s 2026-02-21T00:06:00.333 INFO:teuthology.orchestra.run.trial138.stderr: 2026-02-21T00:06:00.333 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21 00:06:00 (3.57 MB/s) - ‘collectl.tar.gz’ saved [620693/620693] 2026-02-21T00:06:00.333 INFO:teuthology.orchestra.run.trial138.stderr: 2026-02-21T00:06:00.354 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/ 2026-02-21T00:06:00.354 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/collectl.conf 2026-02-21T00:06:00.354 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/ 2026-02-21T00:06:00.354 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/SlowProc.html 2026-02-21T00:06:00.354 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/BuddyInfo.html 2026-02-21T00:06:00.355 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/WhySummary.html 2026-02-21T00:06:00.355 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Socket.html 2026-02-21T00:06:00.355 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/colmux.html 2026-02-21T00:06:00.355 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Tutorial.html 2026-02-21T00:06:00.355 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Gexpr.html 2026-02-21T00:06:00.355 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Import.html 2026-02-21T00:06:00.355 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/style.css 2026-02-21T00:06:00.355 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Ganglia3.jpg 2026-02-21T00:06:00.355 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Ganglia2.jpg 2026-02-21T00:06:00.355 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/FileNaming.html 2026-02-21T00:06:00.355 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/TheMath.html 2026-02-21T00:06:00.355 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Environmental.html 2026-02-21T00:06:00.356 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Process.html 2026-02-21T00:06:00.356 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/OutputFiles.html 2026-02-21T00:06:00.356 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/HiResTime.html 2026-02-21T00:06:00.356 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/OutputFormats.html 2026-02-21T00:06:00.356 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/slub.jpg 2026-02-21T00:06:00.356 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Lustre.html 2026-02-21T00:06:00.356 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Documentation.html 2026-02-21T00:06:00.356 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Inodes.html 2026-02-21T00:06:00.356 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Graphite.html 2026-02-21T00:06:00.356 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/InputFiles.html 2026-02-21T00:06:00.356 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Export.html 2026-02-21T00:06:00.356 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Interrupts.html 2026-02-21T00:06:00.356 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Startup.html 2026-02-21T00:06:00.357 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Performance.html 2026-02-21T00:06:00.357 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Messages.html 2026-02-21T00:06:00.357 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Logging.html 2026-02-21T00:06:00.357 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Data-detail.html 2026-02-21T00:06:00.357 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/CPUs.html 2026-02-21T00:06:00.357 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Matrix.html 2026-02-21T00:06:00.357 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Data-brief.html 2026-02-21T00:06:00.357 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/SlabInfo.html 2026-02-21T00:06:00.357 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Misc.html 2026-02-21T00:06:00.357 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Examples.html 2026-02-21T00:06:00.357 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/OpenStack.html 2026-02-21T00:06:00.357 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Plotfiles.html 2026-02-21T00:06:00.357 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/ColmuxCPU.jpg 2026-02-21T00:06:00.358 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Tutorial-Lustre.html 2026-02-21T00:06:00.358 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Data-verbose.html 2026-02-21T00:06:00.358 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Data.html 2026-02-21T00:06:00.358 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Exceptions.html 2026-02-21T00:06:00.358 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/RunningAsAService.html 2026-02-21T00:06:00.358 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/FAQ-collectl.html 2026-02-21T00:06:00.358 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Ganglia.jpg 2026-02-21T00:06:00.358 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/OperationalModes.html 2026-02-21T00:06:00.358 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Playback.html 2026-02-21T00:06:00.358 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Hello.html 2026-02-21T00:06:00.358 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Sockets.html 2026-02-21T00:06:00.358 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Network.html 2026-02-21T00:06:00.358 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Infiniband.html 2026-02-21T00:06:00.359 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Memory.html 2026-02-21T00:06:00.359 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/ColmuxLustre.jpg 2026-02-21T00:06:00.359 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/Disks.html 2026-02-21T00:06:00.359 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/docs/NfsInfo.html 2026-02-21T00:06:00.359 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/service/ 2026-02-21T00:06:00.359 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/service/collectl.service 2026-02-21T00:06:00.359 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/initd/ 2026-02-21T00:06:00.359 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/initd/collectl-suse 2026-02-21T00:06:00.359 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/initd/collectl-debian 2026-02-21T00:06:00.359 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/initd/collectl-generic 2026-02-21T00:06:00.359 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/initd/collectl 2026-02-21T00:06:00.359 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/misc.ph 2026-02-21T00:06:00.359 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/gexpr.ph 2026-02-21T00:06:00.359 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/statsd.ph 2026-02-21T00:06:00.360 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/envrules.std 2026-02-21T00:06:00.360 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/hello.ph 2026-02-21T00:06:00.360 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/vmsum.ph 2026-02-21T00:06:00.360 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/proctree.ph 2026-02-21T00:06:00.360 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/README-WINDOWS 2026-02-21T00:06:00.360 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/formatit.ph 2026-02-21T00:06:00.360 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/colmux 2026-02-21T00:06:00.360 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/INSTALL 2026-02-21T00:06:00.360 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/graphite.ph 2026-02-21T00:06:00.360 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/RELEASE-collectl 2026-02-21T00:06:00.360 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/vmstat.ph 2026-02-21T00:06:00.360 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/README 2026-02-21T00:06:00.360 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/man1/ 2026-02-21T00:06:00.360 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/man1/collectl.1 2026-02-21T00:06:00.361 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/man1/colmux.1 2026-02-21T00:06:00.361 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/lexpr.ph 2026-02-21T00:06:00.361 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/client.pl 2026-02-21T00:06:00.361 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/collectl 2026-02-21T00:06:00.361 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/COPYING 2026-02-21T00:06:00.361 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/vnet.ph 2026-02-21T00:06:00.361 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/GPL 2026-02-21T00:06:00.361 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/UNINSTALL 2026-02-21T00:06:00.361 INFO:teuthology.orchestra.run.trial138.stdout:collectl-4.3.1/ARTISTIC 2026-02-21T00:06:00.398 INFO:teuthology.orchestra.run.trial138.stderr:./INSTALL: line 90: chkconfig: command not found 2026-02-21T00:06:00.400 DEBUG:teuthology.orchestra.run.trial138:> sudo yum -y install librbd-devel pdsh pdsh-rcmd-ssh perf 2026-02-21T00:06:00.755 INFO:teuthology.orchestra.run.trial138.stdout:Last metadata expiration check: 0:00:56 ago on Sat 21 Feb 2026 12:05:04 AM UTC. 2026-02-21T00:06:00.840 INFO:teuthology.orchestra.run.trial138.stdout:Dependencies resolved. 2026-02-21T00:06:00.841 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-21T00:06:00.841 INFO:teuthology.orchestra.run.trial138.stdout: Package Arch Version Repository Size 2026-02-21T00:06:00.841 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-21T00:06:00.841 INFO:teuthology.orchestra.run.trial138.stdout:Installing: 2026-02-21T00:06:00.841 INFO:teuthology.orchestra.run.trial138.stdout: librbd-devel x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 24 k 2026-02-21T00:06:00.841 INFO:teuthology.orchestra.run.trial138.stdout: pdsh x86_64 2.34-7.el9 epel 118 k 2026-02-21T00:06:00.841 INFO:teuthology.orchestra.run.trial138.stdout: pdsh-rcmd-ssh x86_64 2.34-7.el9 epel 13 k 2026-02-21T00:06:00.841 INFO:teuthology.orchestra.run.trial138.stdout: perf x86_64 5.14.0-681.el9 appstream 3.2 M 2026-02-21T00:06:00.842 INFO:teuthology.orchestra.run.trial138.stdout:Installing dependencies: 2026-02-21T00:06:00.842 INFO:teuthology.orchestra.run.trial138.stdout: libradospp-devel x86_64 2:20.3.0-5343.gbcdd429b.el9 ceph 31 k 2026-02-21T00:06:00.842 INFO:teuthology.orchestra.run.trial138.stdout: libtraceevent x86_64 1.8.4-2.el9 baseos 290 k 2026-02-21T00:06:00.842 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:06:00.842 INFO:teuthology.orchestra.run.trial138.stdout:Transaction Summary 2026-02-21T00:06:00.842 INFO:teuthology.orchestra.run.trial138.stdout:================================================================================ 2026-02-21T00:06:00.842 INFO:teuthology.orchestra.run.trial138.stdout:Install 6 Packages 2026-02-21T00:06:00.842 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:06:00.842 INFO:teuthology.orchestra.run.trial138.stdout:Total download size: 3.7 M 2026-02-21T00:06:00.842 INFO:teuthology.orchestra.run.trial138.stdout:Installed size: 14 M 2026-02-21T00:06:00.842 INFO:teuthology.orchestra.run.trial138.stdout:Downloading Packages: 2026-02-21T00:06:01.633 INFO:teuthology.orchestra.run.trial138.stdout:(1/6): librbd-devel-20.3.0-5343.gbcdd429b.el9.x 66 kB/s | 24 kB 00:00 2026-02-21T00:06:01.791 INFO:teuthology.orchestra.run.trial138.stdout:(2/6): libradospp-devel-20.3.0-5343.gbcdd429b.e 60 kB/s | 31 kB 00:00 2026-02-21T00:06:02.115 INFO:teuthology.orchestra.run.trial138.stdout:(3/6): pdsh-2.34-7.el9.x86_64.rpm 366 kB/s | 118 kB 00:00 2026-02-21T00:06:02.181 INFO:teuthology.orchestra.run.trial138.stdout:(4/6): pdsh-rcmd-ssh-2.34-7.el9.x86_64.rpm 200 kB/s | 13 kB 00:00 2026-02-21T00:06:02.215 INFO:teuthology.orchestra.run.trial138.stdout:(5/6): libtraceevent-1.8.4-2.el9.x86_64.rpm 308 kB/s | 290 kB 00:00 2026-02-21T00:06:05.748 INFO:teuthology.orchestra.run.trial138.stdout:(6/6): perf-5.14.0-681.el9.x86_64.rpm 794 kB/s | 3.2 MB 00:04 2026-02-21T00:06:05.750 INFO:teuthology.orchestra.run.trial138.stdout:-------------------------------------------------------------------------------- 2026-02-21T00:06:05.751 INFO:teuthology.orchestra.run.trial138.stdout:Total 762 kB/s | 3.7 MB 00:04 2026-02-21T00:06:05.774 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction check 2026-02-21T00:06:05.785 INFO:teuthology.orchestra.run.trial138.stdout:Transaction check succeeded. 2026-02-21T00:06:05.785 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction test 2026-02-21T00:06:05.841 INFO:teuthology.orchestra.run.trial138.stdout:Transaction test succeeded. 2026-02-21T00:06:05.841 INFO:teuthology.orchestra.run.trial138.stdout:Running transaction 2026-02-21T00:06:05.915 INFO:teuthology.orchestra.run.trial138.stdout: Preparing : 1/1 2026-02-21T00:06:05.921 INFO:teuthology.orchestra.run.trial138.stdout: Installing : pdsh-rcmd-ssh-2.34-7.el9.x86_64 1/6 2026-02-21T00:06:05.962 INFO:teuthology.orchestra.run.trial138.stdout: Installing : pdsh-2.34-7.el9.x86_64 2/6 2026-02-21T00:06:05.966 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libtraceevent-1.8.4-2.el9.x86_64 3/6 2026-02-21T00:06:05.969 INFO:teuthology.orchestra.run.trial138.stdout: Installing : libradospp-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 4/6 2026-02-21T00:06:06.130 INFO:teuthology.orchestra.run.trial138.stdout: Installing : librbd-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 5/6 2026-02-21T00:06:06.152 INFO:teuthology.orchestra.run.trial138.stdout: Installing : perf-5.14.0-681.el9.x86_64 6/6 2026-02-21T00:06:06.266 INFO:teuthology.orchestra.run.trial138.stdout: Running scriptlet: perf-5.14.0-681.el9.x86_64 6/6 2026-02-21T00:06:06.266 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libradospp-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 1/6 2026-02-21T00:06:06.266 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : librbd-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2/6 2026-02-21T00:06:06.266 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : libtraceevent-1.8.4-2.el9.x86_64 3/6 2026-02-21T00:06:06.266 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : perf-5.14.0-681.el9.x86_64 4/6 2026-02-21T00:06:06.266 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : pdsh-2.34-7.el9.x86_64 5/6 2026-02-21T00:06:06.325 INFO:teuthology.orchestra.run.trial138.stdout: Verifying : pdsh-rcmd-ssh-2.34-7.el9.x86_64 6/6 2026-02-21T00:06:06.325 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:06:06.325 INFO:teuthology.orchestra.run.trial138.stdout:Installed: 2026-02-21T00:06:06.326 INFO:teuthology.orchestra.run.trial138.stdout: libradospp-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:06:06.326 INFO:teuthology.orchestra.run.trial138.stdout: librbd-devel-2:20.3.0-5343.gbcdd429b.el9.x86_64 2026-02-21T00:06:06.326 INFO:teuthology.orchestra.run.trial138.stdout: libtraceevent-1.8.4-2.el9.x86_64 2026-02-21T00:06:06.326 INFO:teuthology.orchestra.run.trial138.stdout: pdsh-2.34-7.el9.x86_64 2026-02-21T00:06:06.326 INFO:teuthology.orchestra.run.trial138.stdout: pdsh-rcmd-ssh-2.34-7.el9.x86_64 2026-02-21T00:06:06.326 INFO:teuthology.orchestra.run.trial138.stdout: perf-5.14.0-681.el9.x86_64 2026-02-21T00:06:06.326 INFO:teuthology.orchestra.run.trial138.stdout: 2026-02-21T00:06:06.326 INFO:teuthology.orchestra.run.trial138.stdout:Complete! 2026-02-21T00:06:06.359 DEBUG:teuthology.orchestra.run.trial138:> python3 -m pip install -r /home/ubuntu/cephtest/cbt/requirements.txt 2026-02-21T00:06:06.498 INFO:teuthology.orchestra.run.trial138.stdout:Defaulting to user installation because normal site-packages is not writeable 2026-02-21T00:06:06.508 INFO:teuthology.orchestra.run.trial138.stdout:Requirement already satisfied: pyyaml in /usr/lib64/python3.9/site-packages (from -r /home/ubuntu/cephtest/cbt/requirements.txt (line 1)) (5.4.1) 2026-02-21T00:06:06.969 INFO:teuthology.orchestra.run.trial138.stdout:Collecting lxml 2026-02-21T00:06:06.993 INFO:teuthology.orchestra.run.trial138.stdout: Downloading lxml-6.0.2-cp39-cp39-manylinux_2_26_x86_64.manylinux_2_28_x86_64.whl (5.3 MB) 2026-02-21T00:06:07.386 INFO:teuthology.orchestra.run.trial138.stdout:Collecting matplotlib 2026-02-21T00:06:07.404 INFO:teuthology.orchestra.run.trial138.stdout: Downloading matplotlib-3.9.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.3 MB) 2026-02-21T00:06:07.818 INFO:teuthology.orchestra.run.trial138.stdout:Collecting fonttools>=4.22.0 2026-02-21T00:06:07.824 INFO:teuthology.orchestra.run.trial138.stdout: Downloading fonttools-4.60.2-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.8 MB) 2026-02-21T00:06:07.915 INFO:teuthology.orchestra.run.trial138.stdout:Requirement already satisfied: packaging>=20.0 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (20.9) 2026-02-21T00:06:07.915 INFO:teuthology.orchestra.run.trial138.stdout:Requirement already satisfied: python-dateutil>=2.7 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (2.9.0.post0) 2026-02-21T00:06:07.951 INFO:teuthology.orchestra.run.trial138.stdout:Collecting importlib-resources>=3.2.0 2026-02-21T00:06:07.956 INFO:teuthology.orchestra.run.trial138.stdout: Downloading importlib_resources-6.5.2-py3-none-any.whl (37 kB) 2026-02-21T00:06:08.108 INFO:teuthology.orchestra.run.trial138.stdout:Collecting contourpy>=1.0.1 2026-02-21T00:06:08.118 INFO:teuthology.orchestra.run.trial138.stdout: Downloading contourpy-1.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (321 kB) 2026-02-21T00:06:08.120 INFO:teuthology.orchestra.run.trial138.stdout:Requirement already satisfied: pyparsing>=2.3.1 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (2.4.7) 2026-02-21T00:06:08.272 INFO:teuthology.orchestra.run.trial138.stdout:Collecting kiwisolver>=1.3.1 2026-02-21T00:06:08.282 INFO:teuthology.orchestra.run.trial138.stdout: Downloading kiwisolver-1.4.7-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB) 2026-02-21T00:06:08.295 INFO:teuthology.orchestra.run.trial138.stdout:Requirement already satisfied: numpy>=1.23 in /usr/lib64/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (1.23.5) 2026-02-21T00:06:08.310 INFO:teuthology.orchestra.run.trial138.stdout:Collecting cycler>=0.10 2026-02-21T00:06:08.314 INFO:teuthology.orchestra.run.trial138.stdout: Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB) 2026-02-21T00:06:08.724 INFO:teuthology.orchestra.run.trial138.stdout:Collecting pillow>=8 2026-02-21T00:06:08.731 INFO:teuthology.orchestra.run.trial138.stdout: Downloading pillow-11.3.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (6.6 MB) 2026-02-21T00:06:08.872 INFO:teuthology.orchestra.run.trial138.stdout:Collecting zipp>=3.1.0 2026-02-21T00:06:08.881 INFO:teuthology.orchestra.run.trial138.stdout: Downloading zipp-3.23.0-py3-none-any.whl (10 kB) 2026-02-21T00:06:08.892 INFO:teuthology.orchestra.run.trial138.stdout:Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (1.15.0) 2026-02-21T00:06:09.005 INFO:teuthology.orchestra.run.trial138.stdout:Installing collected packages: zipp, pillow, kiwisolver, importlib-resources, fonttools, cycler, contourpy, matplotlib, lxml 2026-02-21T00:06:10.134 INFO:teuthology.orchestra.run.trial138.stdout:Successfully installed contourpy-1.3.0 cycler-0.12.1 fonttools-4.60.2 importlib-resources-6.5.2 kiwisolver-1.4.7 lxml-6.0.2 matplotlib-3.9.4 pillow-11.3.0 zipp-3.23.0 2026-02-21T00:06:10.254 INFO:tasks.cbt:benchmark: radosbench 2026-02-21T00:06:10.255 DEBUG:teuthology.orchestra.run.trial138:> sudo /sbin/sysctl -q -w kernel.perf_event_paranoid=0 2026-02-21T00:06:10.281 DEBUG:teuthology.orchestra.run.trial138:> /home/ubuntu/cephtest/cbt/cbt.py -a /home/ubuntu/cephtest/archive/cbt /home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2026-02-21T00:06:10.394 INFO:teuthology.orchestra.run.trial138.stderr:00:06:10 - DEBUG - cbt - Settings.cluster: 2026-02-21T00:06:10.394 INFO:teuthology.orchestra.run.trial138.stderr: {'archive_dir': '/home/ubuntu/cephtest/archive/cbt', 2026-02-21T00:06:10.394 INFO:teuthology.orchestra.run.trial138.stderr: 'clients': ['ubuntu@trial138.front.sepia.ceph.com'], 2026-02-21T00:06:10.394 INFO:teuthology.orchestra.run.trial138.stderr: 'conf_file': '/etc/ceph/ceph.conf', 2026-02-21T00:06:10.394 INFO:teuthology.orchestra.run.trial138.stderr: 'head': 'ubuntu@trial138.front.sepia.ceph.com', 2026-02-21T00:06:10.394 INFO:teuthology.orchestra.run.trial138.stderr: 'is_teuthology': True, 2026-02-21T00:06:10.395 INFO:teuthology.orchestra.run.trial138.stderr: 'iterations': 1, 2026-02-21T00:06:10.395 INFO:teuthology.orchestra.run.trial138.stderr: 'mons': ['ubuntu@trial138.front.sepia.ceph.com'], 2026-02-21T00:06:10.395 INFO:teuthology.orchestra.run.trial138.stderr: 'osds': ['ubuntu@trial138.front.sepia.ceph.com'], 2026-02-21T00:06:10.395 INFO:teuthology.orchestra.run.trial138.stderr: 'osds_per_node': 3, 2026-02-21T00:06:10.395 INFO:teuthology.orchestra.run.trial138.stderr: 'pid_dir': '/var/run/ceph', 2026-02-21T00:06:10.395 INFO:teuthology.orchestra.run.trial138.stderr: 'pool_profiles': {'replicated': {'pg_size': 256, 2026-02-21T00:06:10.395 INFO:teuthology.orchestra.run.trial138.stderr: 'pgp_size': 256, 2026-02-21T00:06:10.395 INFO:teuthology.orchestra.run.trial138.stderr: 'replication': 'replicated'}}, 2026-02-21T00:06:10.395 INFO:teuthology.orchestra.run.trial138.stderr: 'rebuild_every_test': False, 2026-02-21T00:06:10.395 INFO:teuthology.orchestra.run.trial138.stderr: 'rgws': {'ubuntu@trial138.front.sepia.ceph.com': None}, 2026-02-21T00:06:10.395 INFO:teuthology.orchestra.run.trial138.stderr: 'tmp_dir': '/tmp/cbt', 2026-02-21T00:06:10.395 INFO:teuthology.orchestra.run.trial138.stderr: 'use_existing': True, 2026-02-21T00:06:10.395 INFO:teuthology.orchestra.run.trial138.stderr: 'user': 'ubuntu'} 2026-02-21T00:06:10.396 INFO:teuthology.orchestra.run.trial138.stderr:00:06:10 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af 2026-02-21T00:06:10.396 INFO:teuthology.orchestra.run.trial138.stderr:00:06:10 - INFO - cbt - Pausing for 60s for idle monitoring. 2026-02-21T00:06:10.396 INFO:teuthology.orchestra.run.trial138.stderr:00:06:10 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/collectl 2026-02-21T00:06:10.567 INFO:teuthology.orchestra.run.trial138.stderr:00:06:10 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-21T00:06:10.569 INFO:teuthology.orchestra.run.trial138.stderr:00:06:10 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/perf 2026-02-21T00:06:10.739 INFO:teuthology.orchestra.run.trial138.stderr:00:06:10 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-21T00:06:10.739 INFO:teuthology.orchestra.run.trial138.stderr:00:06:10 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/perf/perf_stat.${pid} & done 2026-02-21T00:07:10.787 INFO:teuthology.orchestra.run.trial138.stderr:00:07:10 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-21T00:07:10.964 INFO:teuthology.orchestra.run.trial138.stderr:00:07:10 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-21T00:07:11.148 INFO:teuthology.orchestra.run.trial138.stderr:00:07:11 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/perf/perf.data 2026-02-21T00:07:11.150 INFO:teuthology.orchestra.run.trial138.stderr:00:07:11 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/perf/perf_stat.* 2026-02-21T00:07:11.152 INFO:teuthology.orchestra.run.trial138.stderr:00:07:11 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/* 2026-02-21T00:07:11.340 INFO:teuthology.orchestra.run.trial138.stderr:00:07:11 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial138.front.sepia.ceph.com -r /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af 2026-02-21T00:07:11.502 INFO:teuthology.orchestra.run.trial138.stderr:00:07:11 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo killall -9 rados 2026-02-21T00:07:11.678 INFO:teuthology.orchestra.run.trial138.stderr:00:07:11 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af 2026-02-21T00:07:11.678 INFO:teuthology.orchestra.run.trial138.stderr:00:07:11 - INFO - cbt - Skipping existing test in /home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af. 2026-02-21T00:07:11.679 INFO:teuthology.orchestra.run.trial138.stderr:00:07:11 - INFO - cbt - Running benchmark /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004 2026-02-21T00:07:11.679 INFO:teuthology.orchestra.run.trial138.stderr:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af 2026-02-21T00:07:11.679 INFO:teuthology.orchestra.run.trial138.stderr:{'concurrent_ops': 4, 'concurrent_procs': 2, 'pool_profile': 'replicated', 'time': 60, 'write_only': False, 'op_size': 4096, 'pool_monitoring_list': 'collectl', 'run_monitoring_list': 'collectl', 'benchmark': 'radosbench', 'iteration': 0} == iteration 0 == 2026-02-21T00:07:11.679 INFO:teuthology.orchestra.run.trial138.stderr:00:07:11 - INFO - cbt - Setting OSD Read Ahead to: 0 2026-02-21T00:07:11.679 INFO:teuthology.orchestra.run.trial138.stderr:00:07:11 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com find /dev/disk/by-partlabel/osd-device-*data -exec readlink {} \; | cut -d"/" -f 3 | sed "s/[0-9]$//" | xargs -I{} sudo sh -c "echo 0 > /sys/block/'{}'/queue/read_ahead_kb" 2026-02-21T00:07:11.680 INFO:teuthology.orchestra.run.trial138.stderr:00:07:11 - DEBUG - cbt - Cleaning existing temporary run directory: /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004 2026-02-21T00:07:11.680 INFO:teuthology.orchestra.run.trial138.stderr:00:07:11 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo rm -rf /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004 2026-02-21T00:07:11.852 INFO:teuthology.orchestra.run.trial138.stderr:00:07:11 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/collectl 2026-02-21T00:07:12.013 INFO:teuthology.orchestra.run.trial138.stderr:00:07:12 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-21T00:07:12.014 INFO:teuthology.orchestra.run.trial138.stderr:00:07:12 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/perf 2026-02-21T00:07:12.182 INFO:teuthology.orchestra.run.trial138.stderr:00:07:12 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-21T00:07:12.182 INFO:teuthology.orchestra.run.trial138.stderr:00:07:12 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/perf/perf_stat.${pid} & done 2026-02-21T00:07:12.184 INFO:teuthology.orchestra.run.trial138.stderr:00:07:12 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool delete rados-bench-cbt rados-bench-cbt --yes-i-really-really-mean-it 2026-02-21T00:07:12.525 INFO:teuthology.orchestra.run.trial138.stderr:00:07:12 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool create rados-bench-cbt 256 256 2026-02-21T00:07:13.680 INFO:teuthology.orchestra.run.trial138.stderr:00:07:13 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool application enable rados-bench-cbt radosbench 2026-02-21T00:07:14.683 INFO:teuthology.orchestra.run.trial138.stderr:00:07:14 - INFO - cbt - Checking Health after pool creation. 2026-02-21T00:07:19.688 INFO:teuthology.orchestra.run.trial138.stderr:00:07:19 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com /usr/bin/ceph -c /etc/ceph/ceph.conf health 2026-02-21T00:07:20.042 INFO:teuthology.orchestra.run.trial138.stderr:00:07:20 - INFO - cbt - Final Pool Health Check. 2026-02-21T00:07:25.047 INFO:teuthology.orchestra.run.trial138.stderr:00:07:25 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com /usr/bin/ceph -c /etc/ceph/ceph.conf health 2026-02-21T00:07:25.407 INFO:teuthology.orchestra.run.trial138.stderr:00:07:25 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-21T00:07:25.575 INFO:teuthology.orchestra.run.trial138.stderr:00:07:25 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-21T00:07:25.752 INFO:teuthology.orchestra.run.trial138.stderr:00:07:25 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/perf/perf.data 2026-02-21T00:07:25.754 INFO:teuthology.orchestra.run.trial138.stderr:00:07:25 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/perf/perf_stat.* 2026-02-21T00:07:25.756 INFO:teuthology.orchestra.run.trial138.stderr:00:07:25 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sync 2026-02-21T00:07:26.008 INFO:teuthology.orchestra.run.trial138.stderr:00:07:26 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com echo 3 | sudo tee /proc/sys/vm/drop_caches 2026-02-21T00:07:26.983 INFO:teuthology.orchestra.run.trial138.stderr:00:07:26 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -v 2026-02-21T00:07:27.196 INFO:teuthology.orchestra.run.trial138.stderr:00:07:27 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write 2026-02-21T00:07:27.361 INFO:teuthology.orchestra.run.trial138.stderr:00:07:27 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf daemon osd.0 config show > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/ceph_settings.out 2026-02-21T00:07:27.657 INFO:teuthology.orchestra.run.trial138.stderr:00:07:27 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/collectl 2026-02-21T00:07:27.819 INFO:teuthology.orchestra.run.trial138.stderr:00:07:27 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-21T00:07:27.820 INFO:teuthology.orchestra.run.trial138.stderr:00:07:27 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/perf 2026-02-21T00:07:27.979 INFO:teuthology.orchestra.run.trial138.stderr:00:07:27 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-21T00:07:27.979 INFO:teuthology.orchestra.run.trial138.stderr:00:07:27 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/perf/perf_stat.${pid} & done 2026-02-21T00:07:27.981 INFO:teuthology.orchestra.run.trial138.stderr:00:07:27 - INFO - cbt - Running radosbench write test. 2026-02-21T00:07:27.981 INFO:teuthology.orchestra.run.trial138.stderr:00:07:27 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench -b 4096 60 write --concurrent-ios 4 --run-name `hostname -f`-0 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/objecter.0.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/output.0 2026-02-21T00:07:27.983 INFO:teuthology.orchestra.run.trial138.stderr:00:07:27 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench -b 4096 60 write --concurrent-ios 4 --run-name `hostname -f`-1 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/objecter.1.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/output.1 2026-02-21T00:08:29.197 INFO:teuthology.orchestra.run.trial138.stderr:00:08:29 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-21T00:08:29.387 INFO:teuthology.orchestra.run.trial138.stderr:00:08:29 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-21T00:08:29.575 INFO:teuthology.orchestra.run.trial138.stderr:00:08:29 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/perf/perf.data 2026-02-21T00:08:29.576 INFO:teuthology.orchestra.run.trial138.stderr:00:08:29 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/perf/perf_stat.* 2026-02-21T00:08:29.578 INFO:teuthology.orchestra.run.trial138.stderr:00:08:29 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com find /var/run/ceph/ceph-osd*.asok -maxdepth 1 -exec sudo /usr/bin/ceph --admin-daemon {} dump_historic_ops \; > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/historic_ops.out 2026-02-21T00:08:29.943 INFO:teuthology.orchestra.run.trial138.stderr:00:08:29 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/* 2026-02-21T00:08:30.121 INFO:teuthology.orchestra.run.trial138.stderr:00:08:30 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial138.front.sepia.ceph.com -r /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af/write 2026-02-21T00:08:30.286 INFO:teuthology.orchestra.run.trial138.stderr:00:08:30 - INFO - cbt - Convert results to json format. 2026-02-21T00:08:30.287 INFO:teuthology.orchestra.run.trial138.stderr:00:08:30 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sync 2026-02-21T00:08:30.448 INFO:teuthology.orchestra.run.trial138.stderr:00:08:30 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com echo 3 | sudo tee /proc/sys/vm/drop_caches 2026-02-21T00:08:30.941 INFO:teuthology.orchestra.run.trial138.stderr:00:08:30 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -v 2026-02-21T00:08:31.150 INFO:teuthology.orchestra.run.trial138.stderr:00:08:31 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/seq 2026-02-21T00:08:31.317 INFO:teuthology.orchestra.run.trial138.stderr:00:08:31 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf daemon osd.0 config show > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/seq/ceph_settings.out 2026-02-21T00:08:31.603 INFO:teuthology.orchestra.run.trial138.stderr:00:08:31 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/seq/collectl 2026-02-21T00:08:31.768 INFO:teuthology.orchestra.run.trial138.stderr:00:08:31 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/seq/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-21T00:08:31.770 INFO:teuthology.orchestra.run.trial138.stderr:00:08:31 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/seq/perf 2026-02-21T00:08:31.939 INFO:teuthology.orchestra.run.trial138.stderr:00:08:31 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-21T00:08:31.939 INFO:teuthology.orchestra.run.trial138.stderr:00:08:31 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/seq/perf/perf_stat.${pid} & done 2026-02-21T00:08:31.940 INFO:teuthology.orchestra.run.trial138.stderr:00:08:31 - INFO - cbt - Running radosbench seq test. 2026-02-21T00:08:31.941 INFO:teuthology.orchestra.run.trial138.stderr:00:08:31 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench 60 seq --concurrent-ios 4 --run-name `hostname -f`-0 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/seq/objecter.0.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/seq/output.0 2026-02-21T00:08:31.942 INFO:teuthology.orchestra.run.trial138.stderr:00:08:31 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench 60 seq --concurrent-ios 4 --run-name `hostname -f`-1 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/seq/objecter.1.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/seq/output.1 2026-02-21T00:08:48.145 INFO:teuthology.orchestra.run.trial138.stderr:00:08:48 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-21T00:08:48.317 INFO:teuthology.orchestra.run.trial138.stderr:00:08:48 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-21T00:08:48.495 INFO:teuthology.orchestra.run.trial138.stderr:00:08:48 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/seq/perf/perf.data 2026-02-21T00:08:48.497 INFO:teuthology.orchestra.run.trial138.stderr:00:08:48 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/seq/perf/perf_stat.* 2026-02-21T00:08:48.498 INFO:teuthology.orchestra.run.trial138.stderr:00:08:48 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com find /var/run/ceph/ceph-osd*.asok -maxdepth 1 -exec sudo /usr/bin/ceph --admin-daemon {} dump_historic_ops \; > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/seq/historic_ops.out 2026-02-21T00:08:48.852 INFO:teuthology.orchestra.run.trial138.stderr:00:08:48 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial138.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/seq/* 2026-02-21T00:08:49.023 INFO:teuthology.orchestra.run.trial138.stderr:00:08:49 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial138.front.sepia.ceph.com -r /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/seq/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af/seq 2026-02-21T00:08:49.185 INFO:teuthology.orchestra.run.trial138.stderr:00:08:49 - INFO - cbt - Convert results to json format. 2026-02-21T00:08:49.198 DEBUG:teuthology.run_tasks:Unwinding manager cbt 2026-02-21T00:08:49.217 DEBUG:teuthology.orchestra.run.trial138:> rm --one-file-system -rf -- /home/ubuntu/cephtest/cbt 2026-02-21T00:08:49.244 INFO:tasks.cbt_performance:Collecting CBT performance data config 2026-02-21T00:08:49.244 INFO:tasks.cbt_performance:reading cbt preferences 2026-02-21T00:08:49.244 DEBUG:teuthology.orchestra.run.trial138:> find /home/ubuntu/cephtest -name 'json_output.*' 2026-02-21T00:08:49.301 INFO:teuthology.orchestra.run.trial138.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af/write/json_output.1.trial138.front.sepia.ceph.com 2026-02-21T00:08:49.302 INFO:teuthology.orchestra.run.trial138.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af/write/json_output.0.trial138.front.sepia.ceph.com 2026-02-21T00:08:49.302 INFO:teuthology.orchestra.run.trial138.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af/seq/json_output.1.trial138.front.sepia.ceph.com 2026-02-21T00:08:49.302 INFO:teuthology.orchestra.run.trial138.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af/seq/json_output.0.trial138.front.sepia.ceph.com 2026-02-21T00:08:49.303 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:08:49.303 DEBUG:teuthology.orchestra.run.trial138:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af/write/json_output.1.trial138.front.sepia.ceph.com of=/dev/stdout 2026-02-21T00:08:49.359 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-21T00:08:49.359 DEBUG:teuthology.orchestra.run.trial138:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af/write -name 'perf_stat.*' 2026-02-21T00:08:49.415 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-21T00:08:49.415 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-21T00:08:49.415 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:08:49.415 DEBUG:teuthology.orchestra.run.trial138:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af/write/json_output.0.trial138.front.sepia.ceph.com of=/dev/stdout 2026-02-21T00:08:49.471 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-21T00:08:49.471 DEBUG:teuthology.orchestra.run.trial138:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af/write -name 'perf_stat.*' 2026-02-21T00:08:49.527 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-21T00:08:49.527 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-21T00:08:49.527 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:08:49.528 DEBUG:teuthology.orchestra.run.trial138:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af/seq/json_output.1.trial138.front.sepia.ceph.com of=/dev/stdout 2026-02-21T00:08:49.584 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-21T00:08:49.585 DEBUG:teuthology.orchestra.run.trial138:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af/seq -name 'perf_stat.*' 2026-02-21T00:08:49.641 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-21T00:08:49.641 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-21T00:08:49.642 DEBUG:teuthology.orchestra.run.trial138:> set -ex 2026-02-21T00:08:49.642 DEBUG:teuthology.orchestra.run.trial138:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af/seq/json_output.0.trial138.front.sepia.ceph.com of=/dev/stdout 2026-02-21T00:08:49.697 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-21T00:08:49.697 DEBUG:teuthology.orchestra.run.trial138:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-31cd68af/seq -name 'perf_stat.*' 2026-02-21T00:08:49.752 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-21T00:08:49.753 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-21T00:08:49.757 WARNING:tasks.cbt_performance:Could not send performance data to http://mira118.front.sepia.ceph.com:4000/cbt_performance: HTTPConnectionPool(host='mira118.front.sepia.ceph.com', port=4000): Max retries exceeded with url: /cbt_performance (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known')) 2026-02-21T00:08:49.757 WARNING:tasks.cbt_performance:Continuing without performance data collection 2026-02-21T00:08:49.758 DEBUG:teuthology.run_tasks:Unwinding manager ssh_keys 2026-02-21T00:08:49.772 INFO:teuthology.task.ssh_keys:Cleaning up SSH keys 2026-02-21T00:08:49.772 INFO:teuthology.task.ssh_keys:cleaning up keys added for testing 2026-02-21T00:08:49.773 INFO:teuthology.task.ssh_keys: cleaning up keys for user ubuntu on trial138.front.sepia.ceph.com 2026-02-21T00:08:49.773 DEBUG:teuthology.orchestra.run.trial138:> rm -- /home/ubuntu/.ssh/id_rsa 2026-02-21T00:08:49.809 DEBUG:teuthology.orchestra.run.trial138:> rm -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-21T00:08:49.865 DEBUG:teuthology.orchestra.run.trial138:> stat -c '"%a"' /home/ubuntu/.ssh/authorized_keys 2026-02-21T00:08:49.922 INFO:teuthology.orchestra.run.trial138.stdout:"600" 2026-02-21T00:08:49.922 DEBUG:teuthology.orchestra.run.trial138:> mv -- /home/ubuntu/.ssh/authorized_keys_2026-02-21_00:05:58:854213 /home/ubuntu/.ssh/authorized_keys 2026-02-21T00:08:49.979 DEBUG:teuthology.orchestra.run.trial138:> chmod 600 /home/ubuntu/.ssh/authorized_keys 2026-02-21T00:08:50.035 DEBUG:teuthology.orchestra.run.trial138:> rm /home/ubuntu/.ssh/config 2026-02-21T00:08:50.092 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-21T00:08:50.107 DEBUG:teuthology.orchestra.run.trial138:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-21T00:08:50.209 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:08:50.203+0000 7ff7517dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:08:50.210 INFO:teuthology.orchestra.run.trial138.stderr:2026-02-21T00:08:50.208+0000 7ff7517dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-21T00:08:50.335 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-21T00:08:50.335 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-21T00:08:50.335 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-21T00:08:50.335 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:08:50.411 INFO:tasks.ceph.osd.0:Stopped 2026-02-21T00:08:50.411 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-21T00:08:50.411 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:08:50.502 INFO:tasks.ceph.osd.1:Stopped 2026-02-21T00:08:50.502 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-21T00:08:50.502 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:08:50.599 INFO:tasks.ceph.osd.2:Stopped 2026-02-21T00:08:50.600 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-21T00:08:50.600 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-21T00:08:50.600 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:08:50.633 INFO:tasks.ceph.mgr.x:Stopped 2026-02-21T00:08:50.633 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-21T00:08:50.633 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-21T00:08:50.633 INFO:teuthology.orchestra.run:waiting for 300 2026-02-21T00:08:50.685 INFO:tasks.ceph.mon.a:Stopped 2026-02-21T00:08:50.685 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-21T00:08:50.686 DEBUG:teuthology.orchestra.run.trial138:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v '\(PG_' | grep -E -v '\(OSD_' | grep -E -v '\(OBJECT_' | grep -E -v 'overall HEALTH' | head -n 1 2026-02-21T00:08:50.751 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial138.front.sepia.ceph.com 2026-02-21T00:08:50.751 DEBUG:teuthology.orchestra.run.trial138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-21T00:08:51.025 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial138.front.sepia.ceph.com 2026-02-21T00:08:51.026 DEBUG:teuthology.orchestra.run.trial138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-21T00:08:51.055 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial138.front.sepia.ceph.com 2026-02-21T00:08:51.056 DEBUG:teuthology.orchestra.run.trial138:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-21T00:08:51.125 INFO:tasks.ceph:Archiving mon data... 2026-02-21T00:08:51.126 DEBUG:teuthology.misc:Transferring archived files from trial138:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61909/data/mon.a.tgz 2026-02-21T00:08:51.126 DEBUG:teuthology.orchestra.run.trial138:> mktemp 2026-02-21T00:08:51.181 INFO:teuthology.orchestra.run.trial138.stdout:/tmp/tmp.kSGWU4qrsX 2026-02-21T00:08:51.182 DEBUG:teuthology.orchestra.run.trial138:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.kSGWU4qrsX 2026-02-21T00:08:51.271 DEBUG:teuthology.orchestra.run.trial138:> sudo chmod 0666 /tmp/tmp.kSGWU4qrsX 2026-02-21T00:08:51.353 DEBUG:teuthology.orchestra.remote:trial138:/tmp/tmp.kSGWU4qrsX is 198KB 2026-02-21T00:08:51.405 DEBUG:teuthology.orchestra.run.trial138:> rm -fr /tmp/tmp.kSGWU4qrsX 2026-02-21T00:08:51.421 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-21T00:08:51.421 DEBUG:teuthology.orchestra.run.trial138:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-21T00:08:51.542 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-21T00:08:51.542 INFO:tasks.ceph:Archiving crash dumps... 2026-02-21T00:08:51.544 DEBUG:teuthology.misc:Transferring archived files from trial138:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61909/remote/trial138/crash 2026-02-21T00:08:51.545 DEBUG:teuthology.orchestra.run.trial138:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-21T00:08:51.605 INFO:tasks.ceph:Compressing logs... 2026-02-21T00:08:51.606 DEBUG:teuthology.orchestra.run.trial138:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T00:08:51.671 INFO:teuthology.orchestra.run.trial138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-21T00:08:51.672 INFO:teuthology.orchestra.run.trial138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46166.log 2026-02-21T00:08:51.672 INFO:teuthology.orchestra.run.trial138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41130.log 2026-02-21T00:08:51.672 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-client.admin.46166.log: gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-21T00:08:51.672 INFO:teuthology.orchestra.run.trial138.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46166.log.gz 2026-02-21T00:08:51.672 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.41130.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40984.log 2026-02-21T00:08:51.672 INFO:teuthology.orchestra.run.trial138.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41130.log.gz 2026-02-21T00:08:51.672 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-21T00:08:51.672 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.40984.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40017.log 2026-02-21T00:08:51.672 INFO:teuthology.orchestra.run.trial138.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40984.log.gz 2026-02-21T00:08:51.672 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41322.log 2026-02-21T00:08:51.683 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.40017.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40111.log 2026-02-21T00:08:51.683 INFO:teuthology.orchestra.run.trial138.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-21T00:08:51.683 INFO:teuthology.orchestra.run.trial138.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40017.log.gz 2026-02-21T00:08:51.683 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.41322.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44191.log 2026-02-21T00:08:51.683 INFO:teuthology.orchestra.run.trial138.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41322.log.gz 2026-02-21T00:08:51.684 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.40111.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44036.log 2026-02-21T00:08:51.684 INFO:teuthology.orchestra.run.trial138.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40111.log.gz 2026-02-21T00:08:51.684 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.44191.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45486.log 2026-02-21T00:08:51.684 INFO:teuthology.orchestra.run.trial138.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44191.log.gz 2026-02-21T00:08:51.684 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.44036.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40982.log 2026-02-21T00:08:51.684 INFO:teuthology.orchestra.run.trial138.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44036.log.gz 2026-02-21T00:08:51.684 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.45486.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40644.log 2026-02-21T00:08:51.684 INFO:teuthology.orchestra.run.trial138.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.45486.log.gz 2026-02-21T00:08:51.684 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.40982.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41131.log 2026-02-21T00:08:51.684 INFO:teuthology.orchestra.run.trial138.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40982.log.gz 2026-02-21T00:08:51.684 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.40644.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40983.log 2026-02-21T00:08:51.684 INFO:teuthology.orchestra.run.trial138.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40644.log.gz 2026-02-21T00:08:51.685 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.41131.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40734.log 2026-02-21T00:08:51.685 INFO:teuthology.orchestra.run.trial138.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41131.log.gz 2026-02-21T00:08:51.685 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.40983.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40064.log 2026-02-21T00:08:51.685 INFO:teuthology.orchestra.run.trial138.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40983.log.gz 2026-02-21T00:08:51.685 INFO:teuthology.orchestra.run.trial138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44803.log 2026-02-21T00:08:51.685 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.40734.log: /var/log/ceph/ceph-client.admin.40064.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40734.log.gz 2026-02-21T00:08:51.685 INFO:teuthology.orchestra.run.trial138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39880.log 2026-02-21T00:08:51.685 INFO:teuthology.orchestra.run.trial138.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40064.log.gz 2026-02-21T00:08:51.685 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.44803.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45489.log 2026-02-21T00:08:51.685 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.39880.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44245.log 2026-02-21T00:08:51.685 INFO:teuthology.orchestra.run.trial138.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.44803.log.gz 2026-02-21T00:08:51.685 INFO:teuthology.orchestra.run.trial138.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39880.log.gz 2026-02-21T00:08:51.685 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.45489.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-21T00:08:51.686 INFO:teuthology.orchestra.run.trial138.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.45489.log.gz 2026-02-21T00:08:51.686 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.44245.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39738.log 2026-02-21T00:08:51.686 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- 50.0% /var/log/ceph/ceph-osd.1.log -- replaced with /var/log/ceph/ceph-client.admin.44245.log.gz 2026-02-21T00:08:51.686 INFO:teuthology.orchestra.run.trial138.stderr: 2026-02-21T00:08:51.686 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.39738.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.38444.log 2026-02-21T00:08:51.686 INFO:teuthology.orchestra.run.trial138.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39738.log.gz 2026-02-21T00:08:51.686 INFO:teuthology.orchestra.run.trial138.stderr: 87.6%/var/log/ceph/ceph-osd.1.log: -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-21T00:08:51.686 INFO:teuthology.orchestra.run.trial138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41224.log 2026-02-21T00:08:51.686 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph.tmp-client.admin.38444.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40330.log 2026-02-21T00:08:51.686 INFO:teuthology.orchestra.run.trial138.stderr: 68.0% -- replaced with /var/log/ceph/ceph.tmp-client.admin.38444.log.gz 2026-02-21T00:08:51.686 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.41224.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44151.log 2026-02-21T00:08:51.686 INFO:teuthology.orchestra.run.trial138.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41224.log.gz 2026-02-21T00:08:51.687 INFO:teuthology.orchestra.run.trial138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41273.log 2026-02-21T00:08:51.687 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.40330.log: /var/log/ceph/ceph-client.admin.44151.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40330.log.gz 2026-02-21T00:08:51.687 INFO:teuthology.orchestra.run.trial138.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39833.log 2026-02-21T00:08:51.687 INFO:teuthology.orchestra.run.trial138.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44151.log.gz 2026-02-21T00:08:51.687 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.41273.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40438.log 2026-02-21T00:08:51.687 INFO:teuthology.orchestra.run.trial138.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41273.log.gz 2026-02-21T00:08:51.687 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.39833.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41129.log 2026-02-21T00:08:51.687 INFO:teuthology.orchestra.run.trial138.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39833.log.gz 2026-02-21T00:08:51.687 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.40438.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-21T00:08:51.687 INFO:teuthology.orchestra.run.trial138.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40438.log.gz 2026-02-21T00:08:51.688 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.41129.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40881.log 2026-02-21T00:08:51.688 INFO:teuthology.orchestra.run.trial138.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41129.log.gz 2026-02-21T00:08:51.688 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40507.log 2026-02-21T00:08:51.688 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.40881.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44799.log 2026-02-21T00:08:51.688 INFO:teuthology.orchestra.run.trial138.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40881.log.gz 2026-02-21T00:08:51.688 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.40507.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40574.log 2026-02-21T00:08:51.688 INFO:teuthology.orchestra.run.trial138.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40507.log.gz 2026-02-21T00:08:51.688 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.44799.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-21T00:08:51.688 INFO:teuthology.orchestra.run.trial138.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.44799.log.gz 2026-02-21T00:08:51.688 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.40574.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44092.log 2026-02-21T00:08:51.688 INFO:teuthology.orchestra.run.trial138.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40574.log.gz 2026-02-21T00:08:51.688 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40783.log 2026-02-21T00:08:51.688 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.44092.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40669.log 2026-02-21T00:08:51.689 INFO:teuthology.orchestra.run.trial138.stderr: 85.4% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-21T00:08:51.689 INFO:teuthology.orchestra.run.trial138.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44092.log.gz 2026-02-21T00:08:51.689 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.40783.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40832.log 2026-02-21T00:08:51.689 INFO:teuthology.orchestra.run.trial138.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40783.log.gz 2026-02-21T00:08:51.689 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.40669.log: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.40669.log.gz 2026-02-21T00:08:51.689 INFO:teuthology.orchestra.run.trial138.stderr:/var/log/ceph/ceph-client.admin.40832.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40832.log.gz 2026-02-21T00:08:51.689 INFO:teuthology.orchestra.run.trial138.stderr: 91.9% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-21T00:08:56.404 INFO:teuthology.orchestra.run.trial138.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-21T00:08:56.538 INFO:teuthology.orchestra.run.trial138.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-21T00:08:56.805 INFO:teuthology.orchestra.run.trial138.stderr: 92.2% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-21T00:08:56.807 INFO:teuthology.orchestra.run.trial138.stderr: 2026-02-21T00:08:56.808 INFO:teuthology.orchestra.run.trial138.stderr:real 0m5.145s 2026-02-21T00:08:56.808 INFO:teuthology.orchestra.run.trial138.stderr:user 0m13.752s 2026-02-21T00:08:56.808 INFO:teuthology.orchestra.run.trial138.stderr:sys 0m0.969s 2026-02-21T00:08:56.808 INFO:tasks.ceph:Archiving logs... 2026-02-21T00:08:56.808 DEBUG:teuthology.misc:Transferring archived files from trial138:/var/log/ceph to /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61909/remote/trial138/log 2026-02-21T00:08:56.810 DEBUG:teuthology.orchestra.run.trial138:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-21T00:08:58.063 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-21T00:08:58.133 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-21T00:08:58.144 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-21T00:08:58.144 DEBUG:teuthology.orchestra.run.trial138:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-21T00:08:58.170 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-21T00:08:58.181 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-21T00:08:58.181 DEBUG:teuthology.orchestra.run.trial138:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-21T00:08:58.225 INFO:teuthology.orchestra.run.trial138.stderr:bash: line 1: ntpq: command not found 2026-02-21T00:08:58.279 INFO:teuthology.orchestra.run.trial138.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-21T00:08:58.279 INFO:teuthology.orchestra.run.trial138.stdout:=============================================================================== 2026-02-21T00:08:58.279 INFO:teuthology.orchestra.run.trial138.stdout:^* ntp.netlinkify.com 2 7 377 105 -9069ns[ +47us] +/- 7066us 2026-02-21T00:08:58.279 INFO:teuthology.orchestra.run.trial138.stdout:^- 172-235-154-118.ip.linod> 4 7 377 104 -1383us[-1383us] +/- 53ms 2026-02-21T00:08:58.279 INFO:teuthology.orchestra.run.trial138.stdout:^- time2.beanman.net 2 6 377 234 +706us[ +763us] +/- 42ms 2026-02-21T00:08:58.279 INFO:teuthology.orchestra.run.trial138.stdout:^- 172-233-177-198.ip.linod> 2 7 377 103 -3653us[-3653us] +/- 84ms 2026-02-21T00:08:58.282 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-21T00:08:58.293 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-21T00:08:58.294 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-21T00:08:58.309 DEBUG:teuthology.orchestra.run.trial138:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-21T00:08:58.380 DEBUG:teuthology.orchestra.run.trial138:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-21T00:08:58.406 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T00:08:58.407 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-21T00:08:58.423 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-21T00:08:58.438 INFO:teuthology.task.internal:Duration was 1514.293713 seconds 2026-02-21T00:08:58.438 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-21T00:08:58.454 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-21T00:08:58.454 DEBUG:teuthology.orchestra.run.trial138:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-21T00:08:58.493 INFO:teuthology.orchestra.run.trial138.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-21T00:08:58.629 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-21T00:08:58.629 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial138.front.sepia.ceph.com 2026-02-21T00:08:58.629 DEBUG:teuthology.orchestra.run.trial138:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-21T00:08:58.687 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-21T00:08:58.687 DEBUG:teuthology.orchestra.run.trial138:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T00:08:58.792 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-21T00:08:58.792 DEBUG:teuthology.orchestra.run.trial138:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-21T00:08:58.815 INFO:teuthology.orchestra.run.trial138.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-21T00:08:58.815 INFO:teuthology.orchestra.run.trial138.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-21T00:08:58.816 INFO:teuthology.orchestra.run.trial138.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-21T00:08:58.817 INFO:teuthology.orchestra.run.trial138.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-21T00:08:58.817 INFO:teuthology.orchestra.run.trial138.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-21T00:08:58.827 INFO:teuthology.orchestra.run.trial138.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 83.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-21T00:08:58.829 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-21T00:08:58.846 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-21T00:08:58.846 DEBUG:teuthology.orchestra.run.trial138:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-21T00:08:58.894 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-21T00:08:58.911 DEBUG:teuthology.orchestra.run.trial138:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-21T00:08:58.958 INFO:teuthology.orchestra.run.trial138.stdout:kernel.core_pattern = core 2026-02-21T00:08:58.973 DEBUG:teuthology.orchestra.run.trial138:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-21T00:08:59.026 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-21T00:08:59.027 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-21T00:08:59.044 INFO:teuthology.task.internal:Transferring archived files... 2026-02-21T00:08:59.045 DEBUG:teuthology.misc:Transferring archived files from trial138:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/liucm-2026-02-20_22:16:48-crimson-rados-wip-liucm-osd-shard-redesign-crimson-only-distro-debug-trial/61909/remote/trial138 2026-02-21T00:08:59.045 DEBUG:teuthology.orchestra.run.trial138:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-21T00:08:59.121 INFO:teuthology.task.internal:Removing archive directory... 2026-02-21T00:08:59.121 DEBUG:teuthology.orchestra.run.trial138:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-21T00:08:59.153 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-21T00:08:59.284 INFO:teuthology.task.internal:Not uploading archives. 2026-02-21T00:08:59.284 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-21T00:08:59.301 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-21T00:08:59.301 DEBUG:teuthology.orchestra.run.trial138:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-21T00:08:59.319 INFO:teuthology.orchestra.run.trial138.stdout: 917576 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 21 00:08 /home/ubuntu/cephtest 2026-02-21T00:08:59.320 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-21T00:08:59.337 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-21T00:08:59.376 INFO:teuthology.run:Summary data: description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} perf_ignorelist settings/optimized workloads/radosbench_4K_seq_read} duration: 1514.2937133312225 flavor: debug owner: scheduled_liucm@soko04.front.sepia.ceph.com success: true 2026-02-21T00:08:59.376 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-21T00:08:59.500 INFO:teuthology.run:pass