2026-02-12T09:04:35.414 INFO:root:teuthology version: 0.0.1.dev29+gc433f1062 2026-02-12T09:04:35.430 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-12T09:04:35.508 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41608 branch: main description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} perf_ignorelist settings/optimized workloads/sample_radosbench} email: null first_in_suite: false flavor: debug job_id: '41608' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: global: auth client required: none auth cluster required: none auth service required: none auth supported: none debug asok: 0/0 debug auth: 0/0 debug bdev: 0/0 debug bluefs: 0/0 debug bluestore: 0/0 debug buffer: 0/0 debug civetweb: 0/0 debug client: 0/0 debug compressor: 0/0 debug context: 0/0 debug crush: 0/0 debug crypto: 0/0 debug dpdk: 0/0 debug eventtrace: 0/0 debug filer: 0/0 debug filestore: 0/0 debug finisher: 0/0 debug fuse: 0/0 debug heartbeatmap: 0/0 debug javaclient: 0/0 debug journal: 0/0 debug journaler: 0/0 debug kstore: 0/0 debug leveldb: 0/0 debug lockdep: 0/0 debug mds: 0/0 debug mds balancer: 0/0 debug mds locker: 0/0 debug mds log: 0/0 debug mds log expire: 0/0 debug mds migrator: 0/0 debug memdb: 0/0 debug mgr: 0/0 debug mgrc: 0/0 debug mon: 0/0 debug monc: 0/0 debug ms: 0/0 debug objclass: 0/0 debug objectcacher: 0/0 debug objecter: 0/0 debug optracker: 0/0 debug osd: 0/0 debug paxos: 0/0 debug perfcounter: 0/0 debug rados: 0/0 debug rbd: 0/0 debug rbd mirror: 0/0 debug rbd replay: 0/0 debug refs: 0/0 debug rgw: 0/0 debug rgw lifecycle: 0/0 debug rgw notification: 0/0 debug rgw sync: 0/0 debug rocksdb: 0/0 debug striper: 0/0 debug throttle: 0/0 debug timer: 0/0 debug tp: 0/0 enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 0/0 debug ms: 0/0 debug paxos: 0/0 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug filestore: 0/0 debug journal: 0/0 debug ms: 0/0 debug osd: 0/0 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore main device type: RANDOM_BLOCK_SSD seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - \(PG_ - \(OSD_ - \(OBJECT_ - overall HEALTH sha1: 2401a109e2ac85dfa20fc57328da663aa754c875 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: flavor: debug sha1: 2401a109e2ac85dfa20fc57328da663aa754c875 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: 6bf51b64d7733c58c2fe4a45746d4b9e65a38d1c owner: scheduled_matan@soko04.front.sepia.ceph.com priority: 100 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 9353 sha1: 2401a109e2ac85dfa20fc57328da663aa754c875 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_6bf51b64d7733c58c2fe4a45746d4b9e65a38d1c/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: 6bf51b64d7733c58c2fe4a45746d4b9e65a38d1c targets: trial044.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOaZc0C2NAzuUJPgXqCMBzDA7HYYFtbI1LoQ+w3pmVlQPfopASy97WhHUjHuJy5DGTm9BXm8EQJckg33aMoCPGY= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: osd: debug monc: 20 wait-for-scrub: false - ssh_keys: null - cbt: benchmarks: radosbench: concurrent_ops: 4 concurrent_procs: 2 op_size: - 4096 pool_monitoring_list: - collectl pool_profile: replicated run_monitoring_list: - collectl time: 60 write_only: true cluster: iterations: 1 osds_per_node: 3 pool_profiles: replicated: pg_size: 256 pgp_size: 256 replication: replicated user: ubuntu monitoring_profiles: perf: args: stat -p {pid} -o {perf_dir}/perf_stat.{pid} nodes: - osds perf_cmd: perf pid_glob: crimson-osd.*.pid teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: c433f1062990a0488dc29a553589bc609a460691 timestamp: 2026-02-09_15:50:17 tube: trial user: matan verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.2161924 2026-02-12T09:04:35.508 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_6bf51b64d7733c58c2fe4a45746d4b9e65a38d1c/qa; will attempt to use it 2026-02-12T09:04:35.509 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_6bf51b64d7733c58c2fe4a45746d4b9e65a38d1c/qa/tasks 2026-02-12T09:04:35.509 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-12T09:04:35.510 INFO:teuthology.task.internal:Checking packages... 2026-02-12T09:04:35.510 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash '2401a109e2ac85dfa20fc57328da663aa754c875' 2026-02-12T09:04:35.510 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-12T09:04:35.510 INFO:teuthology.packaging:ref: None 2026-02-12T09:04:35.510 INFO:teuthology.packaging:tag: None 2026-02-12T09:04:35.511 INFO:teuthology.packaging:branch: main 2026-02-12T09:04:35.511 INFO:teuthology.packaging:sha1: 2401a109e2ac85dfa20fc57328da663aa754c875 2026-02-12T09:04:35.511 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=main 2026-02-12T09:04:35.655 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5271.gdbd7cb72 2026-02-12T09:04:35.657 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-12T09:04:35.670 INFO:teuthology.task.internal:no buildpackages task found 2026-02-12T09:04:35.670 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-12T09:04:35.685 INFO:teuthology.task.internal:Saving configuration 2026-02-12T09:04:35.705 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-12T09:04:35.717 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-12T09:04:35.800 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial044.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41608', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-12 09:01:22.387898', 'locked_by': 'scheduled_matan@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOaZc0C2NAzuUJPgXqCMBzDA7HYYFtbI1LoQ+w3pmVlQPfopASy97WhHUjHuJy5DGTm9BXm8EQJckg33aMoCPGY='} 2026-02-12T09:04:35.800 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-12T09:04:35.811 INFO:teuthology.task.internal:roles: ubuntu@trial044.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-12T09:04:35.812 INFO:teuthology.run_tasks:Running task console_log... 2026-02-12T09:04:35.898 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7e1c4ce0d760>, signals=[15]) 2026-02-12T09:04:35.898 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-12T09:04:35.910 INFO:teuthology.task.internal:Opening connections... 2026-02-12T09:04:35.910 DEBUG:teuthology.task.internal:connecting to ubuntu@trial044.front.sepia.ceph.com 2026-02-12T09:04:35.910 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T09:04:35.973 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-12T09:04:35.985 DEBUG:teuthology.orchestra.run.trial044:> uname -m 2026-02-12T09:04:36.049 INFO:teuthology.orchestra.run.trial044.stdout:x86_64 2026-02-12T09:04:36.050 DEBUG:teuthology.orchestra.run.trial044:> cat /etc/os-release 2026-02-12T09:04:36.104 INFO:teuthology.orchestra.run.trial044.stdout:NAME="CentOS Stream" 2026-02-12T09:04:36.104 INFO:teuthology.orchestra.run.trial044.stdout:VERSION="9" 2026-02-12T09:04:36.104 INFO:teuthology.orchestra.run.trial044.stdout:ID="centos" 2026-02-12T09:04:36.104 INFO:teuthology.orchestra.run.trial044.stdout:ID_LIKE="rhel fedora" 2026-02-12T09:04:36.104 INFO:teuthology.orchestra.run.trial044.stdout:VERSION_ID="9" 2026-02-12T09:04:36.105 INFO:teuthology.orchestra.run.trial044.stdout:PLATFORM_ID="platform:el9" 2026-02-12T09:04:36.105 INFO:teuthology.orchestra.run.trial044.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-12T09:04:36.105 INFO:teuthology.orchestra.run.trial044.stdout:ANSI_COLOR="0;31" 2026-02-12T09:04:36.105 INFO:teuthology.orchestra.run.trial044.stdout:LOGO="fedora-logo-icon" 2026-02-12T09:04:36.105 INFO:teuthology.orchestra.run.trial044.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-12T09:04:36.105 INFO:teuthology.orchestra.run.trial044.stdout:HOME_URL="https://centos.org/" 2026-02-12T09:04:36.105 INFO:teuthology.orchestra.run.trial044.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-12T09:04:36.105 INFO:teuthology.orchestra.run.trial044.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-12T09:04:36.105 INFO:teuthology.orchestra.run.trial044.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-12T09:04:36.105 INFO:teuthology.lock.ops:Updating trial044.front.sepia.ceph.com on lock server 2026-02-12T09:04:36.192 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-12T09:04:36.205 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-12T09:04:36.217 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-12T09:04:36.217 DEBUG:teuthology.orchestra.run.trial044:> test '!' -e /home/ubuntu/cephtest 2026-02-12T09:04:36.232 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-12T09:04:36.244 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-12T09:04:36.244 DEBUG:teuthology.orchestra.run.trial044:> test -z $(ls -A /var/lib/ceph) 2026-02-12T09:04:36.289 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-12T09:04:36.397 INFO:teuthology.run_tasks:Running task kernel... 2026-02-12T09:04:36.419 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-12T09:04:36.419 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-12T09:04:36.419 DEBUG:teuthology.orchestra.run.trial044:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-12T09:04:36.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T09:04:36.434 DEBUG:teuthology.orchestra.run.trial044:> uname -r 2026-02-12T09:04:36.490 INFO:teuthology.orchestra.run.trial044.stdout:5.14.0-645.el9.x86_64 2026-02-12T09:04:36.490 INFO:teuthology.task.kernel:Running kernel on trial044: 5.14.0-645.el9.x86_64 2026-02-12T09:04:36.490 DEBUG:teuthology.orchestra.run.trial044:> sudo yum install -y kernel 2026-02-12T09:04:37.893 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - BaseOS 8.6 MB/s | 8.9 MB 00:01 2026-02-12T09:05:04.350 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 27 MB 00:25 2026-02-12T09:05:15.092 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - CRB 949 kB/s | 7.8 MB 00:08 2026-02-12T09:05:15.914 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - Extras packages 198 kB/s | 20 kB 00:00 2026-02-12T09:05:16.923 INFO:teuthology.orchestra.run.trial044.stdout:Extra Packages for Enterprise Linux 22 MB/s | 20 MB 00:00 2026-02-12T09:05:20.068 INFO:teuthology.orchestra.run.trial044.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-12T09:05:21.124 INFO:teuthology.orchestra.run.trial044.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-12T09:05:21.144 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:05:21.145 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:05:21.145 INFO:teuthology.orchestra.run.trial044.stdout: Package Architecture Version Repository Size 2026-02-12T09:05:21.145 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:05:21.145 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-12T09:05:21.145 INFO:teuthology.orchestra.run.trial044.stdout: kernel x86_64 5.14.0-677.el9 baseos 768 k 2026-02-12T09:05:21.145 INFO:teuthology.orchestra.run.trial044.stdout:Installing dependencies: 2026-02-12T09:05:21.145 INFO:teuthology.orchestra.run.trial044.stdout: kernel-core x86_64 5.14.0-677.el9 baseos 17 M 2026-02-12T09:05:21.146 INFO:teuthology.orchestra.run.trial044.stdout: kernel-modules x86_64 5.14.0-677.el9 baseos 40 M 2026-02-12T09:05:21.146 INFO:teuthology.orchestra.run.trial044.stdout: kernel-modules-core x86_64 5.14.0-677.el9 baseos 31 M 2026-02-12T09:05:21.146 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:05:21.146 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-12T09:05:21.146 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:05:21.146 INFO:teuthology.orchestra.run.trial044.stdout:Install 4 Packages 2026-02-12T09:05:21.146 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:05:21.146 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 88 M 2026-02-12T09:05:21.146 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 135 M 2026-02-12T09:05:21.146 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-12T09:05:21.332 INFO:teuthology.orchestra.run.trial044.stdout:(1/4): kernel-5.14.0-677.el9.x86_64.rpm 5.8 MB/s | 768 kB 00:00 2026-02-12T09:05:21.479 INFO:teuthology.orchestra.run.trial044.stdout:(2/4): kernel-core-5.14.0-677.el9.x86_64.rpm 62 MB/s | 17 MB 00:00 2026-02-12T09:05:21.891 INFO:teuthology.orchestra.run.trial044.stdout:(3/4): kernel-modules-core-5.14.0-677.el9.x86_6 55 MB/s | 31 MB 00:00 2026-02-12T09:05:21.988 INFO:teuthology.orchestra.run.trial044.stdout:(4/4): kernel-modules-5.14.0-677.el9.x86_64.rpm 50 MB/s | 40 MB 00:00 2026-02-12T09:05:21.988 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:05:21.988 INFO:teuthology.orchestra.run.trial044.stdout:Total 105 MB/s | 88 MB 00:00 2026-02-12T09:05:22.175 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-12T09:05:22.377 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-12T09:05:22.377 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-12T09:05:22.724 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-12T09:05:22.724 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-12T09:05:23.820 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-12T09:05:24.103 INFO:teuthology.orchestra.run.trial044.stdout: Installing : kernel-modules-core-5.14.0-677.el9.x86_64 1/4 2026-02-12T09:05:24.122 INFO:teuthology.orchestra.run.trial044.stdout: Installing : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T09:05:25.877 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T09:05:25.905 INFO:teuthology.orchestra.run.trial044.stdout: Installing : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T09:05:29.135 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T09:05:29.141 INFO:teuthology.orchestra.run.trial044.stdout: Installing : kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T09:05:32.182 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T09:06:02.535 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: kernel-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T09:06:02.556 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: kernel-modules-5.14.0-677.el9.x86_64 4/4 2026-02-12T09:06:02.728 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: kernel-5.14.0-677.el9.x86_64 4/4 2026-02-12T09:06:02.728 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : kernel-5.14.0-677.el9.x86_64 1/4 2026-02-12T09:06:02.728 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : kernel-core-5.14.0-677.el9.x86_64 2/4 2026-02-12T09:06:02.728 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : kernel-modules-5.14.0-677.el9.x86_64 3/4 2026-02-12T09:06:02.780 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : kernel-modules-core-5.14.0-677.el9.x86_64 4/4 2026-02-12T09:06:02.780 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:06:02.780 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-12T09:06:02.780 INFO:teuthology.orchestra.run.trial044.stdout: kernel-5.14.0-677.el9.x86_64 2026-02-12T09:06:02.781 INFO:teuthology.orchestra.run.trial044.stdout: kernel-core-5.14.0-677.el9.x86_64 2026-02-12T09:06:02.781 INFO:teuthology.orchestra.run.trial044.stdout: kernel-modules-5.14.0-677.el9.x86_64 2026-02-12T09:06:02.781 INFO:teuthology.orchestra.run.trial044.stdout: kernel-modules-core-5.14.0-677.el9.x86_64 2026-02-12T09:06:02.781 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:06:02.781 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:06:02.839 DEBUG:teuthology.orchestra.run.trial044:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T09:06:02.864 INFO:teuthology.orchestra.run.trial044.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T09:06:02.866 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T09:06:02.866 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-677.el9.x86_64 2026-02-12T09:06:02.866 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-12T09:06:02.866 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-12T09:06:02.866 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-12T09:06:02.866 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial044.front.sepia.ceph.com, path=None, version=distro) 2026-02-12T09:06:02.866 DEBUG:teuthology.orchestra.run.trial044:> rpm -q kernel | sort -rV | head -n 1 2026-02-12T09:06:02.928 INFO:teuthology.orchestra.run.trial044.stdout:kernel-5.14.0-677.el9.x86_64 2026-02-12T09:06:02.929 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-677.el9.x86_64 2026-02-12T09:06:02.930 DEBUG:teuthology.orchestra.run.trial044:> sudo rpm -qi grub2-tools 2026-02-12T09:06:02.998 INFO:teuthology.orchestra.run.trial044.stdout:Name : grub2-tools 2026-02-12T09:06:02.999 INFO:teuthology.orchestra.run.trial044.stdout:Epoch : 1 2026-02-12T09:06:02.999 INFO:teuthology.orchestra.run.trial044.stdout:Version : 2.06 2026-02-12T09:06:02.999 INFO:teuthology.orchestra.run.trial044.stdout:Release : 118.el9 2026-02-12T09:06:02.999 INFO:teuthology.orchestra.run.trial044.stdout:Architecture: x86_64 2026-02-12T09:06:02.999 INFO:teuthology.orchestra.run.trial044.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-12T09:06:02.999 INFO:teuthology.orchestra.run.trial044.stdout:Group : Unspecified 2026-02-12T09:06:02.999 INFO:teuthology.orchestra.run.trial044.stdout:Size : 8393909 2026-02-12T09:06:02.999 INFO:teuthology.orchestra.run.trial044.stdout:License : GPLv3+ 2026-02-12T09:06:02.999 INFO:teuthology.orchestra.run.trial044.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-12T09:06:03.000 INFO:teuthology.orchestra.run.trial044.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-12T09:06:03.000 INFO:teuthology.orchestra.run.trial044.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-12T09:06:03.000 INFO:teuthology.orchestra.run.trial044.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-12T09:06:03.000 INFO:teuthology.orchestra.run.trial044.stdout:Packager : builder@centos.org 2026-02-12T09:06:03.000 INFO:teuthology.orchestra.run.trial044.stdout:Vendor : CentOS 2026-02-12T09:06:03.000 INFO:teuthology.orchestra.run.trial044.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-12T09:06:03.000 INFO:teuthology.orchestra.run.trial044.stdout:Summary : Support tools for GRUB. 2026-02-12T09:06:03.000 INFO:teuthology.orchestra.run.trial044.stdout:Description : 2026-02-12T09:06:03.000 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:06:03.000 INFO:teuthology.orchestra.run.trial044.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-12T09:06:03.000 INFO:teuthology.orchestra.run.trial044.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-12T09:06:03.000 INFO:teuthology.orchestra.run.trial044.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-12T09:06:03.001 INFO:teuthology.orchestra.run.trial044.stdout:hardware devices. 2026-02-12T09:06:03.001 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:06:03.001 INFO:teuthology.orchestra.run.trial044.stdout:This subpackage provides tools for support of all platforms. 2026-02-12T09:06:03.002 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-12T09:06:03.002 INFO:teuthology.task.kernel:Updating grub on trial044 to boot 5.14.0-677.el9.x86_64 2026-02-12T09:06:03.002 DEBUG:teuthology.orchestra.run.trial044:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-12T09:06:03.065 INFO:teuthology.orchestra.run.trial044.stdout:/boot/loader/entries/2734456005d743dca7f3dc77c35a2d0d-0-rescue.conf 2026-02-12T09:06:03.066 DEBUG:teuthology.orchestra.run.trial044:> sudo command -v grubby && echo yes || echo no 2026-02-12T09:06:03.132 INFO:teuthology.orchestra.run.trial044.stdout:/sbin/grubby 2026-02-12T09:06:03.133 INFO:teuthology.orchestra.run.trial044.stdout:yes 2026-02-12T09:06:03.134 DEBUG:teuthology.orchestra.run.trial044:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-677.el9.x86_64" | head -n 1 2026-02-12T09:06:03.197 INFO:teuthology.orchestra.run.trial044.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64.conf 2026-02-12T09:06:03.198 DEBUG:teuthology.orchestra.run.trial044:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-677.el9.x86_64 2026-02-12T09:06:03.270 DEBUG:teuthology.orchestra.run.trial044:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-12T09:06:03.478 INFO:teuthology.orchestra.run.trial044.stderr:Generating grub configuration file ... 2026-02-12T09:06:03.542 INFO:teuthology.orchestra.run.trial044.stderr:Found linux image: /boot/vmlinuz-5.14.0-677.el9.x86_64 2026-02-12T09:06:03.548 INFO:teuthology.orchestra.run.trial044.stderr:Found initrd image: /boot/initramfs-5.14.0-677.el9.x86_64.img 2026-02-12T09:06:03.796 INFO:teuthology.orchestra.run.trial044.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-12T09:06:03.802 INFO:teuthology.orchestra.run.trial044.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-12T09:06:03.834 INFO:teuthology.orchestra.run.trial044.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-12T09:06:03.841 INFO:teuthology.orchestra.run.trial044.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-12T09:06:03.864 INFO:teuthology.orchestra.run.trial044.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-12T09:06:03.871 INFO:teuthology.orchestra.run.trial044.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-12T09:06:03.953 INFO:teuthology.orchestra.run.trial044.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-12T09:06:03.961 INFO:teuthology.orchestra.run.trial044.stderr:done 2026-02-12T09:06:03.963 DEBUG:teuthology.orchestra.run.trial044:> test -d /sys/firmware/efi 2026-02-12T09:06:03.979 DEBUG:teuthology.orchestra.run.trial044:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-12T09:06:04.044 INFO:teuthology.orchestra.run.trial044.stdout:/boot/efi/EFI/BOOT 2026-02-12T09:06:04.044 INFO:teuthology.orchestra.run.trial044.stdout:/boot/efi/EFI/centos 2026-02-12T09:06:04.046 DEBUG:teuthology.orchestra.run.trial044:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-12T09:06:04.111 INFO:teuthology.orchestra.run.trial044.stdout:no 2026-02-12T09:06:04.111 DEBUG:teuthology.orchestra.run.trial044:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-12T09:06:04.175 INFO:teuthology.orchestra.run.trial044.stdout:yes 2026-02-12T09:06:04.175 DEBUG:teuthology.orchestra.run.trial044:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-12T09:06:04.240 DEBUG:teuthology.orchestra.run.trial044:> sudo shutdown -r now 2026-02-12T09:06:34.313 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-12T09:06:34.313 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial044.front.sepia.ceph.com' 2026-02-12T09:06:34.314 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T09:07:34.368 DEBUG:teuthology.orchestra.remote:timed out 2026-02-12T09:07:43.377 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial044.front.sepia.ceph.com' 2026-02-12T09:07:43.378 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T09:07:43.513 DEBUG:teuthology.orchestra.run.trial044:> true 2026-02-12T09:07:43.675 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial044.front.sepia.ceph.com' 2026-02-12T09:07:43.676 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-677.el9.x86_64"... 2026-02-12T09:07:43.676 DEBUG:teuthology.orchestra.run.trial044:> uname -r 2026-02-12T09:07:43.731 INFO:teuthology.orchestra.run.trial044.stdout:5.14.0-677.el9.x86_64 2026-02-12T09:07:43.732 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-677.el9.x86_64 vs 5.14.0-677.el9.x86_64 2026-02-12T09:07:43.732 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-12T09:07:43.732 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-12T09:07:44.732 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-12T09:07:44.732 DEBUG:teuthology.orchestra.run.trial044:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-12T09:07:44.762 INFO:teuthology.orchestra.run.trial044.stdout:ttyS1 2026-02-12T09:07:44.784 DEBUG:teuthology.parallel:result is None 2026-02-12T09:07:44.784 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-12T09:07:44.794 INFO:teuthology.task.internal:Creating test directory... 2026-02-12T09:07:44.794 DEBUG:teuthology.orchestra.run.trial044:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-12T09:07:44.841 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-12T09:07:44.851 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-12T09:07:44.865 INFO:teuthology.task.internal:Creating archive directory... 2026-02-12T09:07:44.865 DEBUG:teuthology.orchestra.run.trial044:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-12T09:07:44.904 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-12T09:07:44.915 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-12T09:07:44.915 DEBUG:teuthology.orchestra.run.trial044:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-12T09:07:44.963 INFO:teuthology.orchestra.run.trial044.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T09:07:44.972 INFO:teuthology.orchestra.run.trial044.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-12T09:07:44.974 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-12T09:07:44.985 INFO:teuthology.task.internal:Configuring sudo... 2026-02-12T09:07:44.985 DEBUG:teuthology.orchestra.run.trial044:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-12T09:07:45.039 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-12T09:07:45.054 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-12T09:07:45.055 DEBUG:teuthology.orchestra.run.trial044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-12T09:07:45.094 DEBUG:teuthology.orchestra.run.trial044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T09:07:45.256 DEBUG:teuthology.orchestra.run.trial044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T09:07:45.281 DEBUG:teuthology.orchestra.run.trial044:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T09:07:45.433 DEBUG:teuthology.orchestra.run.trial044:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T09:07:45.458 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:07:45.458 DEBUG:teuthology.orchestra.run.trial044:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-12T09:07:45.522 DEBUG:teuthology.orchestra.run.trial044:> sudo service rsyslog restart 2026-02-12T09:07:45.589 INFO:teuthology.orchestra.run.trial044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-12T09:07:45.989 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-12T09:07:46.000 INFO:teuthology.task.internal:Starting timer... 2026-02-12T09:07:46.001 INFO:teuthology.run_tasks:Running task pcp... 2026-02-12T09:07:46.024 INFO:teuthology.run_tasks:Running task selinux... 2026-02-12T09:07:46.042 DEBUG:teuthology.orchestra.run.trial044:> sudo service auditd rotate 2026-02-12T09:07:46.071 INFO:teuthology.orchestra.run.trial044.stdout:Rotating logs: 2026-02-12T09:07:46.073 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-12T09:07:46.073 DEBUG:teuthology.orchestra.run.trial044:> /usr/sbin/getenforce 2026-02-12T09:07:46.129 INFO:teuthology.orchestra.run.trial044.stdout:Permissive 2026-02-12T09:07:46.130 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial044.front.sepia.ceph.com': 'permissive'} 2026-02-12T09:07:46.130 DEBUG:teuthology.orchestra.run.trial044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-12T09:07:46.194 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T09:07:46.194 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-12T09:07:46.194 DEBUG:teuthology.orchestra.run.trial044:> sudo /usr/sbin/setenforce permissive 2026-02-12T09:07:46.259 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-12T09:07:46.276 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-12T09:07:46.289 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-12T09:07:46.290 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial044.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-12T09:08:52.424 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial044.front.sepia.ceph.com')] 2026-02-12T09:08:52.426 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial044.front.sepia.ceph.com' 2026-02-12T09:08:52.427 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial044.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-12T09:08:52.536 DEBUG:teuthology.orchestra.run.trial044:> true 2026-02-12T09:08:52.609 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial044.front.sepia.ceph.com' 2026-02-12T09:08:52.609 INFO:teuthology.run_tasks:Running task clock... 2026-02-12T09:08:52.622 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-12T09:08:52.623 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-12T09:08:52.623 DEBUG:teuthology.orchestra.run.trial044:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-12T09:08:52.676 INFO:teuthology.orchestra.run.trial044.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-12T09:08:52.687 INFO:teuthology.orchestra.run.trial044.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-12T09:08:52.707 INFO:teuthology.orchestra.run.trial044.stderr:sudo: ntpd: command not found 2026-02-12T09:08:52.716 INFO:teuthology.orchestra.run.trial044.stdout:506 Cannot talk to daemon 2026-02-12T09:08:52.728 INFO:teuthology.orchestra.run.trial044.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-12T09:08:52.739 INFO:teuthology.orchestra.run.trial044.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-12T09:08:52.785 INFO:teuthology.orchestra.run.trial044.stderr:bash: line 1: ntpq: command not found 2026-02-12T09:08:52.787 INFO:teuthology.orchestra.run.trial044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-12T09:08:52.787 INFO:teuthology.orchestra.run.trial044.stdout:=============================================================================== 2026-02-12T09:08:52.788 INFO:teuthology.run_tasks:Running task install... 2026-02-12T09:08:52.803 DEBUG:teuthology.task.install:project ceph 2026-02-12T09:08:52.803 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875'}} 2026-02-12T09:08:52.803 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875'} 2026-02-12T09:08:52.803 INFO:teuthology.task.install:Using flavor: debug 2026-02-12T09:08:52.812 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-12T09:08:52.813 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-12T09:08:52.813 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875', 'tag': None, 'wait_for_package': False} 2026-02-12T09:08:52.814 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=2401a109e2ac85dfa20fc57328da663aa754c875 2026-02-12T09:08:52.971 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-66488-revert/2401a109e2ac85dfa20fc57328da663aa754c875/centos/9/flavors/debug/ 2026-02-12T09:08:52.971 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5161.g2401a109 2026-02-12T09:08:53.216 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-66488-revert/2401a109e2ac85dfa20fc57328da663aa754c875/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-66488-revert/2401a109e2ac85dfa20fc57328da663aa754c875/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-66488-revert/2401a109e2ac85dfa20fc57328da663aa754c875/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-12T09:08:53.216 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:08:53.216 DEBUG:teuthology.orchestra.run.trial044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-12T09:08:53.242 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-12T09:08:53.242 DEBUG:teuthology.orchestra.run.trial044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2401a109e2ac85dfa20fc57328da663aa754c875/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-12T09:08:53.307 DEBUG:teuthology.orchestra.run.trial044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-12T09:08:53.380 DEBUG:teuthology.orchestra.run.trial044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-12T09:08:53.443 INFO:teuthology.orchestra.run.trial044.stdout:check_obsoletes = 1 2026-02-12T09:08:53.445 DEBUG:teuthology.orchestra.run.trial044:> sudo yum clean all 2026-02-12T09:08:53.596 INFO:teuthology.orchestra.run.trial044.stdout:41 files removed 2026-02-12T09:08:53.611 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-radosgw 2026-02-12T09:08:54.075 INFO:teuthology.orchestra.run.trial044.stdout:ceph packages for x86_64 293 kB/s | 94 kB 00:00 2026-02-12T09:08:54.468 INFO:teuthology.orchestra.run.trial044.stdout:ceph noarch packages 66 kB/s | 24 kB 00:00 2026-02-12T09:08:54.874 INFO:teuthology.orchestra.run.trial044.stdout:ceph source packages 5.4 kB/s | 2.1 kB 00:00 2026-02-12T09:08:55.565 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.9 MB 00:00 2026-02-12T09:09:21.692 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - AppStream 1.0 MB/s | 27 MB 00:25 2026-02-12T09:09:25.796 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 7.8 MB 00:01 2026-02-12T09:09:26.625 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - Extras packages 193 kB/s | 20 kB 00:00 2026-02-12T09:09:26.870 INFO:teuthology.orchestra.run.trial044.stdout:Extra Packages for Enterprise Linux 119 MB/s | 20 MB 00:00 2026-02-12T09:09:30.133 INFO:teuthology.orchestra.run.trial044.stdout:lab-extras 2.4 MB/s | 50 kB 00:00 2026-02-12T09:09:31.414 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:09:31.415 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:09:31.415 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repository Size 2026-02-12T09:09:31.415 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:09:31.415 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-12T09:09:31.416 INFO:teuthology.orchestra.run.trial044.stdout: ceph-radosgw x86_64 2:20.3.0-5161.g2401a109.el9 ceph 11 M 2026-02-12T09:09:31.416 INFO:teuthology.orchestra.run.trial044.stdout:Upgrading: 2026-02-12T09:09:31.416 INFO:teuthology.orchestra.run.trial044.stdout: librados2 x86_64 2:20.3.0-5161.g2401a109.el9 ceph 3.3 M 2026-02-12T09:09:31.416 INFO:teuthology.orchestra.run.trial044.stdout: librbd1 x86_64 2:20.3.0-5161.g2401a109.el9 ceph 2.7 M 2026-02-12T09:09:31.416 INFO:teuthology.orchestra.run.trial044.stdout:Installing dependencies: 2026-02-12T09:09:31.416 INFO:teuthology.orchestra.run.trial044.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-12T09:09:31.416 INFO:teuthology.orchestra.run.trial044.stdout: ceph-base x86_64 2:20.3.0-5161.g2401a109.el9 ceph 5.1 M 2026-02-12T09:09:31.416 INFO:teuthology.orchestra.run.trial044.stdout: ceph-common x86_64 2:20.3.0-5161.g2401a109.el9 ceph 21 M 2026-02-12T09:09:31.416 INFO:teuthology.orchestra.run.trial044.stdout: ceph-selinux x86_64 2:20.3.0-5161.g2401a109.el9 ceph 25 k 2026-02-12T09:09:31.416 INFO:teuthology.orchestra.run.trial044.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-12T09:09:31.416 INFO:teuthology.orchestra.run.trial044.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-12T09:09:31.417 INFO:teuthology.orchestra.run.trial044.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-12T09:09:31.417 INFO:teuthology.orchestra.run.trial044.stdout: libcephfs2 x86_64 2:20.3.0-5161.g2401a109.el9 ceph 860 k 2026-02-12T09:09:31.417 INFO:teuthology.orchestra.run.trial044.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-12T09:09:31.417 INFO:teuthology.orchestra.run.trial044.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-12T09:09:31.417 INFO:teuthology.orchestra.run.trial044.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-12T09:09:31.417 INFO:teuthology.orchestra.run.trial044.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-12T09:09:31.417 INFO:teuthology.orchestra.run.trial044.stdout: libradosstriper1 x86_64 2:20.3.0-5161.g2401a109.el9 ceph 263 k 2026-02-12T09:09:31.417 INFO:teuthology.orchestra.run.trial044.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-12T09:09:31.417 INFO:teuthology.orchestra.run.trial044.stdout: librgw2 x86_64 2:20.3.0-5161.g2401a109.el9 ceph 5.6 M 2026-02-12T09:09:31.417 INFO:teuthology.orchestra.run.trial044.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-12T09:09:31.417 INFO:teuthology.orchestra.run.trial044.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-12T09:09:31.417 INFO:teuthology.orchestra.run.trial044.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-12T09:09:31.418 INFO:teuthology.orchestra.run.trial044.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-12T09:09:31.418 INFO:teuthology.orchestra.run.trial044.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-12T09:09:31.418 INFO:teuthology.orchestra.run.trial044.stdout: python3-ceph-argparse x86_64 2:20.3.0-5161.g2401a109.el9 ceph 45 k 2026-02-12T09:09:31.418 INFO:teuthology.orchestra.run.trial044.stdout: python3-ceph-common x86_64 2:20.3.0-5161.g2401a109.el9 ceph 184 k 2026-02-12T09:09:31.418 INFO:teuthology.orchestra.run.trial044.stdout: python3-cephfs x86_64 2:20.3.0-5161.g2401a109.el9 ceph 204 k 2026-02-12T09:09:31.418 INFO:teuthology.orchestra.run.trial044.stdout: python3-rados x86_64 2:20.3.0-5161.g2401a109.el9 ceph 323 k 2026-02-12T09:09:31.418 INFO:teuthology.orchestra.run.trial044.stdout: python3-rbd x86_64 2:20.3.0-5161.g2401a109.el9 ceph 305 k 2026-02-12T09:09:31.418 INFO:teuthology.orchestra.run.trial044.stdout: python3-rgw x86_64 2:20.3.0-5161.g2401a109.el9 ceph 100 k 2026-02-12T09:09:31.418 INFO:teuthology.orchestra.run.trial044.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-12T09:09:31.418 INFO:teuthology.orchestra.run.trial044.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-12T09:09:31.418 INFO:teuthology.orchestra.run.trial044.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-12T09:09:31.418 INFO:teuthology.orchestra.run.trial044.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-12T09:09:31.418 INFO:teuthology.orchestra.run.trial044.stdout:Installing weak dependencies: 2026-02-12T09:09:31.419 INFO:teuthology.orchestra.run.trial044.stdout: libcephfs-daemon x86_64 2:20.3.0-5161.g2401a109.el9 ceph 35 k 2026-02-12T09:09:31.419 INFO:teuthology.orchestra.run.trial044.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5161.g2401a109.el9 ceph 24 k 2026-02-12T09:09:31.419 INFO:teuthology.orchestra.run.trial044.stdout: nvme-cli x86_64 2.13-1.el9 baseos 1.1 M 2026-02-12T09:09:31.419 INFO:teuthology.orchestra.run.trial044.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-12T09:09:31.419 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:09:31.419 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-12T09:09:31.419 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:09:31.419 INFO:teuthology.orchestra.run.trial044.stdout:Install 35 Packages 2026-02-12T09:09:31.419 INFO:teuthology.orchestra.run.trial044.stdout:Upgrade 2 Packages 2026-02-12T09:09:31.419 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:09:31.420 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 61 M 2026-02-12T09:09:31.420 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-12T09:09:33.379 INFO:teuthology.orchestra.run.trial044.stdout:(1/37): ceph-radosgw-20.3.0-5161.g2401a109.el9. 7.8 MB/s | 11 MB 00:01 2026-02-12T09:09:33.430 INFO:teuthology.orchestra.run.trial044.stdout:(2/37): ceph-base-20.3.0-5161.g2401a109.el9.x86 3.4 MB/s | 5.1 MB 00:01 2026-02-12T09:09:33.430 INFO:teuthology.orchestra.run.trial044.stdout:(3/37): ceph-selinux-20.3.0-5161.g2401a109.el9. 491 kB/s | 25 kB 00:00 2026-02-12T09:09:33.471 INFO:teuthology.orchestra.run.trial044.stdout:(4/37): libcephfs-daemon-20.3.0-5161.g2401a109. 846 kB/s | 35 kB 00:00 2026-02-12T09:09:33.472 INFO:teuthology.orchestra.run.trial044.stdout:(5/37): libcephfs-proxy2-20.3.0-5161.g2401a109. 581 kB/s | 24 kB 00:00 2026-02-12T09:09:33.524 INFO:teuthology.orchestra.run.trial044.stdout:(6/37): libradosstriper1-20.3.0-5161.g2401a109. 4.9 MB/s | 263 kB 00:00 2026-02-12T09:09:33.630 INFO:teuthology.orchestra.run.trial044.stdout:(7/37): libcephfs2-20.3.0-5161.g2401a109.el9.x8 5.3 MB/s | 860 kB 00:00 2026-02-12T09:09:33.690 INFO:teuthology.orchestra.run.trial044.stdout:(8/37): python3-ceph-argparse-20.3.0-5161.g2401 753 kB/s | 45 kB 00:00 2026-02-12T09:09:33.752 INFO:teuthology.orchestra.run.trial044.stdout:(9/37): python3-ceph-common-20.3.0-5161.g2401a1 2.9 MB/s | 184 kB 00:00 2026-02-12T09:09:33.815 INFO:teuthology.orchestra.run.trial044.stdout:(10/37): python3-cephfs-20.3.0-5161.g2401a109.e 3.2 MB/s | 204 kB 00:00 2026-02-12T09:09:33.862 INFO:teuthology.orchestra.run.trial044.stdout:(11/37): ceph-common-20.3.0-5161.g2401a109.el9. 11 MB/s | 21 MB 00:01 2026-02-12T09:09:33.865 INFO:teuthology.orchestra.run.trial044.stdout:(12/37): python3-rados-20.3.0-5161.g2401a109.el 6.3 MB/s | 323 kB 00:00 2026-02-12T09:09:33.911 INFO:teuthology.orchestra.run.trial044.stdout:(13/37): python3-rbd-20.3.0-5161.g2401a109.el9. 6.1 MB/s | 305 kB 00:00 2026-02-12T09:09:33.913 INFO:teuthology.orchestra.run.trial044.stdout:(14/37): python3-rgw-20.3.0-5161.g2401a109.el9. 2.1 MB/s | 100 kB 00:00 2026-02-12T09:09:33.920 INFO:teuthology.orchestra.run.trial044.stdout:(15/37): librgw2-20.3.0-5161.g2401a109.el9.x86_ 14 MB/s | 5.6 MB 00:00 2026-02-12T09:09:34.471 INFO:teuthology.orchestra.run.trial044.stdout:(16/37): mailcap-2.1.49-5.el9.noarch.rpm 59 kB/s | 33 kB 00:00 2026-02-12T09:09:34.510 INFO:teuthology.orchestra.run.trial044.stdout:(17/37): boost-program-options-1.75.0-13.el9.x8 2.7 MB/s | 104 kB 00:00 2026-02-12T09:09:34.528 INFO:teuthology.orchestra.run.trial044.stdout:(18/37): libnbd-1.20.3-4.el9.x86_64.rpm 9.1 MB/s | 164 kB 00:00 2026-02-12T09:09:34.539 INFO:teuthology.orchestra.run.trial044.stdout:(19/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 14 MB/s | 160 kB 00:00 2026-02-12T09:09:34.548 INFO:teuthology.orchestra.run.trial044.stdout:(20/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 5.0 MB/s | 45 kB 00:00 2026-02-12T09:09:34.568 INFO:teuthology.orchestra.run.trial044.stdout:(21/37): librdkafka-1.6.1-102.el9.x86_64.rpm 32 MB/s | 662 kB 00:00 2026-02-12T09:09:34.580 INFO:teuthology.orchestra.run.trial044.stdout:(22/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 25 MB/s | 292 kB 00:00 2026-02-12T09:09:34.582 INFO:teuthology.orchestra.run.trial044.stdout:(23/37): pciutils-3.7.0-7.el9.x86_64.rpm 141 kB/s | 93 kB 00:00 2026-02-12T09:09:34.591 INFO:teuthology.orchestra.run.trial044.stdout:(24/37): qatlib-25.08.0-2.el9.x86_64.rpm 22 MB/s | 240 kB 00:00 2026-02-12T09:09:34.600 INFO:teuthology.orchestra.run.trial044.stdout:(25/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 7.1 MB/s | 66 kB 00:00 2026-02-12T09:09:34.609 INFO:teuthology.orchestra.run.trial044.stdout:(26/37): qatlib-service-25.08.0-2.el9.x86_64.rp 1.3 MB/s | 37 kB 00:00 2026-02-12T09:09:34.853 INFO:teuthology.orchestra.run.trial044.stdout:(27/37): nvme-cli-2.13-1.el9.x86_64.rpm 1.2 MB/s | 1.1 MB 00:00 2026-02-12T09:09:35.061 INFO:teuthology.orchestra.run.trial044.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 119 kB/s | 25 kB 00:00 2026-02-12T09:09:35.089 INFO:teuthology.orchestra.run.trial044.stdout:(29/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 629 kB/s | 308 kB 00:00 2026-02-12T09:09:35.162 INFO:teuthology.orchestra.run.trial044.stdout:(30/37): libunwind-1.6.2-1.el9.x86_64.rpm 930 kB/s | 67 kB 00:00 2026-02-12T09:09:35.200 INFO:teuthology.orchestra.run.trial044.stdout:(31/37): liboath-2.6.12-1.el9.x86_64.rpm 352 kB/s | 49 kB 00:00 2026-02-12T09:09:35.306 INFO:teuthology.orchestra.run.trial044.stdout:(32/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 5.7 MB/s | 838 kB 00:00 2026-02-12T09:09:35.341 INFO:teuthology.orchestra.run.trial044.stdout:(33/37): re2-20211101-20.el9.x86_64.rpm 1.3 MB/s | 191 kB 00:00 2026-02-12T09:09:35.366 INFO:teuthology.orchestra.run.trial044.stdout:(34/37): libarrow-9.0.0-15.el9.x86_64.rpm 5.8 MB/s | 4.4 MB 00:00 2026-02-12T09:09:35.473 INFO:teuthology.orchestra.run.trial044.stdout:(35/37): thrift-0.15.0-4.el9.x86_64.rpm 9.5 MB/s | 1.6 MB 00:00 2026-02-12T09:09:35.601 INFO:teuthology.orchestra.run.trial044.stdout:(36/37): librbd1-20.3.0-5161.g2401a109.el9.x86_ 11 MB/s | 2.7 MB 00:00 2026-02-12T09:09:35.723 INFO:teuthology.orchestra.run.trial044.stdout:(37/37): librados2-20.3.0-5161.g2401a109.el9.x8 8.6 MB/s | 3.3 MB 00:00 2026-02-12T09:09:35.724 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:09:35.724 INFO:teuthology.orchestra.run.trial044.stdout:Total 14 MB/s | 61 MB 00:04 2026-02-12T09:09:35.824 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-12T09:09:35.843 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-12T09:09:35.843 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-12T09:09:36.088 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-12T09:09:36.088 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-12T09:09:36.387 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-12T09:09:36.526 INFO:teuthology.orchestra.run.trial044.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-12T09:09:36.531 INFO:teuthology.orchestra.run.trial044.stdout: Upgrading : librados2-2:20.3.0-5161.g2401a109.el9.x86_64 2/39 2026-02-12T09:09:36.577 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: librados2-2:20.3.0-5161.g2401a109.el9.x86_64 2/39 2026-02-12T09:09:36.578 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libcephfs2-2:20.3.0-5161.g2401a109.el9.x86_64 3/39 2026-02-12T09:09:36.591 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: libcephfs2-2:20.3.0-5161.g2401a109.el9.x86_64 3/39 2026-02-12T09:09:36.593 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libcephfs-daemon-2:20.3.0-5161.g2401a109.el9.x86_6 4/39 2026-02-12T09:09:36.594 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libcephfs-proxy2-2:20.3.0-5161.g2401a109.el9.x86_6 5/39 2026-02-12T09:09:36.620 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5161.g2401a109.el9.x86_6 5/39 2026-02-12T09:09:36.622 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-rados-2:20.3.0-5161.g2401a109.el9.x86_64 6/39 2026-02-12T09:09:36.630 INFO:teuthology.orchestra.run.trial044.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-12T09:09:36.633 INFO:teuthology.orchestra.run.trial044.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-12T09:09:36.636 INFO:teuthology.orchestra.run.trial044.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-12T09:09:36.641 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-12T09:09:36.652 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-ceph-argparse-2:20.3.0-5161.g2401a109.el9. 11/39 2026-02-12T09:09:36.664 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-cephfs-2:20.3.0-5161.g2401a109.el9.x86_64 12/39 2026-02-12T09:09:36.665 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libradosstriper1-2:20.3.0-5161.g2401a109.el9.x86_6 13/39 2026-02-12T09:09:36.686 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5161.g2401a109.el9.x86_6 13/39 2026-02-12T09:09:36.689 INFO:teuthology.orchestra.run.trial044.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-12T09:09:36.719 INFO:teuthology.orchestra.run.trial044.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-12T09:09:36.724 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-12T09:09:36.730 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-12T09:09:36.732 INFO:teuthology.orchestra.run.trial044.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-12T09:09:36.736 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-12T09:09:36.850 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-12T09:09:36.851 INFO:teuthology.orchestra.run.trial044.stdout: Upgrading : librbd1-2:20.3.0-5161.g2401a109.el9.x86_64 21/39 2026-02-12T09:09:36.877 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: librbd1-2:20.3.0-5161.g2401a109.el9.x86_64 21/39 2026-02-12T09:09:36.880 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-rbd-2:20.3.0-5161.g2401a109.el9.x86_64 22/39 2026-02-12T09:09:36.885 INFO:teuthology.orchestra.run.trial044.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-12T09:09:37.133 INFO:teuthology.orchestra.run.trial044.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-12T09:09:37.134 INFO:teuthology.orchestra.run.trial044.stdout: Installing : librgw2-2:20.3.0-5161.g2401a109.el9.x86_64 25/39 2026-02-12T09:09:37.150 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: librgw2-2:20.3.0-5161.g2401a109.el9.x86_64 25/39 2026-02-12T09:09:37.155 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-rgw-2:20.3.0-5161.g2401a109.el9.x86_64 26/39 2026-02-12T09:09:37.160 INFO:teuthology.orchestra.run.trial044.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-12T09:09:37.162 INFO:teuthology.orchestra.run.trial044.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-12T09:09:37.163 INFO:teuthology.orchestra.run.trial044.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-12T09:09:37.182 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-12T09:09:37.294 INFO:teuthology.orchestra.run.trial044.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-12T09:09:37.296 INFO:teuthology.orchestra.run.trial044.stdout: Installing : nvme-cli-2.13-1.el9.x86_64 31/39 2026-02-12T09:09:37.576 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: nvme-cli-2.13-1.el9.x86_64 31/39 2026-02-12T09:09:37.576 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-12T09:09:37.576 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:09:37.620 INFO:teuthology.orchestra.run.trial044.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-12T09:09:37.622 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-ceph-common-2:20.3.0-5161.g2401a109.el9.x8 33/39 2026-02-12T09:09:38.562 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-common-2:20.3.0-5161.g2401a109.el9.x86_64 34/39 2026-02-12T09:09:38.565 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-common-2:20.3.0-5161.g2401a109.el9.x86_64 34/39 2026-02-12T09:09:38.824 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-common-2:20.3.0-5161.g2401a109.el9.x86_64 34/39 2026-02-12T09:09:38.827 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-base-2:20.3.0-5161.g2401a109.el9.x86_64 35/39 2026-02-12T09:09:38.866 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-base-2:20.3.0-5161.g2401a109.el9.x86_64 35/39 2026-02-12T09:09:38.867 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-12T09:09:38.867 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-12T09:09:38.867 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:09:38.870 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-selinux-2:20.3.0-5161.g2401a109.el9.x86_64 36/39 2026-02-12T09:09:43.382 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5161.g2401a109.el9.x86_64 36/39 2026-02-12T09:09:43.382 INFO:teuthology.orchestra.run.trial044.stdout:skipping the directory /sys 2026-02-12T09:09:43.382 INFO:teuthology.orchestra.run.trial044.stdout:skipping the directory /proc 2026-02-12T09:09:43.382 INFO:teuthology.orchestra.run.trial044.stdout:skipping the directory /mnt 2026-02-12T09:09:43.382 INFO:teuthology.orchestra.run.trial044.stdout:skipping the directory /var/tmp 2026-02-12T09:09:43.382 INFO:teuthology.orchestra.run.trial044.stdout:skipping the directory /home 2026-02-12T09:09:43.382 INFO:teuthology.orchestra.run.trial044.stdout:skipping the directory /root 2026-02-12T09:09:43.382 INFO:teuthology.orchestra.run.trial044.stdout:skipping the directory /tmp 2026-02-12T09:09:43.383 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:09:43.873 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-radosgw-2:20.3.0-5161.g2401a109.el9.x86_64 37/39 2026-02-12T09:09:43.897 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5161.g2401a109.el9.x86_64 37/39 2026-02-12T09:09:43.897 INFO:teuthology.orchestra.run.trial044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T09:09:43.898 INFO:teuthology.orchestra.run.trial044.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-12T09:09:43.898 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-12T09:09:43.898 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-12T09:09:43.898 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:09:43.899 INFO:teuthology.orchestra.run.trial044.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-12T09:09:43.913 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-12T09:09:43.913 INFO:teuthology.orchestra.run.trial044.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-12T09:09:43.968 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-12T09:09:43.968 INFO:teuthology.orchestra.run.trial044.stdout:Creating group 'qat' with GID 992. 2026-02-12T09:09:43.968 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:09:44.244 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-base-2:20.3.0-5161.g2401a109.el9.x86_64 1/39 2026-02-12T09:09:44.245 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-common-2:20.3.0-5161.g2401a109.el9.x86_64 2/39 2026-02-12T09:09:44.245 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-radosgw-2:20.3.0-5161.g2401a109.el9.x86_64 3/39 2026-02-12T09:09:44.245 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-selinux-2:20.3.0-5161.g2401a109.el9.x86_64 4/39 2026-02-12T09:09:44.245 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libcephfs-daemon-2:20.3.0-5161.g2401a109.el9.x86_6 5/39 2026-02-12T09:09:44.245 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5161.g2401a109.el9.x86_6 6/39 2026-02-12T09:09:44.245 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libcephfs2-2:20.3.0-5161.g2401a109.el9.x86_64 7/39 2026-02-12T09:09:44.245 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libradosstriper1-2:20.3.0-5161.g2401a109.el9.x86_6 8/39 2026-02-12T09:09:44.245 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librgw2-2:20.3.0-5161.g2401a109.el9.x86_64 9/39 2026-02-12T09:09:44.245 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5161.g2401a109.el9. 10/39 2026-02-12T09:09:44.246 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-ceph-common-2:20.3.0-5161.g2401a109.el9.x8 11/39 2026-02-12T09:09:44.246 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-cephfs-2:20.3.0-5161.g2401a109.el9.x86_64 12/39 2026-02-12T09:09:44.246 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-rados-2:20.3.0-5161.g2401a109.el9.x86_64 13/39 2026-02-12T09:09:44.246 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-rbd-2:20.3.0-5161.g2401a109.el9.x86_64 14/39 2026-02-12T09:09:44.246 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-rgw-2:20.3.0-5161.g2401a109.el9.x86_64 15/39 2026-02-12T09:09:44.246 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-12T09:09:44.246 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : nvme-cli-2.13-1.el9.x86_64 17/39 2026-02-12T09:09:44.246 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-12T09:09:44.246 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-12T09:09:44.246 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-12T09:09:44.246 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-12T09:09:44.246 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-12T09:09:44.247 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-12T09:09:44.247 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-12T09:09:44.247 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-12T09:09:44.247 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-12T09:09:44.247 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-12T09:09:44.247 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-12T09:09:44.247 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-12T09:09:44.247 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-12T09:09:44.247 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-12T09:09:44.247 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-12T09:09:44.247 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-12T09:09:44.248 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-12T09:09:44.248 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-12T09:09:44.248 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librados2-2:20.3.0-5161.g2401a109.el9.x86_64 36/39 2026-02-12T09:09:44.248 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-12T09:09:44.248 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librbd1-2:20.3.0-5161.g2401a109.el9.x86_64 38/39 2026-02-12T09:09:44.304 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-12T09:09:44.304 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:09:44.304 INFO:teuthology.orchestra.run.trial044.stdout:Upgraded: 2026-02-12T09:09:44.304 INFO:teuthology.orchestra.run.trial044.stdout: librados2-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.304 INFO:teuthology.orchestra.run.trial044.stdout: librbd1-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.304 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-12T09:09:44.305 INFO:teuthology.orchestra.run.trial044.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-12T09:09:44.305 INFO:teuthology.orchestra.run.trial044.stdout: ceph-base-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.305 INFO:teuthology.orchestra.run.trial044.stdout: ceph-common-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.305 INFO:teuthology.orchestra.run.trial044.stdout: ceph-radosgw-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.305 INFO:teuthology.orchestra.run.trial044.stdout: ceph-selinux-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.305 INFO:teuthology.orchestra.run.trial044.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-12T09:09:44.305 INFO:teuthology.orchestra.run.trial044.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-12T09:09:44.305 INFO:teuthology.orchestra.run.trial044.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-12T09:09:44.305 INFO:teuthology.orchestra.run.trial044.stdout: libcephfs-daemon-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.305 INFO:teuthology.orchestra.run.trial044.stdout: libcephfs-proxy2-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.306 INFO:teuthology.orchestra.run.trial044.stdout: libcephfs2-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.306 INFO:teuthology.orchestra.run.trial044.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-12T09:09:44.306 INFO:teuthology.orchestra.run.trial044.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-12T09:09:44.306 INFO:teuthology.orchestra.run.trial044.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-12T09:09:44.306 INFO:teuthology.orchestra.run.trial044.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-12T09:09:44.306 INFO:teuthology.orchestra.run.trial044.stdout: libradosstriper1-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.306 INFO:teuthology.orchestra.run.trial044.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-12T09:09:44.306 INFO:teuthology.orchestra.run.trial044.stdout: librgw2-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.306 INFO:teuthology.orchestra.run.trial044.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-12T09:09:44.306 INFO:teuthology.orchestra.run.trial044.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-12T09:09:44.306 INFO:teuthology.orchestra.run.trial044.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-12T09:09:44.307 INFO:teuthology.orchestra.run.trial044.stdout: nvme-cli-2.13-1.el9.x86_64 2026-02-12T09:09:44.307 INFO:teuthology.orchestra.run.trial044.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-12T09:09:44.307 INFO:teuthology.orchestra.run.trial044.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-12T09:09:44.307 INFO:teuthology.orchestra.run.trial044.stdout: python3-ceph-argparse-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.307 INFO:teuthology.orchestra.run.trial044.stdout: python3-ceph-common-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.307 INFO:teuthology.orchestra.run.trial044.stdout: python3-cephfs-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.307 INFO:teuthology.orchestra.run.trial044.stdout: python3-rados-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.307 INFO:teuthology.orchestra.run.trial044.stdout: python3-rbd-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.307 INFO:teuthology.orchestra.run.trial044.stdout: python3-rgw-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:09:44.307 INFO:teuthology.orchestra.run.trial044.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-12T09:09:44.307 INFO:teuthology.orchestra.run.trial044.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-12T09:09:44.308 INFO:teuthology.orchestra.run.trial044.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-12T09:09:44.308 INFO:teuthology.orchestra.run.trial044.stdout: re2-1:20211101-20.el9.x86_64 2026-02-12T09:09:44.308 INFO:teuthology.orchestra.run.trial044.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-12T09:09:44.308 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:09:44.308 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:09:44.347 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-test 2026-02-12T09:09:44.649 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:14 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:09:44.731 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:09:44.732 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:09:44.732 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repository Size 2026-02-12T09:09:44.732 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:09:44.732 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-12T09:09:44.732 INFO:teuthology.orchestra.run.trial044.stdout: ceph-test x86_64 2:20.3.0-5161.g2401a109.el9 ceph 126 M 2026-02-12T09:09:44.732 INFO:teuthology.orchestra.run.trial044.stdout:Installing dependencies: 2026-02-12T09:09:44.732 INFO:teuthology.orchestra.run.trial044.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-12T09:09:44.732 INFO:teuthology.orchestra.run.trial044.stdout: libasan8 x86_64 15.1.1-2.5.el9 appstream 523 k 2026-02-12T09:09:44.732 INFO:teuthology.orchestra.run.trial044.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-12T09:09:44.733 INFO:teuthology.orchestra.run.trial044.stdout: libcephsqlite x86_64 2:20.3.0-5161.g2401a109.el9 ceph 153 k 2026-02-12T09:09:44.733 INFO:teuthology.orchestra.run.trial044.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-12T09:09:44.733 INFO:teuthology.orchestra.run.trial044.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-12T09:09:44.733 INFO:teuthology.orchestra.run.trial044.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-12T09:09:44.733 INFO:teuthology.orchestra.run.trial044.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-12T09:09:44.733 INFO:teuthology.orchestra.run.trial044.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-12T09:09:44.733 INFO:teuthology.orchestra.run.trial044.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-12T09:09:44.733 INFO:teuthology.orchestra.run.trial044.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-12T09:09:44.733 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:09:44.733 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-12T09:09:44.734 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:09:44.734 INFO:teuthology.orchestra.run.trial044.stdout:Install 12 Packages 2026-02-12T09:09:44.734 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:09:44.734 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 131 M 2026-02-12T09:09:44.735 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 976 M 2026-02-12T09:09:44.735 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-12T09:09:45.666 INFO:teuthology.orchestra.run.trial044.stdout:(1/12): libcephsqlite-20.3.0-5161.g2401a109.el9 338 kB/s | 153 kB 00:00 2026-02-12T09:09:46.073 INFO:teuthology.orchestra.run.trial044.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 56 kB/s | 23 kB 00:00 2026-02-12T09:09:46.321 INFO:teuthology.orchestra.run.trial044.stdout:(3/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 379 kB/s | 94 kB 00:00 2026-02-12T09:09:46.937 INFO:teuthology.orchestra.run.trial044.stdout:(4/12): libasan8-15.1.1-2.5.el9.x86_64.rpm 850 kB/s | 523 kB 00:00 2026-02-12T09:09:47.170 INFO:teuthology.orchestra.run.trial044.stdout:(5/12): libubsan-11.5.0-14.el9.x86_64.rpm 755 kB/s | 176 kB 00:00 2026-02-12T09:09:47.478 INFO:teuthology.orchestra.run.trial044.stdout:(6/12): libxslt-1.1.34-12.el9.x86_64.rpm 755 kB/s | 233 kB 00:00 2026-02-12T09:09:47.960 INFO:teuthology.orchestra.run.trial044.stdout:(7/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 782 kB/s | 2.1 MB 00:02 2026-02-12T09:09:48.341 INFO:teuthology.orchestra.run.trial044.stdout:(8/12): socat-1.7.4.1-8.el9.x86_64.rpm 797 kB/s | 303 kB 00:00 2026-02-12T09:09:48.421 INFO:teuthology.orchestra.run.trial044.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 796 kB/s | 64 kB 00:00 2026-02-12T09:09:48.620 INFO:teuthology.orchestra.run.trial044.stdout:(10/12): protobuf-3.14.0-17.el9.x86_64.rpm 901 kB/s | 1.0 MB 00:01 2026-02-12T09:09:48.699 INFO:teuthology.orchestra.run.trial044.stdout:(11/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 438 kB/s | 122 kB 00:00 2026-02-12T09:10:07.679 INFO:teuthology.orchestra.run.trial044.stdout:(12/12): ceph-test-20.3.0-5161.g2401a109.el9.x8 5.6 MB/s | 126 MB 00:22 2026-02-12T09:10:07.682 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:10:07.682 INFO:teuthology.orchestra.run.trial044.stdout:Total 5.7 MB/s | 131 MB 00:22 2026-02-12T09:10:07.761 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-12T09:10:07.773 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-12T09:10:07.773 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-12T09:10:08.199 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-12T09:10:08.199 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-12T09:10:08.651 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-12T09:10:08.660 INFO:teuthology.orchestra.run.trial044.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-12T09:10:08.669 INFO:teuthology.orchestra.run.trial044.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-12T09:10:08.674 INFO:teuthology.orchestra.run.trial044.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-12T09:10:08.678 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-12T09:10:08.680 INFO:teuthology.orchestra.run.trial044.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-12T09:10:08.685 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-12T09:10:08.686 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libasan8-15.1.1-2.5.el9.x86_64 7/12 2026-02-12T09:10:08.703 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: libasan8-15.1.1-2.5.el9.x86_64 7/12 2026-02-12T09:10:08.705 INFO:teuthology.orchestra.run.trial044.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-12T09:10:08.717 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-12T09:10:08.727 INFO:teuthology.orchestra.run.trial044.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-12T09:10:08.728 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libcephsqlite-2:20.3.0-5161.g2401a109.el9.x86_64 11/12 2026-02-12T09:10:15.844 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5161.g2401a109.el9.x86_64 11/12 2026-02-12T09:10:15.863 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-test-2:20.3.0-5161.g2401a109.el9.x86_64 12/12 2026-02-12T09:10:16.643 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-test-2:20.3.0-5161.g2401a109.el9.x86_64 12/12 2026-02-12T09:10:16.644 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-test-2:20.3.0-5161.g2401a109.el9.x86_64 1/12 2026-02-12T09:10:16.644 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libcephsqlite-2:20.3.0-5161.g2401a109.el9.x86_64 2/12 2026-02-12T09:10:16.644 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-12T09:10:16.644 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-12T09:10:16.644 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-12T09:10:16.644 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libasan8-15.1.1-2.5.el9.x86_64 6/12 2026-02-12T09:10:16.644 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-12T09:10:16.644 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-12T09:10:16.644 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-12T09:10:16.644 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-12T09:10:16.644 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout: ceph-test-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout: libasan8-15.1.1-2.5.el9.x86_64 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout: libcephsqlite-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:16.698 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:16.742 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph 2026-02-12T09:10:17.041 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:47 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:17.127 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout:======================================================================================= 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repository Size 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout:======================================================================================= 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: ceph x86_64 2:20.3.0-5161.g2401a109.el9 ceph 6.5 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout:Installing dependencies: 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 57 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mds x86_64 2:20.3.0-5161.g2401a109.el9 ceph 2.0 M 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr x86_64 2:20.3.0-5161.g2401a109.el9 ceph 937 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 297 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mon x86_64 2:20.3.0-5161.g2401a109.el9 ceph 4.8 M 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: ceph-osd x86_64 2:20.3.0-5161.g2401a109.el9 ceph 163 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: ceph-osd-classic x86_64 2:20.3.0-5161.g2401a109.el9 ceph 15 M 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: ceph-osd-crimson x86_64 2:20.3.0-5161.g2401a109.el9 ceph 87 M 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 17 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: cephadm noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 1.0 M 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: flexiblas x86_64 3.0.4-8.el9 appstream 33 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: flexiblas-netlib x86_64 3.0.4-8.el9 appstream 3.0 M 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: flexiblas-openblas-openmp x86_64 3.0.4-8.el9 appstream 18 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-12T09:10:17.130 INFO:teuthology.orchestra.run.trial044.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-12T09:10:17.131 INFO:teuthology.orchestra.run.trial044.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout:Installing weak dependencies: 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 183 k 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 15 M 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 7.4 M 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 22 k 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-rook noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 50 k 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: ceph-volume noarch 2:20.3.0-5161.g2401a109.el9 ceph-noarch 299 k 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout:======================================================================================= 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout:Install 96 Packages 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 189 M 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 1.4 G 2026-02-12T09:10:17.132 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-12T09:10:18.112 INFO:teuthology.orchestra.run.trial044.stdout:(1/96): ceph-20.3.0-5161.g2401a109.el9.x86_64.r 33 kB/s | 6.5 kB 00:00 2026-02-12T09:10:18.471 INFO:teuthology.orchestra.run.trial044.stdout:(2/96): ceph-mgr-20.3.0-5161.g2401a109.el9.x86_ 1.6 MB/s | 937 kB 00:00 2026-02-12T09:10:18.537 INFO:teuthology.orchestra.run.trial044.stdout:(3/96): ceph-osd-20.3.0-5161.g2401a109.el9.x86_ 2.4 MB/s | 163 kB 00:00 2026-02-12T09:10:18.587 INFO:teuthology.orchestra.run.trial044.stdout:(4/96): ceph-mds-20.3.0-5161.g2401a109.el9.x86_ 3.0 MB/s | 2.0 MB 00:00 2026-02-12T09:10:19.391 INFO:teuthology.orchestra.run.trial044.stdout:(5/96): ceph-mon-20.3.0-5161.g2401a109.el9.x86_ 3.8 MB/s | 4.8 MB 00:01 2026-02-12T09:10:19.452 INFO:teuthology.orchestra.run.trial044.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5161.g24 956 kB/s | 57 kB 00:00 2026-02-12T09:10:19.515 INFO:teuthology.orchestra.run.trial044.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5161.g2401a109. 2.8 MB/s | 183 kB 00:00 2026-02-12T09:10:20.451 INFO:teuthology.orchestra.run.trial044.stdout:(8/96): ceph-osd-classic-20.3.0-5161.g2401a109. 7.7 MB/s | 15 MB 00:01 2026-02-12T09:10:21.183 INFO:teuthology.orchestra.run.trial044.stdout:(9/96): ceph-mgr-diskprediction-local-20.3.0-51 10 MB/s | 7.4 MB 00:00 2026-02-12T09:10:21.231 INFO:teuthology.orchestra.run.trial044.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5161.g2401a1 467 kB/s | 22 kB 00:00 2026-02-12T09:10:21.285 INFO:teuthology.orchestra.run.trial044.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5161.g240 5.4 MB/s | 297 kB 00:00 2026-02-12T09:10:21.335 INFO:teuthology.orchestra.run.trial044.stdout:(12/96): ceph-mgr-rook-20.3.0-5161.g2401a109.el 1.0 MB/s | 50 kB 00:00 2026-02-12T09:10:21.380 INFO:teuthology.orchestra.run.trial044.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5161.g24 381 kB/s | 17 kB 00:00 2026-02-12T09:10:21.431 INFO:teuthology.orchestra.run.trial044.stdout:(14/96): ceph-volume-20.3.0-5161.g2401a109.el9. 5.7 MB/s | 299 kB 00:00 2026-02-12T09:10:21.541 INFO:teuthology.orchestra.run.trial044.stdout:(15/96): cephadm-20.3.0-5161.g2401a109.el9.noar 9.3 MB/s | 1.0 MB 00:00 2026-02-12T09:10:22.106 INFO:teuthology.orchestra.run.trial044.stdout:(16/96): ceph-mgr-dashboard-20.3.0-5161.g2401a1 5.8 MB/s | 15 MB 00:02 2026-02-12T09:10:22.414 INFO:teuthology.orchestra.run.trial044.stdout:(17/96): cryptsetup-2.8.1-2.el9.x86_64.rpm 401 kB/s | 350 kB 00:00 2026-02-12T09:10:22.512 INFO:teuthology.orchestra.run.trial044.stdout:(18/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 100 kB/s | 40 kB 00:00 2026-02-12T09:10:22.584 INFO:teuthology.orchestra.run.trial044.stdout:(19/96): libconfig-1.7.2-9.el9.x86_64.rpm 422 kB/s | 72 kB 00:00 2026-02-12T09:10:22.830 INFO:teuthology.orchestra.run.trial044.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 751 kB/s | 184 kB 00:00 2026-02-12T09:10:23.152 INFO:teuthology.orchestra.run.trial044.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 787 kB/s | 253 kB 00:00 2026-02-12T09:10:23.516 INFO:teuthology.orchestra.run.trial044.stdout:(22/96): libgfortran-11.5.0-14.el9.x86_64.rpm 791 kB/s | 794 kB 00:01 2026-02-12T09:10:23.689 INFO:teuthology.orchestra.run.trial044.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 617 kB/s | 106 kB 00:00 2026-02-12T09:10:23.862 INFO:teuthology.orchestra.run.trial044.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 782 kB/s | 135 kB 00:00 2026-02-12T09:10:24.110 INFO:teuthology.orchestra.run.trial044.stdout:(25/96): python3-pyparsing-2.4.7-9.el9.noarch.r 606 kB/s | 150 kB 00:00 2026-02-12T09:10:24.284 INFO:teuthology.orchestra.run.trial044.stdout:(26/96): python3-requests-2.25.1-10.el9.noarch. 729 kB/s | 126 kB 00:00 2026-02-12T09:10:24.600 INFO:teuthology.orchestra.run.trial044.stdout:(27/96): python3-cryptography-36.0.1-5.el9.x86_ 881 kB/s | 1.2 MB 00:01 2026-02-12T09:10:24.608 INFO:teuthology.orchestra.run.trial044.stdout:(28/96): python3-urllib3-1.26.5-7.el9.noarch.rp 673 kB/s | 218 kB 00:00 2026-02-12T09:10:24.846 INFO:teuthology.orchestra.run.trial044.stdout:(29/96): unzip-6.0-59.el9.x86_64.rpm 740 kB/s | 182 kB 00:00 2026-02-12T09:10:24.856 INFO:teuthology.orchestra.run.trial044.stdout:(30/96): flexiblas-3.0.4-8.el9.x86_64.rpm 3.0 MB/s | 33 kB 00:00 2026-02-12T09:10:24.884 INFO:teuthology.orchestra.run.trial044.stdout:(31/96): flexiblas-netlib-3.0.4-8.el9.x86_64.rp 108 MB/s | 3.0 MB 00:00 2026-02-12T09:10:24.889 INFO:teuthology.orchestra.run.trial044.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-8.el9. 4.4 MB/s | 18 kB 00:00 2026-02-12T09:10:24.894 INFO:teuthology.orchestra.run.trial044.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 42 MB/s | 246 kB 00:00 2026-02-12T09:10:24.900 INFO:teuthology.orchestra.run.trial044.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 32 MB/s | 188 kB 00:00 2026-02-12T09:10:24.905 INFO:teuthology.orchestra.run.trial044.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 9.8 MB/s | 42 kB 00:00 2026-02-12T09:10:24.925 INFO:teuthology.orchestra.run.trial044.stdout:(36/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 264 MB/s | 5.3 MB 00:00 2026-02-12T09:10:24.949 INFO:teuthology.orchestra.run.trial044.stdout:(37/96): python3-babel-2.9.1-2.el9.noarch.rpm 247 MB/s | 6.0 MB 00:00 2026-02-12T09:10:24.951 INFO:teuthology.orchestra.run.trial044.stdout:(38/96): zip-3.0-35.el9.x86_64.rpm 775 kB/s | 266 kB 00:00 2026-02-12T09:10:24.956 INFO:teuthology.orchestra.run.trial044.stdout:(39/96): python3-devel-3.9.25-2.el9.x86_64.rpm 34 MB/s | 244 kB 00:00 2026-02-12T09:10:24.961 INFO:teuthology.orchestra.run.trial044.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 9.6 MB/s | 48 kB 00:00 2026-02-12T09:10:24.967 INFO:teuthology.orchestra.run.trial044.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 30 MB/s | 177 kB 00:00 2026-02-12T09:10:24.972 INFO:teuthology.orchestra.run.trial044.stdout:(42/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 11 MB/s | 249 kB 00:00 2026-02-12T09:10:24.977 INFO:teuthology.orchestra.run.trial044.stdout:(43/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 130 MB/s | 1.2 MB 00:00 2026-02-12T09:10:24.977 INFO:teuthology.orchestra.run.trial044.stdout:(44/96): python3-markupsafe-1.1.1-12.el9.x86_64 7.0 MB/s | 35 kB 00:00 2026-02-12T09:10:24.993 INFO:teuthology.orchestra.run.trial044.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 28 MB/s | 442 kB 00:00 2026-02-12T09:10:25.002 INFO:teuthology.orchestra.run.trial044.stdout:(46/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 242 MB/s | 6.1 MB 00:00 2026-02-12T09:10:25.003 INFO:teuthology.orchestra.run.trial044.stdout:(47/96): python3-packaging-20.9-5.el9.noarch.rp 8.1 MB/s | 77 kB 00:00 2026-02-12T09:10:25.009 INFO:teuthology.orchestra.run.trial044.stdout:(48/96): python3-protobuf-3.14.0-17.el9.noarch. 37 MB/s | 267 kB 00:00 2026-02-12T09:10:25.010 INFO:teuthology.orchestra.run.trial044.stdout:(49/96): python3-pyasn1-0.4.8-6.el9.noarch.rpm 22 MB/s | 159 kB 00:00 2026-02-12T09:10:25.015 INFO:teuthology.orchestra.run.trial044.stdout:(50/96): python3-requests-oauthlib-1.3.0-12.el9 10 MB/s | 54 kB 00:00 2026-02-12T09:10:25.016 INFO:teuthology.orchestra.run.trial044.stdout:(51/96): python3-pyasn1-modules-0.4.8-6.el9.noa 39 MB/s | 279 kB 00:00 2026-02-12T09:10:25.021 INFO:teuthology.orchestra.run.trial044.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 8.4 MB/s | 42 kB 00:00 2026-02-12T09:10:25.027 INFO:teuthology.orchestra.run.trial044.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 34 MB/s | 189 kB 00:00 2026-02-12T09:10:25.032 INFO:teuthology.orchestra.run.trial044.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 18 MB/s | 90 kB 00:00 2026-02-12T09:10:25.110 INFO:teuthology.orchestra.run.trial044.stdout:(55/96): lua-devel-5.4.4-4.el9.x86_64.rpm 285 kB/s | 22 kB 00:00 2026-02-12T09:10:25.143 INFO:teuthology.orchestra.run.trial044.stdout:(56/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 151 MB/s | 19 MB 00:00 2026-02-12T09:10:25.269 INFO:teuthology.orchestra.run.trial044.stdout:(57/96): protobuf-compiler-3.14.0-17.el9.x86_64 5.3 MB/s | 862 kB 00:00 2026-02-12T09:10:25.511 INFO:teuthology.orchestra.run.trial044.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 80 kB/s | 19 kB 00:00 2026-02-12T09:10:25.621 INFO:teuthology.orchestra.run.trial044.stdout:(59/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.1 MB/s | 551 kB 00:00 2026-02-12T09:10:25.725 INFO:teuthology.orchestra.run.trial044.stdout:(60/96): python3-asyncssh-2.13.2-5.el9.noarch.r 5.1 MB/s | 548 kB 00:00 2026-02-12T09:10:25.788 INFO:teuthology.orchestra.run.trial044.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 468 kB/s | 29 kB 00:00 2026-02-12T09:10:25.803 INFO:teuthology.orchestra.run.trial044.stdout:(62/96): luarocks-3.9.2-5.el9.noarch.rpm 516 kB/s | 151 kB 00:00 2026-02-12T09:10:25.852 INFO:teuthology.orchestra.run.trial044.stdout:(63/96): python3-backports-tarfile-1.2.0-1.el9. 929 kB/s | 60 kB 00:00 2026-02-12T09:10:25.898 INFO:teuthology.orchestra.run.trial044.stdout:(64/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 460 kB/s | 43 kB 00:00 2026-02-12T09:10:25.916 INFO:teuthology.orchestra.run.trial044.stdout:(65/96): python3-cachetools-4.2.4-1.el9.noarch. 510 kB/s | 32 kB 00:00 2026-02-12T09:10:25.983 INFO:teuthology.orchestra.run.trial044.stdout:(66/96): python3-certifi-2023.05.07-4.el9.noarc 166 kB/s | 14 kB 00:00 2026-02-12T09:10:25.988 INFO:teuthology.orchestra.run.trial044.stdout:(67/96): python3-cheroot-10.0.1-4.el9.noarch.rp 2.3 MB/s | 173 kB 00:00 2026-02-12T09:10:26.066 INFO:teuthology.orchestra.run.trial044.stdout:(68/96): python3-google-auth-2.45.0-1.el9.noarc 3.2 MB/s | 254 kB 00:00 2026-02-12T09:10:26.184 INFO:teuthology.orchestra.run.trial044.stdout:(69/96): python3-cherrypy-18.6.1-2.el9.noarch.r 1.7 MB/s | 358 kB 00:00 2026-02-12T09:10:26.253 INFO:teuthology.orchestra.run.trial044.stdout:(70/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 11 MB/s | 2.0 MB 00:00 2026-02-12T09:10:26.281 INFO:teuthology.orchestra.run.trial044.stdout:(71/96): python3-grpcio-tools-1.46.7-10.el9.x86 1.5 MB/s | 144 kB 00:00 2026-02-12T09:10:26.420 INFO:teuthology.orchestra.run.trial044.stdout:(72/96): ceph-osd-crimson-20.3.0-5161.g2401a109 11 MB/s | 87 MB 00:07 2026-02-12T09:10:26.421 INFO:teuthology.orchestra.run.trial044.stdout:(73/96): python3-isodate-0.6.1-3.el9.noarch.rpm 399 kB/s | 56 kB 00:00 2026-02-12T09:10:26.422 INFO:teuthology.orchestra.run.trial044.stdout:(74/96): python3-influxdb-5.3.1-1.el9.noarch.rp 822 kB/s | 139 kB 00:00 2026-02-12T09:10:26.484 INFO:teuthology.orchestra.run.trial044.stdout:(75/96): python3-jaraco-collections-3.0.0-8.el9 375 kB/s | 23 kB 00:00 2026-02-12T09:10:26.504 INFO:teuthology.orchestra.run.trial044.stdout:(76/96): python3-jaraco-classes-3.2.1-5.el9.noa 216 kB/s | 18 kB 00:00 2026-02-12T09:10:26.543 INFO:teuthology.orchestra.run.trial044.stdout:(77/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 87 kB/s | 11 kB 00:00 2026-02-12T09:10:26.546 INFO:teuthology.orchestra.run.trial044.stdout:(78/96): python3-jaraco-context-6.0.1-3.el9.noa 317 kB/s | 20 kB 00:00 2026-02-12T09:10:26.585 INFO:teuthology.orchestra.run.trial044.stdout:(79/96): python3-jaraco-functools-3.5.0-2.el9.n 238 kB/s | 19 kB 00:00 2026-02-12T09:10:26.627 INFO:teuthology.orchestra.run.trial044.stdout:(80/96): python3-kubernetes-26.1.0-3.el9.noarch 13 MB/s | 1.0 MB 00:00 2026-02-12T09:10:26.666 INFO:teuthology.orchestra.run.trial044.stdout:(81/96): python3-jaraco-text-4.0.0-2.el9.noarch 216 kB/s | 26 kB 00:00 2026-02-12T09:10:26.675 INFO:teuthology.orchestra.run.trial044.stdout:(82/96): python3-more-itertools-8.12.0-2.el9.no 883 kB/s | 79 kB 00:00 2026-02-12T09:10:26.691 INFO:teuthology.orchestra.run.trial044.stdout:(83/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 1.3 MB/s | 86 kB 00:00 2026-02-12T09:10:26.754 INFO:teuthology.orchestra.run.trial044.stdout:(84/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.4 MB/s | 90 kB 00:00 2026-02-12T09:10:26.756 INFO:teuthology.orchestra.run.trial044.stdout:(85/96): python3-portend-3.1.0-2.el9.noarch.rpm 201 kB/s | 16 kB 00:00 2026-02-12T09:10:26.789 INFO:teuthology.orchestra.run.trial044.stdout:(86/96): python3-natsort-7.1.1-5.el9.noarch.rpm 467 kB/s | 58 kB 00:00 2026-02-12T09:10:26.816 INFO:teuthology.orchestra.run.trial044.stdout:(87/96): python3-repoze-lru-0.7-16.el9.noarch.r 493 kB/s | 31 kB 00:00 2026-02-12T09:10:26.858 INFO:teuthology.orchestra.run.trial044.stdout:(88/96): python3-routes-2.5.1-5.el9.noarch.rpm 1.8 MB/s | 188 kB 00:00 2026-02-12T09:10:26.874 INFO:teuthology.orchestra.run.trial044.stdout:(89/96): python3-rsa-4.9-2.el9.noarch.rpm 703 kB/s | 59 kB 00:00 2026-02-12T09:10:26.880 INFO:teuthology.orchestra.run.trial044.stdout:(90/96): python3-saml-1.16.0-1.el9.noarch.rpm 1.9 MB/s | 125 kB 00:00 2026-02-12T09:10:26.943 INFO:teuthology.orchestra.run.trial044.stdout:(91/96): python3-websocket-client-1.2.3-2.el9.n 1.4 MB/s | 90 kB 00:00 2026-02-12T09:10:26.943 INFO:teuthology.orchestra.run.trial044.stdout:(92/96): python3-tempora-5.0.0-2.el9.noarch.rpm 421 kB/s | 36 kB 00:00 2026-02-12T09:10:26.970 INFO:teuthology.orchestra.run.trial044.stdout:(93/96): python3-typing-extensions-4.15.0-1.el9 901 kB/s | 86 kB 00:00 2026-02-12T09:10:27.006 INFO:teuthology.orchestra.run.trial044.stdout:(94/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 773 kB/s | 48 kB 00:00 2026-02-12T09:10:27.027 INFO:teuthology.orchestra.run.trial044.stdout:(95/96): python3-xmltodict-0.12.0-15.el9.noarch 265 kB/s | 22 kB 00:00 2026-02-12T09:10:27.038 INFO:teuthology.orchestra.run.trial044.stdout:(96/96): python3-zc-lockfile-2.0-10.el9.noarch. 291 kB/s | 20 kB 00:00 2026-02-12T09:10:27.040 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:10:27.041 INFO:teuthology.orchestra.run.trial044.stdout:Total 19 MB/s | 189 MB 00:09 2026-02-12T09:10:27.469 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-12T09:10:27.509 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-12T09:10:27.510 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-12T09:10:28.199 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-12T09:10:28.199 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-12T09:10:28.919 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-12T09:10:28.921 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-12T09:10:28.925 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-12T09:10:28.946 INFO:teuthology.orchestra.run.trial044.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-12T09:10:28.979 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 4/96 2026-02-12T09:10:28.986 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-12T09:10:28.988 INFO:teuthology.orchestra.run.trial044.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-12T09:10:28.993 INFO:teuthology.orchestra.run.trial044.stdout: Installing : flexiblas-3.0.4-8.el9.x86_64 7/96 2026-02-12T09:10:29.016 INFO:teuthology.orchestra.run.trial044.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-12T09:10:29.029 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-12T09:10:29.032 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-12T09:10:29.038 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-12T09:10:29.040 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-12T09:10:29.051 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-12T09:10:29.057 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-12T09:10:29.081 INFO:teuthology.orchestra.run.trial044.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-12T09:10:29.085 INFO:teuthology.orchestra.run.trial044.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-12T09:10:29.135 INFO:teuthology.orchestra.run.trial044.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-12T09:10:29.148 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 18/96 2026-02-12T09:10:29.150 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-12T09:10:29.154 INFO:teuthology.orchestra.run.trial044.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-12T09:10:29.160 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-12T09:10:29.164 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-12T09:10:29.167 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-12T09:10:29.182 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-12T09:10:29.186 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-12T09:10:29.191 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-12T09:10:29.203 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-12T09:10:29.212 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-12T09:10:29.218 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-12T09:10:29.238 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-12T09:10:29.249 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-12T09:10:29.264 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-12T09:10:29.269 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-12T09:10:29.276 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-12T09:10:29.316 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-12T09:10:29.640 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-12T09:10:29.650 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-12T09:10:29.654 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-12T09:10:29.660 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-12T09:10:29.662 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-12T09:10:29.689 INFO:teuthology.orchestra.run.trial044.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-12T09:10:29.733 INFO:teuthology.orchestra.run.trial044.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-12T09:10:29.743 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-12T09:10:29.749 INFO:teuthology.orchestra.run.trial044.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-12T09:10:29.752 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-12T09:10:29.758 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-12T09:10:29.762 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-12T09:10:29.768 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-12T09:10:29.772 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-12T09:10:29.800 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-12T09:10:29.810 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-12T09:10:29.816 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-12T09:10:29.822 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-12T09:10:29.857 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-12T09:10:30.081 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 55/96 2026-02-12T09:10:30.105 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-12T09:10:30.109 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-12T09:10:30.167 INFO:teuthology.orchestra.run.trial044.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-12T09:10:30.169 INFO:teuthology.orchestra.run.trial044.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-12T09:10:30.190 INFO:teuthology.orchestra.run.trial044.stdout: Installing : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 60/96 2026-02-12T09:10:30.515 INFO:teuthology.orchestra.run.trial044.stdout: Installing : flexiblas-netlib-3.0.4-8.el9.x86_64 61/96 2026-02-12T09:10:30.588 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-12T09:10:31.259 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-12T09:10:31.277 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-12T09:10:31.286 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-12T09:10:31.293 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-12T09:10:31.306 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-12T09:10:31.322 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-12T09:10:31.397 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-12T09:10:31.406 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-12T09:10:31.430 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-12T09:10:31.459 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-12T09:10:31.510 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-12T09:10:31.516 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-12T09:10:31.519 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-12T09:10:31.521 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-12T09:10:31.540 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-12T09:10:31.540 INFO:teuthology.orchestra.run.trial044.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-12T09:10:31.540 INFO:teuthology.orchestra.run.trial044.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-12T09:10:31.540 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:31.547 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-12T09:10:31.573 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-12T09:10:31.573 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-12T09:10:31.574 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:31.609 INFO:teuthology.orchestra.run.trial044.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-12T09:10:32.212 INFO:teuthology.orchestra.run.trial044.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 79/96 2026-02-12T09:10:37.638 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-osd-classic-2:20.3.0-5161.g2401a109.el9.x86_6 80/96 2026-02-12T09:10:37.702 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-osd-crimson-2:20.3.0-5161.g2401a109.el9.x86_6 81/96 2026-02-12T09:10:37.704 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-volume-2:20.3.0-5161.g2401a109.el9.noarch 82/96 2026-02-12T09:10:37.719 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-volume-2:20.3.0-5161.g2401a109.el9.noarch 82/96 2026-02-12T09:10:37.719 INFO:teuthology.orchestra.run.trial044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T09:10:37.719 INFO:teuthology.orchestra.run.trial044.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-12T09:10:37.719 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:37.729 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-osd-2:20.3.0-5161.g2401a109.el9.x86_64 83/96 2026-02-12T09:10:37.754 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-osd-2:20.3.0-5161.g2401a109.el9.x86_64 83/96 2026-02-12T09:10:37.754 INFO:teuthology.orchestra.run.trial044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T09:10:37.755 INFO:teuthology.orchestra.run.trial044.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-12T09:10:37.755 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-12T09:10:37.755 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-12T09:10:37.755 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:37.819 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: cephadm-2:20.3.0-5161.g2401a109.el9.noarch 84/96 2026-02-12T09:10:37.821 INFO:teuthology.orchestra.run.trial044.stdout: Installing : cephadm-2:20.3.0-5161.g2401a109.el9.noarch 84/96 2026-02-12T09:10:37.830 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5161.g2401a109.el9 85/96 2026-02-12T09:10:37.854 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5161.g2401a109.el 86/96 2026-02-12T09:10:37.855 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5161.g2401a109.el9.noarc 87/96 2026-02-12T09:10:39.448 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5161.g2401a109.el9.noarc 87/96 2026-02-12T09:10:39.455 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5161.g2401a109.el9.noa 88/96 2026-02-12T09:10:39.915 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5161.g2401a109.el9.noa 88/96 2026-02-12T09:10:39.916 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5161.g2401a 89/96 2026-02-12T09:10:39.929 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5161.g2401a 89/96 2026-02-12T09:10:39.930 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5161.g2401a109.el9.noa 90/96 2026-02-12T09:10:39.993 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5161.g2401a109.el9.noa 90/96 2026-02-12T09:10:40.034 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5161.g2401a109.el9. 91/96 2026-02-12T09:10:40.035 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mgr-2:20.3.0-5161.g2401a109.el9.x86_64 92/96 2026-02-12T09:10:40.057 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5161.g2401a109.el9.x86_64 92/96 2026-02-12T09:10:40.057 INFO:teuthology.orchestra.run.trial044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T09:10:40.057 INFO:teuthology.orchestra.run.trial044.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-12T09:10:40.058 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-12T09:10:40.058 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-12T09:10:40.058 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:40.069 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mgr-rook-2:20.3.0-5161.g2401a109.el9.noarch 93/96 2026-02-12T09:10:40.287 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5161.g2401a109.el9.noarch 93/96 2026-02-12T09:10:40.289 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mon-2:20.3.0-5161.g2401a109.el9.x86_64 94/96 2026-02-12T09:10:40.310 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-mon-2:20.3.0-5161.g2401a109.el9.x86_64 94/96 2026-02-12T09:10:40.310 INFO:teuthology.orchestra.run.trial044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T09:10:40.311 INFO:teuthology.orchestra.run.trial044.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-12T09:10:40.311 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-12T09:10:40.311 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-12T09:10:40.311 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:40.394 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-mds-2:20.3.0-5161.g2401a109.el9.x86_64 95/96 2026-02-12T09:10:40.416 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-mds-2:20.3.0-5161.g2401a109.el9.x86_64 95/96 2026-02-12T09:10:40.416 INFO:teuthology.orchestra.run.trial044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T09:10:40.416 INFO:teuthology.orchestra.run.trial044.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-12T09:10:40.416 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-12T09:10:40.416 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-12T09:10:40.416 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:40.417 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-2:20.3.0-5161.g2401a109.el9.x86_64 96/96 2026-02-12T09:10:40.424 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5161.g2401a109.el9.x86_6 96/96 2026-02-12T09:10:40.449 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5161.g2401a109.el9.x86_6 96/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-2:20.3.0-5161.g2401a109.el9.x86_64 96/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-2:20.3.0-5161.g2401a109.el9.x86_64 1/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mds-2:20.3.0-5161.g2401a109.el9.x86_64 2/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mgr-2:20.3.0-5161.g2401a109.el9.x86_64 3/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mon-2:20.3.0-5161.g2401a109.el9.x86_64 4/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-osd-2:20.3.0-5161.g2401a109.el9.x86_64 5/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-osd-classic-2:20.3.0-5161.g2401a109.el9.x86_6 6/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5161.g2401a109.el9.x86_6 7/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5161.g2401a109.el 8/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5161.g2401a109.el9.noarc 9/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5161.g2401a109.el9.noa 10/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5161.g2401a 11/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5161.g2401a109.el9.noa 12/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5161.g2401a109.el9. 13/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5161.g2401a109.el9.noarch 14/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5161.g2401a109.el9 15/96 2026-02-12T09:10:41.756 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-volume-2:20.3.0-5161.g2401a109.el9.noarch 16/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : cephadm-2:20.3.0-5161.g2401a109.el9.noarch 17/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 18/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : flexiblas-3.0.4-8.el9.x86_64 32/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : flexiblas-netlib-3.0.4-8.el9.x86_64 33/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 34/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 40/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 50/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 51/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-12T09:10:41.757 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-12T09:10:41.758 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ceph-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ceph-grafana-dashboards-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mds-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-cephadm-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-dashboard-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-k8sevents-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-modules-core-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mgr-rook-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ceph-mon-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ceph-osd-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ceph-osd-classic-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ceph-osd-crimson-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ceph-prometheus-alerts-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ceph-volume-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: cephadm-2:20.3.0-5161.g2401a109.el9.noarch 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: flexiblas-3.0.4-8.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: flexiblas-netlib-3.0.4-8.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: flexiblas-openblas-openmp-3.0.4-8.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: lua-5.4.4-4.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-12T09:10:41.825 INFO:teuthology.orchestra.run.trial044.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-ply-3.11-14.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-12T09:10:41.826 INFO:teuthology.orchestra.run.trial044.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: unzip-6.0-59.el9.x86_64 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: zip-3.0-35.el9.x86_64 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:41.827 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:41.893 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-base 2026-02-12T09:10:42.206 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:12 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:42.271 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-base-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:10:42.290 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:42.290 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:42.290 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:42.313 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install cephadm 2026-02-12T09:10:42.611 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:12 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:42.678 INFO:teuthology.orchestra.run.trial044.stdout:Package cephadm-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:10:42.696 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:42.697 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:42.697 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:42.717 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-immutable-object-cache 2026-02-12T09:10:43.016 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:13 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:43.099 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:43.100 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:43.100 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repo Size 2026-02-12T09:10:43.100 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:43.100 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-12T09:10:43.100 INFO:teuthology.orchestra.run.trial044.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5161.g2401a109.el9 ceph 149 k 2026-02-12T09:10:43.100 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:43.101 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-12T09:10:43.101 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:43.101 INFO:teuthology.orchestra.run.trial044.stdout:Install 1 Package 2026-02-12T09:10:43.101 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:43.101 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 149 k 2026-02-12T09:10:43.101 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 455 k 2026-02-12T09:10:43.101 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-12T09:10:43.489 INFO:teuthology.orchestra.run.trial044.stdout:ceph-immutable-object-cache-20.3.0-5161.g2401a1 383 kB/s | 149 kB 00:00 2026-02-12T09:10:43.490 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:10:43.490 INFO:teuthology.orchestra.run.trial044.stdout:Total 382 kB/s | 149 kB 00:00 2026-02-12T09:10:43.490 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-12T09:10:43.495 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-12T09:10:43.495 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-12T09:10:43.526 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-12T09:10:43.526 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-12T09:10:43.582 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-12T09:10:43.584 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5161.g2401a109. 1/1 2026-02-12T09:10:43.606 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5161.g2401a109. 1/1 2026-02-12T09:10:43.606 INFO:teuthology.orchestra.run.trial044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T09:10:43.606 INFO:teuthology.orchestra.run.trial044.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-12T09:10:43.606 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:43.915 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5161.g2401a109. 1/1 2026-02-12T09:10:43.915 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:43.915 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-12T09:10:43.915 INFO:teuthology.orchestra.run.trial044.stdout: ceph-immutable-object-cache-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:10:43.915 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:43.915 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:43.943 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-mgr 2026-02-12T09:10:44.252 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:14 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:44.317 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-mgr-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:10:44.336 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:44.336 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:44.336 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:44.358 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-mgr-dashboard 2026-02-12T09:10:44.660 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:14 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:44.724 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-mgr-dashboard-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:10:44.742 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:44.743 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:44.743 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:44.764 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-12T09:10:45.072 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:15 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:45.138 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:10:45.157 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:45.171 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:45.171 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:45.178 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-mgr-rook 2026-02-12T09:10:45.477 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:15 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:45.541 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-mgr-rook-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:10:45.559 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:45.560 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:45.560 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:45.580 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-mgr-cephadm 2026-02-12T09:10:45.881 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:15 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:45.946 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-mgr-cephadm-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:10:45.965 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:45.966 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:45.966 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:45.986 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-osd 2026-02-12T09:10:46.286 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:16 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:46.351 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-osd-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:10:46.370 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:46.370 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:46.370 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:46.391 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-osd-classic 2026-02-12T09:10:46.684 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:16 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:46.751 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-osd-classic-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:10:46.770 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:46.771 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:46.771 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:46.791 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-fuse 2026-02-12T09:10:47.094 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:17 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:47.178 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:47.179 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:47.180 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repo Size 2026-02-12T09:10:47.180 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:47.180 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-12T09:10:47.180 INFO:teuthology.orchestra.run.trial044.stdout: ceph-fuse x86_64 2:20.3.0-5161.g2401a109.el9 ceph 914 k 2026-02-12T09:10:47.180 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:47.180 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-12T09:10:47.180 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:47.180 INFO:teuthology.orchestra.run.trial044.stdout:Install 1 Package 2026-02-12T09:10:47.180 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:47.180 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 914 k 2026-02-12T09:10:47.180 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 2.9 M 2026-02-12T09:10:47.181 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-12T09:10:47.888 INFO:teuthology.orchestra.run.trial044.stdout:ceph-fuse-20.3.0-5161.g2401a109.el9.x86_64.rpm 1.3 MB/s | 914 kB 00:00 2026-02-12T09:10:47.888 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:10:47.888 INFO:teuthology.orchestra.run.trial044.stdout:Total 1.3 MB/s | 914 kB 00:00 2026-02-12T09:10:47.888 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-12T09:10:47.893 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-12T09:10:47.893 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-12T09:10:47.928 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-12T09:10:47.928 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-12T09:10:48.018 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-12T09:10:48.035 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:48.251 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:48.306 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:48.307 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:48.307 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-12T09:10:48.307 INFO:teuthology.orchestra.run.trial044.stdout: ceph-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:10:48.307 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:48.307 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:48.334 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-volume 2026-02-12T09:10:48.635 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:18 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:48.701 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-volume-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:10:48.718 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:48.719 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:48.719 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:48.740 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install librados-devel 2026-02-12T09:10:49.044 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:19 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:49.127 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:49.127 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:49.128 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repo Size 2026-02-12T09:10:49.128 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:49.128 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-12T09:10:49.128 INFO:teuthology.orchestra.run.trial044.stdout: librados-devel x86_64 2:20.3.0-5161.g2401a109.el9 ceph 124 k 2026-02-12T09:10:49.128 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:49.128 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-12T09:10:49.128 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:49.128 INFO:teuthology.orchestra.run.trial044.stdout:Install 1 Package 2026-02-12T09:10:49.128 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:49.128 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 124 k 2026-02-12T09:10:49.128 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 434 k 2026-02-12T09:10:49.129 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-12T09:10:49.553 INFO:teuthology.orchestra.run.trial044.stdout:librados-devel-20.3.0-5161.g2401a109.el9.x86_64 291 kB/s | 124 kB 00:00 2026-02-12T09:10:49.553 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:10:49.554 INFO:teuthology.orchestra.run.trial044.stdout:Total 290 kB/s | 124 kB 00:00 2026-02-12T09:10:49.554 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-12T09:10:49.556 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-12T09:10:49.556 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-12T09:10:49.589 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-12T09:10:49.589 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-12T09:10:49.644 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-12T09:10:49.660 INFO:teuthology.orchestra.run.trial044.stdout: Installing : librados-devel-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:49.740 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: librados-devel-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:49.797 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librados-devel-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:49.797 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:49.797 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-12T09:10:49.797 INFO:teuthology.orchestra.run.trial044.stdout: librados-devel-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:10:49.797 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:49.797 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:49.824 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install libcephfs2 2026-02-12T09:10:50.125 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:20 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:50.190 INFO:teuthology.orchestra.run.trial044.stdout:Package libcephfs2-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:10:50.207 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:50.207 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:50.207 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:50.228 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install libcephfs-devel 2026-02-12T09:10:50.528 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:20 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:50.611 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:50.612 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:50.612 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repo Size 2026-02-12T09:10:50.612 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:50.612 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-12T09:10:50.612 INFO:teuthology.orchestra.run.trial044.stdout: libcephfs-devel x86_64 2:20.3.0-5161.g2401a109.el9 ceph 37 k 2026-02-12T09:10:50.612 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:50.612 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-12T09:10:50.612 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:50.612 INFO:teuthology.orchestra.run.trial044.stdout:Install 1 Package 2026-02-12T09:10:50.613 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:50.613 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 37 k 2026-02-12T09:10:50.613 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 171 k 2026-02-12T09:10:50.613 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-12T09:10:50.898 INFO:teuthology.orchestra.run.trial044.stdout:libcephfs-devel-20.3.0-5161.g2401a109.el9.x86_6 130 kB/s | 37 kB 00:00 2026-02-12T09:10:50.899 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:10:50.899 INFO:teuthology.orchestra.run.trial044.stdout:Total 130 kB/s | 37 kB 00:00 2026-02-12T09:10:50.899 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-12T09:10:50.901 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-12T09:10:50.901 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-12T09:10:50.904 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-12T09:10:50.904 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-12T09:10:50.926 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-12T09:10:50.942 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libcephfs-devel-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:50.997 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:51.053 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libcephfs-devel-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:51.053 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:51.053 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-12T09:10:51.053 INFO:teuthology.orchestra.run.trial044.stdout: libcephfs-devel-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:10:51.053 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:51.053 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:51.080 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install librados2 2026-02-12T09:10:51.385 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:21 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:51.452 INFO:teuthology.orchestra.run.trial044.stdout:Package librados2-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:10:51.469 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:51.470 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:51.470 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:51.492 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install librbd1 2026-02-12T09:10:51.796 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:21 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:51.861 INFO:teuthology.orchestra.run.trial044.stdout:Package librbd1-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:10:51.878 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:51.879 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:51.879 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:51.901 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install python3-rados 2026-02-12T09:10:52.209 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:22 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:52.274 INFO:teuthology.orchestra.run.trial044.stdout:Package python3-rados-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:10:52.291 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:52.291 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:52.292 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:52.313 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install python3-rgw 2026-02-12T09:10:52.613 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:22 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:52.678 INFO:teuthology.orchestra.run.trial044.stdout:Package python3-rgw-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:10:52.695 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:52.695 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:52.696 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:52.717 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install python3-cephfs 2026-02-12T09:10:53.018 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:23 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:53.084 INFO:teuthology.orchestra.run.trial044.stdout:Package python3-cephfs-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:10:53.102 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:53.103 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:53.103 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:53.125 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install python3-rbd 2026-02-12T09:10:53.425 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:23 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:53.489 INFO:teuthology.orchestra.run.trial044.stdout:Package python3-rbd-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:10:53.506 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:53.507 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:53.507 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:53.527 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install rbd-fuse 2026-02-12T09:10:53.831 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:23 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:53.915 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:53.915 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:53.916 INFO:teuthology.orchestra.run.trial044.stdout: Package Architecture Version Repository Size 2026-02-12T09:10:53.916 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:53.916 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-12T09:10:53.916 INFO:teuthology.orchestra.run.trial044.stdout: rbd-fuse x86_64 2:20.3.0-5161.g2401a109.el9 ceph 87 k 2026-02-12T09:10:53.916 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:53.916 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-12T09:10:53.916 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:53.916 INFO:teuthology.orchestra.run.trial044.stdout:Install 1 Package 2026-02-12T09:10:53.916 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:53.916 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 87 k 2026-02-12T09:10:53.917 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 236 k 2026-02-12T09:10:53.917 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-12T09:10:54.319 INFO:teuthology.orchestra.run.trial044.stdout:rbd-fuse-20.3.0-5161.g2401a109.el9.x86_64.rpm 214 kB/s | 87 kB 00:00 2026-02-12T09:10:54.320 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:10:54.320 INFO:teuthology.orchestra.run.trial044.stdout:Total 214 kB/s | 87 kB 00:00 2026-02-12T09:10:54.320 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-12T09:10:54.324 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-12T09:10:54.324 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-12T09:10:54.357 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-12T09:10:54.357 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-12T09:10:54.408 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-12T09:10:54.425 INFO:teuthology.orchestra.run.trial044.stdout: Installing : rbd-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:54.506 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:54.564 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : rbd-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:54.564 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:54.564 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-12T09:10:54.564 INFO:teuthology.orchestra.run.trial044.stdout: rbd-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:10:54.564 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:54.564 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:54.591 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install rbd-mirror 2026-02-12T09:10:54.897 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:24 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:54.979 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:54.980 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:54.980 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repo Size 2026-02-12T09:10:54.980 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:54.981 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-12T09:10:54.981 INFO:teuthology.orchestra.run.trial044.stdout: rbd-mirror x86_64 2:20.3.0-5161.g2401a109.el9 ceph 2.6 M 2026-02-12T09:10:54.981 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:54.981 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-12T09:10:54.981 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:54.981 INFO:teuthology.orchestra.run.trial044.stdout:Install 1 Package 2026-02-12T09:10:54.981 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:54.981 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 2.6 M 2026-02-12T09:10:54.981 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 11 M 2026-02-12T09:10:54.981 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-12T09:10:56.260 INFO:teuthology.orchestra.run.trial044.stdout:rbd-mirror-20.3.0-5161.g2401a109.el9.x86_64.rpm 2.0 MB/s | 2.6 MB 00:01 2026-02-12T09:10:56.261 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:10:56.261 INFO:teuthology.orchestra.run.trial044.stdout:Total 2.0 MB/s | 2.6 MB 00:01 2026-02-12T09:10:56.262 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-12T09:10:56.266 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-12T09:10:56.267 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-12T09:10:56.306 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-12T09:10:56.306 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-12T09:10:56.470 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-12T09:10:56.472 INFO:teuthology.orchestra.run.trial044.stdout: Installing : rbd-mirror-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:56.495 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:56.495 INFO:teuthology.orchestra.run.trial044.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-12T09:10:56.496 INFO:teuthology.orchestra.run.trial044.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-12T09:10:56.496 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-12T09:10:56.496 INFO:teuthology.orchestra.run.trial044.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-12T09:10:56.496 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:56.774 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : rbd-mirror-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:56.774 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:56.775 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-12T09:10:56.775 INFO:teuthology.orchestra.run.trial044.stdout: rbd-mirror-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:10:56.775 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:56.775 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:56.803 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install rbd-nbd 2026-02-12T09:10:57.107 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:27 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:57.190 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:57.191 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:57.191 INFO:teuthology.orchestra.run.trial044.stdout: Package Architecture Version Repository Size 2026-02-12T09:10:57.191 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:57.191 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-12T09:10:57.191 INFO:teuthology.orchestra.run.trial044.stdout: rbd-nbd x86_64 2:20.3.0-5161.g2401a109.el9 ceph 166 k 2026-02-12T09:10:57.191 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:57.191 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-12T09:10:57.191 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:57.192 INFO:teuthology.orchestra.run.trial044.stdout:Install 1 Package 2026-02-12T09:10:57.192 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:57.192 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 166 k 2026-02-12T09:10:57.192 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 496 k 2026-02-12T09:10:57.192 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-12T09:10:57.643 INFO:teuthology.orchestra.run.trial044.stdout:rbd-nbd-20.3.0-5161.g2401a109.el9.x86_64.rpm 368 kB/s | 166 kB 00:00 2026-02-12T09:10:57.644 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:10:57.644 INFO:teuthology.orchestra.run.trial044.stdout:Total 368 kB/s | 166 kB 00:00 2026-02-12T09:10:57.644 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-12T09:10:57.648 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-12T09:10:57.648 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-12T09:10:57.680 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-12T09:10:57.681 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-12T09:10:57.736 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-12T09:10:57.753 INFO:teuthology.orchestra.run.trial044.stdout: Installing : rbd-nbd-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:57.836 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:57.893 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : rbd-nbd-2:20.3.0-5161.g2401a109.el9.x86_64 1/1 2026-02-12T09:10:57.893 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:57.894 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-12T09:10:57.894 INFO:teuthology.orchestra.run.trial044.stdout: rbd-nbd-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:10:57.894 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:57.894 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:57.923 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-osd-crimson 2026-02-12T09:10:58.228 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:28 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:58.294 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-osd-crimson-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:10:58.311 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:58.312 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:10:58.312 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:10:58.334 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-12T09:10:58.642 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:28 ago on Thu 12 Feb 2026 09:09:30 AM UTC. 2026-02-12T09:10:58.727 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:10:58.728 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:58.728 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repo Size 2026-02-12T09:10:58.728 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:58.728 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-12T09:10:58.728 INFO:teuthology.orchestra.run.trial044.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5161.g2401a109.el9 ceph 429 M 2026-02-12T09:10:58.728 INFO:teuthology.orchestra.run.trial044.stdout:Installing weak dependencies: 2026-02-12T09:10:58.729 INFO:teuthology.orchestra.run.trial044.stdout: ceph-debugsource x86_64 2:20.3.0-5161.g2401a109.el9 ceph 12 M 2026-02-12T09:10:58.729 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:58.729 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-12T09:10:58.729 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:10:58.729 INFO:teuthology.orchestra.run.trial044.stdout:Install 2 Packages 2026-02-12T09:10:58.729 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:10:58.729 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 441 M 2026-02-12T09:10:58.729 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 2.7 G 2026-02-12T09:10:58.729 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-12T09:11:01.270 INFO:teuthology.orchestra.run.trial044.stdout:(1/2): ceph-debugsource-20.3.0-5161.g2401a109.e 4.7 MB/s | 12 MB 00:02 2026-02-12T09:12:17.585 INFO:teuthology.orchestra.run.trial044.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5161.g 5.4 MB/s | 429 MB 01:18 2026-02-12T09:12:17.586 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:12:17.586 INFO:teuthology.orchestra.run.trial044.stdout:Total 5.6 MB/s | 441 MB 01:18 2026-02-12T09:12:17.586 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-12T09:12:17.592 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-12T09:12:17.592 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-12T09:12:18.954 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-12T09:12:18.954 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-12T09:12:22.553 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-12T09:12:45.030 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-debugsource-2:20.3.0-5161.g2401a109.el9.x86_64 1/2 2026-02-12T09:12:45.047 INFO:teuthology.orchestra.run.trial044.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5161.g2401a109.e 2/2 2026-02-12T09:12:46.991 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5161.g2401a109.e 2/2 2026-02-12T09:12:46.992 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-debugsource-2:20.3.0-5161.g2401a109.el9.x86_64 1/2 2026-02-12T09:12:47.049 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5161.g2401a109.e 2/2 2026-02-12T09:12:47.049 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:12:47.049 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-12T09:12:47.050 INFO:teuthology.orchestra.run.trial044.stdout: ceph-debugsource-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:12:47.050 INFO:teuthology.orchestra.run.trial044.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:12:47.050 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:12:47.050 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:12:47.122 DEBUG:teuthology.parallel:result is None 2026-02-12T09:12:47.123 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=2401a109e2ac85dfa20fc57328da663aa754c875 2026-02-12T09:12:52.303 DEBUG:teuthology.orchestra.run.trial044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-12T09:12:52.324 INFO:teuthology.orchestra.run.trial044.stdout:20.3.0-5161.g2401a109.el9 2026-02-12T09:12:52.325 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5161.g2401a109.el9 2026-02-12T09:12:52.325 INFO:teuthology.task.install:The correct ceph version 20.3.0-5161.g2401a109 is installed. 2026-02-12T09:12:52.327 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-12T09:12:52.327 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:12:52.327 DEBUG:teuthology.orchestra.run.trial044:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-12T09:12:52.390 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-12T09:12:52.390 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:12:52.390 DEBUG:teuthology.orchestra.run.trial044:> sudo dd of=/usr/bin/daemon-helper 2026-02-12T09:12:52.454 DEBUG:teuthology.orchestra.run.trial044:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-12T09:12:52.517 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-12T09:12:52.518 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:12:52.518 DEBUG:teuthology.orchestra.run.trial044:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-12T09:12:52.581 DEBUG:teuthology.orchestra.run.trial044:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-12T09:12:52.644 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-12T09:12:52.644 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:12:52.644 DEBUG:teuthology.orchestra.run.trial044:> sudo dd of=/usr/bin/stdin-killer 2026-02-12T09:12:52.707 DEBUG:teuthology.orchestra.run.trial044:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-12T09:12:52.769 INFO:teuthology.run_tasks:Running task pexec... 2026-02-12T09:12:52.779 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-12T09:12:52.779 DEBUG:teuthology.orchestra.run.trial044:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-12T09:12:52.810 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial044.front.sepia.ceph.com 2026-02-12T09:12:52.811 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-12T09:12:52.860 DEBUG:teuthology.parallel:result is None 2026-02-12T09:12:52.860 INFO:teuthology.run_tasks:Running task install... 2026-02-12T09:12:52.869 DEBUG:teuthology.task.install:project ceph 2026-02-12T09:12:52.869 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875'}} 2026-02-12T09:12:52.869 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875'} 2026-02-12T09:12:52.869 INFO:teuthology.task.install:Using flavor: debug 2026-02-12T09:12:52.872 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-12T09:12:52.872 INFO:teuthology.task.install:extra packages: [] 2026-02-12T09:12:52.872 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875', 'tag': None, 'wait_for_package': False} 2026-02-12T09:12:52.872 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=2401a109e2ac85dfa20fc57328da663aa754c875 2026-02-12T09:12:52.993 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-66488-revert/2401a109e2ac85dfa20fc57328da663aa754c875/centos/9/flavors/debug/ 2026-02-12T09:12:52.993 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5161.g2401a109 2026-02-12T09:12:53.248 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-66488-revert/2401a109e2ac85dfa20fc57328da663aa754c875/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-66488-revert/2401a109e2ac85dfa20fc57328da663aa754c875/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-matanb-crimson-66488-revert/2401a109e2ac85dfa20fc57328da663aa754c875/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-12T09:12:53.248 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:12:53.248 DEBUG:teuthology.orchestra.run.trial044:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-12T09:12:53.272 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-12T09:12:53.272 DEBUG:teuthology.orchestra.run.trial044:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/2401a109e2ac85dfa20fc57328da663aa754c875/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-12T09:12:53.338 DEBUG:teuthology.orchestra.run.trial044:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-12T09:12:53.401 DEBUG:teuthology.orchestra.run.trial044:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-12T09:12:53.456 INFO:teuthology.orchestra.run.trial044.stdout:check_obsoletes = 1 2026-02-12T09:12:53.465 DEBUG:teuthology.orchestra.run.trial044:> sudo yum clean all 2026-02-12T09:12:53.628 INFO:teuthology.orchestra.run.trial044.stdout:56 files removed 2026-02-12T09:12:53.645 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-radosgw 2026-02-12T09:12:54.317 INFO:teuthology.orchestra.run.trial044.stdout:ceph packages for x86_64 181 kB/s | 94 kB 00:00 2026-02-12T09:12:54.845 INFO:teuthology.orchestra.run.trial044.stdout:ceph noarch packages 48 kB/s | 24 kB 00:00 2026-02-12T09:12:55.280 INFO:teuthology.orchestra.run.trial044.stdout:ceph source packages 5.0 kB/s | 2.1 kB 00:00 2026-02-12T09:12:56.984 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.9 MB 00:01 2026-02-12T09:12:59.999 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-12T09:13:03.319 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 7.8 MB 00:00 2026-02-12T09:13:05.050 INFO:teuthology.orchestra.run.trial044.stdout:CentOS Stream 9 - Extras packages 20 kB/s | 20 kB 00:01 2026-02-12T09:13:05.704 INFO:teuthology.orchestra.run.trial044.stdout:Extra Packages for Enterprise Linux 35 MB/s | 20 MB 00:00 2026-02-12T09:13:08.944 INFO:teuthology.orchestra.run.trial044.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-12T09:13:10.202 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-radosgw-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:10.219 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:10.220 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:10.220 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:10.244 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-test 2026-02-12T09:13:10.543 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:02 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:10.610 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-test-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:10.627 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:10.628 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:10.628 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:10.649 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph 2026-02-12T09:13:10.951 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:02 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:11.018 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:11.035 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:11.036 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:11.036 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:11.058 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-base 2026-02-12T09:13:11.359 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:03 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:11.426 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-base-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:11.443 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:11.444 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:11.444 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:11.465 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install cephadm 2026-02-12T09:13:11.767 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:03 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:11.832 INFO:teuthology.orchestra.run.trial044.stdout:Package cephadm-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:13:11.849 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:11.850 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:11.850 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:11.872 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-immutable-object-cache 2026-02-12T09:13:12.174 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:04 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:12.239 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-immutable-object-cache-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:12.256 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:12.257 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:12.257 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:12.280 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-mgr 2026-02-12T09:13:12.587 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:04 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:12.652 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-mgr-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:12.670 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:12.670 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:12.671 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:12.693 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-mgr-dashboard 2026-02-12T09:13:12.995 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:04 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:13.062 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-mgr-dashboard-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:13:13.079 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:13.080 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:13.080 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:13.102 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-12T09:13:13.405 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:05 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:13.470 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:13:13.487 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:13.488 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:13.488 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:13.510 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-mgr-rook 2026-02-12T09:13:13.809 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:05 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:13.873 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-mgr-rook-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:13:13.890 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:13.891 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:13.891 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:13.914 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-mgr-cephadm 2026-02-12T09:13:14.215 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:06 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:14.279 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-mgr-cephadm-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:13:14.297 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:14.297 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:14.297 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:14.320 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-osd 2026-02-12T09:13:14.618 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:06 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:14.683 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-osd-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:14.701 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:14.701 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:14.702 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:14.723 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-osd-classic 2026-02-12T09:13:15.027 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:07 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:15.091 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-osd-classic-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:15.110 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:15.110 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:15.110 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:15.132 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-fuse 2026-02-12T09:13:15.431 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:07 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:15.499 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:15.517 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:15.518 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:15.518 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:15.538 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install ceph-volume 2026-02-12T09:13:15.838 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:07 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:15.901 INFO:teuthology.orchestra.run.trial044.stdout:Package ceph-volume-2:20.3.0-5161.g2401a109.el9.noarch is already installed. 2026-02-12T09:13:15.919 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:15.919 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:15.920 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:15.941 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install librados-devel 2026-02-12T09:13:16.248 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:08 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:16.314 INFO:teuthology.orchestra.run.trial044.stdout:Package librados-devel-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:16.331 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:16.332 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:16.332 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:16.354 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install libcephfs2 2026-02-12T09:13:16.657 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:08 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:16.721 INFO:teuthology.orchestra.run.trial044.stdout:Package libcephfs2-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:16.738 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:16.739 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:16.739 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:16.761 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install libcephfs-devel 2026-02-12T09:13:17.060 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:09 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:17.125 INFO:teuthology.orchestra.run.trial044.stdout:Package libcephfs-devel-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:17.142 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:17.143 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:17.143 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:17.165 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install librados2 2026-02-12T09:13:17.463 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:09 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:17.529 INFO:teuthology.orchestra.run.trial044.stdout:Package librados2-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:17.547 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:17.547 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:17.548 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:17.569 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install librbd1 2026-02-12T09:13:17.865 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:09 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:17.931 INFO:teuthology.orchestra.run.trial044.stdout:Package librbd1-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:17.948 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:17.949 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:17.949 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:17.970 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install python3-rados 2026-02-12T09:13:18.273 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:10 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:18.339 INFO:teuthology.orchestra.run.trial044.stdout:Package python3-rados-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:18.357 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:18.357 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:18.358 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:18.379 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install python3-rgw 2026-02-12T09:13:18.678 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:10 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:18.742 INFO:teuthology.orchestra.run.trial044.stdout:Package python3-rgw-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:18.759 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:18.760 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:18.760 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:18.782 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install python3-cephfs 2026-02-12T09:13:19.083 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:11 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:19.149 INFO:teuthology.orchestra.run.trial044.stdout:Package python3-cephfs-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:19.167 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:19.167 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:19.168 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:19.190 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install python3-rbd 2026-02-12T09:13:19.491 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:11 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:19.556 INFO:teuthology.orchestra.run.trial044.stdout:Package python3-rbd-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:19.573 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:19.574 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:19.574 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:19.596 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install rbd-fuse 2026-02-12T09:13:19.897 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:11 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:19.962 INFO:teuthology.orchestra.run.trial044.stdout:Package rbd-fuse-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:19.980 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:19.980 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:19.980 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:20.003 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install rbd-mirror 2026-02-12T09:13:20.304 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:12 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:20.367 INFO:teuthology.orchestra.run.trial044.stdout:Package rbd-mirror-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:20.385 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:20.385 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:20.386 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:20.407 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install rbd-nbd 2026-02-12T09:13:20.709 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:00:12 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:13:20.778 INFO:teuthology.orchestra.run.trial044.stdout:Package rbd-nbd-2:20.3.0-5161.g2401a109.el9.x86_64 is already installed. 2026-02-12T09:13:20.795 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:13:20.796 INFO:teuthology.orchestra.run.trial044.stdout:Nothing to do. 2026-02-12T09:13:20.796 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:13:20.821 DEBUG:teuthology.parallel:result is None 2026-02-12T09:13:20.822 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=2401a109e2ac85dfa20fc57328da663aa754c875 2026-02-12T09:13:20.995 DEBUG:teuthology.orchestra.run.trial044:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-12T09:13:21.016 INFO:teuthology.orchestra.run.trial044.stdout:20.3.0-5161.g2401a109.el9 2026-02-12T09:13:21.017 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5161.g2401a109.el9 2026-02-12T09:13:21.017 INFO:teuthology.task.install:The correct ceph version 20.3.0-5161.g2401a109 is installed. 2026-02-12T09:13:21.019 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-12T09:13:21.019 INFO:teuthology.run_tasks:Running task ceph... 2026-02-12T09:13:21.120 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-12T09:13:21.120 DEBUG:teuthology.orchestra.run.trial044:> sudo chmod 777 /var/log/ceph 2026-02-12T09:13:21.145 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-12T09:13:21.145 DEBUG:teuthology.orchestra.run.trial044:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-12T09:13:21.209 INFO:tasks.ceph:Creating extra log directories... 2026-02-12T09:13:21.209 DEBUG:teuthology.orchestra.run.trial044:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-12T09:13:21.285 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-12T09:13:21.285 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-12T09:13:21.285 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41608', 'branch': 'main', 'description': 'crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} perf_ignorelist settings/optimized workloads/sample_radosbench}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '41608', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875'}}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '6bf51b64d7733c58c2fe4a45746d4b9e65a38d1c'}}, 'owner': 'scheduled_matan@soko04.front.sepia.ceph.com', 'priority': 100, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 9353, 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_6bf51b64d7733c58c2fe4a45746d4b9e65a38d1c/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': '6bf51b64d7733c58c2fe4a45746d4b9e65a38d1c', 'targets': {'trial044.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBOaZc0C2NAzuUJPgXqCMBzDA7HYYFtbI1LoQ+w3pmVlQPfopASy97WhHUjHuJy5DGTm9BXm8EQJckg33aMoCPGY='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'wait-for-scrub': False, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': '2401a109e2ac85dfa20fc57328da663aa754c875', 'cluster': 'ceph'}}, {'ssh_keys': None}, {'cbt': {'benchmarks': {'radosbench': {'concurrent_ops': 4, 'concurrent_procs': 2, 'op_size': [4096], 'pool_monitoring_list': ['collectl'], 'pool_profile': 'replicated', 'run_monitoring_list': ['collectl'], 'time': 60, 'write_only': True}}, 'cluster': {'iterations': 1, 'osds_per_node': 3, 'pool_profiles': {'replicated': {'pg_size': 256, 'pgp_size': 256, 'replication': 'replicated'}}, 'user': 'ubuntu'}, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'c433f1062990a0488dc29a553589bc609a460691', 'timestamp': '2026-02-09_15:50:17', 'tube': 'trial', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.2161924'} 2026-02-12T09:13:21.286 DEBUG:teuthology.orchestra.run.trial044:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-12T09:13:21.331 DEBUG:teuthology.orchestra.run.trial044:> sudo install -d -m0777 -- /var/run/ceph 2026-02-12T09:13:21.395 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:13:21.396 DEBUG:teuthology.orchestra.run.trial044:> dd if=/scratch_devs of=/dev/stdout 2026-02-12T09:13:21.455 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-12T09:13:21.455 DEBUG:teuthology.orchestra.run.trial044:> stat /dev/vg_nvme/lv_1 2026-02-12T09:13:21.512 INFO:teuthology.orchestra.run.trial044.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-12T09:13:21.512 INFO:teuthology.orchestra.run.trial044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-12T09:13:21.512 INFO:teuthology.orchestra.run.trial044.stdout:Device: 6h/6d Inode: 1308 Links: 1 2026-02-12T09:13:21.512 INFO:teuthology.orchestra.run.trial044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-12T09:13:21.513 INFO:teuthology.orchestra.run.trial044.stdout:Context: system_u:object_r:device_t:s0 2026-02-12T09:13:21.513 INFO:teuthology.orchestra.run.trial044.stdout:Access: 2026-02-12 09:10:56.628290149 +0000 2026-02-12T09:13:21.513 INFO:teuthology.orchestra.run.trial044.stdout:Modify: 2026-02-12 09:09:37.601762150 +0000 2026-02-12T09:13:21.513 INFO:teuthology.orchestra.run.trial044.stdout:Change: 2026-02-12 09:09:37.601762150 +0000 2026-02-12T09:13:21.513 INFO:teuthology.orchestra.run.trial044.stdout: Birth: 2026-02-12 09:09:37.601762150 +0000 2026-02-12T09:13:21.513 DEBUG:teuthology.orchestra.run.trial044:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-12T09:13:21.577 INFO:teuthology.orchestra.run.trial044.stderr:1+0 records in 2026-02-12T09:13:21.577 INFO:teuthology.orchestra.run.trial044.stderr:1+0 records out 2026-02-12T09:13:21.577 INFO:teuthology.orchestra.run.trial044.stderr:512 bytes copied, 0.00014633 s, 3.5 MB/s 2026-02-12T09:13:21.578 DEBUG:teuthology.orchestra.run.trial044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-12T09:13:21.635 DEBUG:teuthology.orchestra.run.trial044:> stat /dev/vg_nvme/lv_2 2026-02-12T09:13:21.692 INFO:teuthology.orchestra.run.trial044.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-12T09:13:21.692 INFO:teuthology.orchestra.run.trial044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-12T09:13:21.692 INFO:teuthology.orchestra.run.trial044.stdout:Device: 6h/6d Inode: 1310 Links: 1 2026-02-12T09:13:21.693 INFO:teuthology.orchestra.run.trial044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-12T09:13:21.693 INFO:teuthology.orchestra.run.trial044.stdout:Context: system_u:object_r:device_t:s0 2026-02-12T09:13:21.693 INFO:teuthology.orchestra.run.trial044.stdout:Access: 2026-02-12 09:10:56.628290149 +0000 2026-02-12T09:13:21.693 INFO:teuthology.orchestra.run.trial044.stdout:Modify: 2026-02-12 09:09:37.601762150 +0000 2026-02-12T09:13:21.693 INFO:teuthology.orchestra.run.trial044.stdout:Change: 2026-02-12 09:09:37.601762150 +0000 2026-02-12T09:13:21.693 INFO:teuthology.orchestra.run.trial044.stdout: Birth: 2026-02-12 09:09:37.601762150 +0000 2026-02-12T09:13:21.693 DEBUG:teuthology.orchestra.run.trial044:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-12T09:13:21.754 INFO:teuthology.orchestra.run.trial044.stderr:1+0 records in 2026-02-12T09:13:21.754 INFO:teuthology.orchestra.run.trial044.stderr:1+0 records out 2026-02-12T09:13:21.755 INFO:teuthology.orchestra.run.trial044.stderr:512 bytes copied, 0.000144037 s, 3.6 MB/s 2026-02-12T09:13:21.756 DEBUG:teuthology.orchestra.run.trial044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-12T09:13:21.813 DEBUG:teuthology.orchestra.run.trial044:> stat /dev/vg_nvme/lv_3 2026-02-12T09:13:21.870 INFO:teuthology.orchestra.run.trial044.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-12T09:13:21.870 INFO:teuthology.orchestra.run.trial044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-12T09:13:21.870 INFO:teuthology.orchestra.run.trial044.stdout:Device: 6h/6d Inode: 1300 Links: 1 2026-02-12T09:13:21.870 INFO:teuthology.orchestra.run.trial044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-12T09:13:21.870 INFO:teuthology.orchestra.run.trial044.stdout:Context: system_u:object_r:device_t:s0 2026-02-12T09:13:21.870 INFO:teuthology.orchestra.run.trial044.stdout:Access: 2026-02-12 09:10:56.628290149 +0000 2026-02-12T09:13:21.870 INFO:teuthology.orchestra.run.trial044.stdout:Modify: 2026-02-12 09:09:37.600762168 +0000 2026-02-12T09:13:21.870 INFO:teuthology.orchestra.run.trial044.stdout:Change: 2026-02-12 09:09:37.600762168 +0000 2026-02-12T09:13:21.871 INFO:teuthology.orchestra.run.trial044.stdout: Birth: 2026-02-12 09:09:37.600762168 +0000 2026-02-12T09:13:21.871 DEBUG:teuthology.orchestra.run.trial044:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-12T09:13:21.933 INFO:teuthology.orchestra.run.trial044.stderr:1+0 records in 2026-02-12T09:13:21.934 INFO:teuthology.orchestra.run.trial044.stderr:1+0 records out 2026-02-12T09:13:21.934 INFO:teuthology.orchestra.run.trial044.stderr:512 bytes copied, 0.000146411 s, 3.5 MB/s 2026-02-12T09:13:21.935 DEBUG:teuthology.orchestra.run.trial044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-12T09:13:21.993 DEBUG:teuthology.orchestra.run.trial044:> stat /dev/vg_nvme/lv_4 2026-02-12T09:13:22.051 INFO:teuthology.orchestra.run.trial044.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-12T09:13:22.051 INFO:teuthology.orchestra.run.trial044.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-12T09:13:22.051 INFO:teuthology.orchestra.run.trial044.stdout:Device: 6h/6d Inode: 1289 Links: 1 2026-02-12T09:13:22.051 INFO:teuthology.orchestra.run.trial044.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-12T09:13:22.051 INFO:teuthology.orchestra.run.trial044.stdout:Context: system_u:object_r:device_t:s0 2026-02-12T09:13:22.052 INFO:teuthology.orchestra.run.trial044.stdout:Access: 2026-02-12 09:10:56.629290132 +0000 2026-02-12T09:13:22.052 INFO:teuthology.orchestra.run.trial044.stdout:Modify: 2026-02-12 09:09:37.600762168 +0000 2026-02-12T09:13:22.052 INFO:teuthology.orchestra.run.trial044.stdout:Change: 2026-02-12 09:09:37.600762168 +0000 2026-02-12T09:13:22.052 INFO:teuthology.orchestra.run.trial044.stdout: Birth: 2026-02-12 09:09:37.600762168 +0000 2026-02-12T09:13:22.052 DEBUG:teuthology.orchestra.run.trial044:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-12T09:13:22.113 INFO:teuthology.orchestra.run.trial044.stderr:1+0 records in 2026-02-12T09:13:22.114 INFO:teuthology.orchestra.run.trial044.stderr:1+0 records out 2026-02-12T09:13:22.114 INFO:teuthology.orchestra.run.trial044.stderr:512 bytes copied, 0.000131223 s, 3.9 MB/s 2026-02-12T09:13:22.115 DEBUG:teuthology.orchestra.run.trial044:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-12T09:13:22.171 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-12T09:13:22.172 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial044.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-12T09:13:22.172 INFO:tasks.ceph:Generating config... 2026-02-12T09:13:22.174 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-12T09:13:22.174 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-12T09:13:22.174 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-12T09:13:22.174 INFO:tasks.ceph:[osd] debug filestore = 0/0 2026-02-12T09:13:22.174 INFO:tasks.ceph:[osd] debug journal = 0/0 2026-02-12T09:13:22.175 INFO:tasks.ceph:[osd] debug ms = 0/0 2026-02-12T09:13:22.175 INFO:tasks.ceph:[osd] debug osd = 0/0 2026-02-12T09:13:22.175 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-12T09:13:22.175 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-12T09:13:22.175 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-12T09:13:22.175 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-12T09:13:22.175 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-12T09:13:22.175 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-12T09:13:22.175 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-12T09:13:22.175 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-12T09:13:22.175 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-12T09:13:22.175 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-12T09:13:22.176 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-12T09:13:22.176 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-12T09:13:22.176 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-12T09:13:22.176 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-12T09:13:22.176 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-12T09:13:22.176 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-12T09:13:22.176 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-12T09:13:22.176 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-12T09:13:22.176 INFO:tasks.ceph:[osd] seastore main device type = RANDOM_BLOCK_SSD 2026-02-12T09:13:22.176 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-12T09:13:22.176 INFO:tasks.ceph:[global] auth client required = none 2026-02-12T09:13:22.176 INFO:tasks.ceph:[global] auth cluster required = none 2026-02-12T09:13:22.176 INFO:tasks.ceph:[global] auth service required = none 2026-02-12T09:13:22.177 INFO:tasks.ceph:[global] auth supported = none 2026-02-12T09:13:22.177 INFO:tasks.ceph:[global] debug asok = 0/0 2026-02-12T09:13:22.177 INFO:tasks.ceph:[global] debug auth = 0/0 2026-02-12T09:13:22.177 INFO:tasks.ceph:[global] debug bdev = 0/0 2026-02-12T09:13:22.177 INFO:tasks.ceph:[global] debug bluefs = 0/0 2026-02-12T09:13:22.177 INFO:tasks.ceph:[global] debug bluestore = 0/0 2026-02-12T09:13:22.177 INFO:tasks.ceph:[global] debug buffer = 0/0 2026-02-12T09:13:22.177 INFO:tasks.ceph:[global] debug civetweb = 0/0 2026-02-12T09:13:22.177 INFO:tasks.ceph:[global] debug client = 0/0 2026-02-12T09:13:22.177 INFO:tasks.ceph:[global] debug compressor = 0/0 2026-02-12T09:13:22.177 INFO:tasks.ceph:[global] debug context = 0/0 2026-02-12T09:13:22.177 INFO:tasks.ceph:[global] debug crush = 0/0 2026-02-12T09:13:22.177 INFO:tasks.ceph:[global] debug crypto = 0/0 2026-02-12T09:13:22.178 INFO:tasks.ceph:[global] debug dpdk = 0/0 2026-02-12T09:13:22.178 INFO:tasks.ceph:[global] debug eventtrace = 0/0 2026-02-12T09:13:22.178 INFO:tasks.ceph:[global] debug filer = 0/0 2026-02-12T09:13:22.178 INFO:tasks.ceph:[global] debug filestore = 0/0 2026-02-12T09:13:22.178 INFO:tasks.ceph:[global] debug finisher = 0/0 2026-02-12T09:13:22.178 INFO:tasks.ceph:[global] debug fuse = 0/0 2026-02-12T09:13:22.178 INFO:tasks.ceph:[global] debug heartbeatmap = 0/0 2026-02-12T09:13:22.178 INFO:tasks.ceph:[global] debug javaclient = 0/0 2026-02-12T09:13:22.178 INFO:tasks.ceph:[global] debug journal = 0/0 2026-02-12T09:13:22.178 INFO:tasks.ceph:[global] debug journaler = 0/0 2026-02-12T09:13:22.178 INFO:tasks.ceph:[global] debug kstore = 0/0 2026-02-12T09:13:22.178 INFO:tasks.ceph:[global] debug leveldb = 0/0 2026-02-12T09:13:22.179 INFO:tasks.ceph:[global] debug lockdep = 0/0 2026-02-12T09:13:22.179 INFO:tasks.ceph:[global] debug mds = 0/0 2026-02-12T09:13:22.179 INFO:tasks.ceph:[global] debug mds balancer = 0/0 2026-02-12T09:13:22.179 INFO:tasks.ceph:[global] debug mds locker = 0/0 2026-02-12T09:13:22.179 INFO:tasks.ceph:[global] debug mds log = 0/0 2026-02-12T09:13:22.179 INFO:tasks.ceph:[global] debug mds log expire = 0/0 2026-02-12T09:13:22.179 INFO:tasks.ceph:[global] debug mds migrator = 0/0 2026-02-12T09:13:22.179 INFO:tasks.ceph:[global] debug memdb = 0/0 2026-02-12T09:13:22.179 INFO:tasks.ceph:[global] debug mgr = 0/0 2026-02-12T09:13:22.179 INFO:tasks.ceph:[global] debug mgrc = 0/0 2026-02-12T09:13:22.179 INFO:tasks.ceph:[global] debug mon = 0/0 2026-02-12T09:13:22.179 INFO:tasks.ceph:[global] debug monc = 0/0 2026-02-12T09:13:22.179 INFO:tasks.ceph:[global] debug ms = 0/0 2026-02-12T09:13:22.180 INFO:tasks.ceph:[global] debug objclass = 0/0 2026-02-12T09:13:22.180 INFO:tasks.ceph:[global] debug objectcacher = 0/0 2026-02-12T09:13:22.180 INFO:tasks.ceph:[global] debug objecter = 0/0 2026-02-12T09:13:22.180 INFO:tasks.ceph:[global] debug optracker = 0/0 2026-02-12T09:13:22.180 INFO:tasks.ceph:[global] debug osd = 0/0 2026-02-12T09:13:22.180 INFO:tasks.ceph:[global] debug paxos = 0/0 2026-02-12T09:13:22.180 INFO:tasks.ceph:[global] debug perfcounter = 0/0 2026-02-12T09:13:22.180 INFO:tasks.ceph:[global] debug rados = 0/0 2026-02-12T09:13:22.180 INFO:tasks.ceph:[global] debug rbd = 0/0 2026-02-12T09:13:22.180 INFO:tasks.ceph:[global] debug rbd mirror = 0/0 2026-02-12T09:13:22.180 INFO:tasks.ceph:[global] debug rbd replay = 0/0 2026-02-12T09:13:22.180 INFO:tasks.ceph:[global] debug refs = 0/0 2026-02-12T09:13:22.180 INFO:tasks.ceph:[global] debug rgw = 0/0 2026-02-12T09:13:22.181 INFO:tasks.ceph:[global] debug rgw lifecycle = 0/0 2026-02-12T09:13:22.181 INFO:tasks.ceph:[global] debug rgw notification = 0/0 2026-02-12T09:13:22.181 INFO:tasks.ceph:[global] debug rgw sync = 0/0 2026-02-12T09:13:22.181 INFO:tasks.ceph:[global] debug rocksdb = 0/0 2026-02-12T09:13:22.181 INFO:tasks.ceph:[global] debug striper = 0/0 2026-02-12T09:13:22.181 INFO:tasks.ceph:[global] debug throttle = 0/0 2026-02-12T09:13:22.181 INFO:tasks.ceph:[global] debug timer = 0/0 2026-02-12T09:13:22.181 INFO:tasks.ceph:[global] debug tp = 0/0 2026-02-12T09:13:22.181 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-12T09:13:22.181 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-12T09:13:22.181 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-12T09:13:22.181 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-12T09:13:22.181 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-12T09:13:22.181 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-12T09:13:22.182 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-12T09:13:22.182 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-12T09:13:22.182 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-12T09:13:22.182 INFO:tasks.ceph:[mon] debug mon = 0/0 2026-02-12T09:13:22.182 INFO:tasks.ceph:[mon] debug ms = 0/0 2026-02-12T09:13:22.182 INFO:tasks.ceph:[mon] debug paxos = 0/0 2026-02-12T09:13:22.182 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-12T09:13:22.182 INFO:tasks.ceph:Setting up mon.a... 2026-02-12T09:13:22.182 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-12T09:13:22.252 INFO:teuthology.orchestra.run.trial044.stdout:creating /etc/ceph/ceph.keyring 2026-02-12T09:13:22.255 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-12T09:13:22.341 DEBUG:teuthology.orchestra.run.trial044:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-12T09:13:22.365 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.44')] 2026-02-12T09:13:22.366 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'none', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.44', 'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': '0/0', 'debug mon': '0/0', 'debug paxos': '0/0', 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-12T09:13:22.367 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:13:22.368 DEBUG:teuthology.orchestra.run.trial044:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-12T09:13:22.421 DEBUG:teuthology.orchestra.run.trial044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.44 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-12T09:13:22.500 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:22.496+0000 7fbecca93f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:22.500 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:22.496+0000 7fbecca93f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:22.500 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:22.496+0000 7fbecca93f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:22.501 INFO:teuthology.orchestra.run.trial044.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-12T09:13:22.501 INFO:teuthology.orchestra.run.trial044.stdout:monmaptool: generated fsid 74f73dcc-542d-4c1a-9175-f74dd177fad8 2026-02-12T09:13:22.502 INFO:teuthology.orchestra.run.trial044.stdout:setting min_mon_release = tentacle 2026-02-12T09:13:22.502 INFO:teuthology.orchestra.run.trial044.stdout:epoch 0 2026-02-12T09:13:22.502 INFO:teuthology.orchestra.run.trial044.stdout:fsid 74f73dcc-542d-4c1a-9175-f74dd177fad8 2026-02-12T09:13:22.502 INFO:teuthology.orchestra.run.trial044.stdout:last_changed 2026-02-12T09:13:22.497322+0000 2026-02-12T09:13:22.502 INFO:teuthology.orchestra.run.trial044.stdout:created 2026-02-12T09:13:22.497322+0000 2026-02-12T09:13:22.502 INFO:teuthology.orchestra.run.trial044.stdout:min_mon_release 20 (tentacle) 2026-02-12T09:13:22.502 INFO:teuthology.orchestra.run.trial044.stdout:election_strategy: 1 2026-02-12T09:13:22.502 INFO:teuthology.orchestra.run.trial044.stdout:0: [v2:10.20.193.44:3300/0,v1:10.20.193.44:6789/0] mon.a 2026-02-12T09:13:22.502 INFO:teuthology.orchestra.run.trial044.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-12T09:13:22.503 DEBUG:teuthology.orchestra.run.trial044:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-12T09:13:22.558 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 74f73dcc-542d-4c1a-9175-f74dd177fad8... 2026-02-12T09:13:22.560 DEBUG:teuthology.orchestra.run.trial044:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-12T09:13:22.642 INFO:teuthology.orchestra.run.trial044.stdout:[global] 2026-02-12T09:13:22.642 INFO:teuthology.orchestra.run.trial044.stdout: chdir = "" 2026-02-12T09:13:22.642 INFO:teuthology.orchestra.run.trial044.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-12T09:13:22.642 INFO:teuthology.orchestra.run.trial044.stdout: auth supported = none 2026-02-12T09:13:22.642 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.642 INFO:teuthology.orchestra.run.trial044.stdout: filestore xattr use omap = true 2026-02-12T09:13:22.642 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.642 INFO:teuthology.orchestra.run.trial044.stdout: mon clock drift allowed = 1.000 2026-02-12T09:13:22.642 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.643 INFO:teuthology.orchestra.run.trial044.stdout: osd crush chooseleaf type = 0 2026-02-12T09:13:22.643 INFO:teuthology.orchestra.run.trial044.stdout: auth debug = true 2026-02-12T09:13:22.643 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.643 INFO:teuthology.orchestra.run.trial044.stdout: ms die on old message = true 2026-02-12T09:13:22.643 INFO:teuthology.orchestra.run.trial044.stdout: ms die on bug = true 2026-02-12T09:13:22.643 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.643 INFO:teuthology.orchestra.run.trial044.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-12T09:13:22.643 INFO:teuthology.orchestra.run.trial044.stdout: mon pg warn max object skew = 0 2026-02-12T09:13:22.643 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.643 INFO:teuthology.orchestra.run.trial044.stdout: # disable pg_autoscaler by default for new pools 2026-02-12T09:13:22.643 INFO:teuthology.orchestra.run.trial044.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-12T09:13:22.643 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.644 INFO:teuthology.orchestra.run.trial044.stdout: osd pool default size = 2 2026-02-12T09:13:22.644 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.644 INFO:teuthology.orchestra.run.trial044.stdout: mon osd allow primary affinity = true 2026-02-12T09:13:22.644 INFO:teuthology.orchestra.run.trial044.stdout: mon osd allow pg remap = true 2026-02-12T09:13:22.644 INFO:teuthology.orchestra.run.trial044.stdout: mon warn on legacy crush tunables = false 2026-02-12T09:13:22.644 INFO:teuthology.orchestra.run.trial044.stdout: mon warn on crush straw calc version zero = false 2026-02-12T09:13:22.644 INFO:teuthology.orchestra.run.trial044.stdout: mon warn on no sortbitwise = false 2026-02-12T09:13:22.644 INFO:teuthology.orchestra.run.trial044.stdout: mon warn on osd down out interval zero = false 2026-02-12T09:13:22.644 INFO:teuthology.orchestra.run.trial044.stdout: mon warn on too few osds = false 2026-02-12T09:13:22.644 INFO:teuthology.orchestra.run.trial044.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-12T09:13:22.644 INFO:teuthology.orchestra.run.trial044.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-12T09:13:22.645 INFO:teuthology.orchestra.run.trial044.stdout: mon_allow_pool_size_one = true 2026-02-12T09:13:22.645 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.645 INFO:teuthology.orchestra.run.trial044.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-12T09:13:22.645 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.645 INFO:teuthology.orchestra.run.trial044.stdout: osd default data pool replay window = 5 2026-02-12T09:13:22.645 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.645 INFO:teuthology.orchestra.run.trial044.stdout: mon allow pool delete = true 2026-02-12T09:13:22.645 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.645 INFO:teuthology.orchestra.run.trial044.stdout: mon cluster log file level = debug 2026-02-12T09:13:22.645 INFO:teuthology.orchestra.run.trial044.stdout: debug asserts on shutdown = true 2026-02-12T09:13:22.645 INFO:teuthology.orchestra.run.trial044.stdout: mon health detail to clog = false 2026-02-12T09:13:22.645 INFO:teuthology.orchestra.run.trial044.stdout: mon host = 10.20.193.44 2026-02-12T09:13:22.646 INFO:teuthology.orchestra.run.trial044.stdout: auth client required = none 2026-02-12T09:13:22.646 INFO:teuthology.orchestra.run.trial044.stdout: auth cluster required = none 2026-02-12T09:13:22.646 INFO:teuthology.orchestra.run.trial044.stdout: auth service required = none 2026-02-12T09:13:22.646 INFO:teuthology.orchestra.run.trial044.stdout: debug asok = 0/0 2026-02-12T09:13:22.646 INFO:teuthology.orchestra.run.trial044.stdout: debug auth = 0/0 2026-02-12T09:13:22.646 INFO:teuthology.orchestra.run.trial044.stdout: debug bdev = 0/0 2026-02-12T09:13:22.646 INFO:teuthology.orchestra.run.trial044.stdout: debug bluefs = 0/0 2026-02-12T09:13:22.646 INFO:teuthology.orchestra.run.trial044.stdout: debug bluestore = 0/0 2026-02-12T09:13:22.646 INFO:teuthology.orchestra.run.trial044.stdout: debug buffer = 0/0 2026-02-12T09:13:22.646 INFO:teuthology.orchestra.run.trial044.stdout: debug civetweb = 0/0 2026-02-12T09:13:22.646 INFO:teuthology.orchestra.run.trial044.stdout: debug client = 0/0 2026-02-12T09:13:22.646 INFO:teuthology.orchestra.run.trial044.stdout: debug compressor = 0/0 2026-02-12T09:13:22.646 INFO:teuthology.orchestra.run.trial044.stdout: debug context = 0/0 2026-02-12T09:13:22.647 INFO:teuthology.orchestra.run.trial044.stdout: debug crush = 0/0 2026-02-12T09:13:22.647 INFO:teuthology.orchestra.run.trial044.stdout: debug crypto = 0/0 2026-02-12T09:13:22.647 INFO:teuthology.orchestra.run.trial044.stdout: debug dpdk = 0/0 2026-02-12T09:13:22.647 INFO:teuthology.orchestra.run.trial044.stdout: debug eventtrace = 0/0 2026-02-12T09:13:22.647 INFO:teuthology.orchestra.run.trial044.stdout: debug filer = 0/0 2026-02-12T09:13:22.647 INFO:teuthology.orchestra.run.trial044.stdout: debug filestore = 0/0 2026-02-12T09:13:22.647 INFO:teuthology.orchestra.run.trial044.stdout: debug finisher = 0/0 2026-02-12T09:13:22.647 INFO:teuthology.orchestra.run.trial044.stdout: debug fuse = 0/0 2026-02-12T09:13:22.647 INFO:teuthology.orchestra.run.trial044.stdout: debug heartbeatmap = 0/0 2026-02-12T09:13:22.647 INFO:teuthology.orchestra.run.trial044.stdout: debug javaclient = 0/0 2026-02-12T09:13:22.647 INFO:teuthology.orchestra.run.trial044.stdout: debug journal = 0/0 2026-02-12T09:13:22.647 INFO:teuthology.orchestra.run.trial044.stdout: debug journaler = 0/0 2026-02-12T09:13:22.647 INFO:teuthology.orchestra.run.trial044.stdout: debug kstore = 0/0 2026-02-12T09:13:22.648 INFO:teuthology.orchestra.run.trial044.stdout: debug leveldb = 0/0 2026-02-12T09:13:22.648 INFO:teuthology.orchestra.run.trial044.stdout: debug lockdep = 0/0 2026-02-12T09:13:22.648 INFO:teuthology.orchestra.run.trial044.stdout: debug mds = 0/0 2026-02-12T09:13:22.648 INFO:teuthology.orchestra.run.trial044.stdout: debug mds balancer = 0/0 2026-02-12T09:13:22.648 INFO:teuthology.orchestra.run.trial044.stdout: debug mds locker = 0/0 2026-02-12T09:13:22.648 INFO:teuthology.orchestra.run.trial044.stdout: debug mds log = 0/0 2026-02-12T09:13:22.648 INFO:teuthology.orchestra.run.trial044.stdout: debug mds log expire = 0/0 2026-02-12T09:13:22.648 INFO:teuthology.orchestra.run.trial044.stdout: debug mds migrator = 0/0 2026-02-12T09:13:22.648 INFO:teuthology.orchestra.run.trial044.stdout: debug memdb = 0/0 2026-02-12T09:13:22.648 INFO:teuthology.orchestra.run.trial044.stdout: debug mgr = 0/0 2026-02-12T09:13:22.648 INFO:teuthology.orchestra.run.trial044.stdout: debug mgrc = 0/0 2026-02-12T09:13:22.648 INFO:teuthology.orchestra.run.trial044.stdout: debug mon = 0/0 2026-02-12T09:13:22.648 INFO:teuthology.orchestra.run.trial044.stdout: debug monc = 0/0 2026-02-12T09:13:22.649 INFO:teuthology.orchestra.run.trial044.stdout: debug ms = 0/0 2026-02-12T09:13:22.649 INFO:teuthology.orchestra.run.trial044.stdout: debug objclass = 0/0 2026-02-12T09:13:22.649 INFO:teuthology.orchestra.run.trial044.stdout: debug objectcacher = 0/0 2026-02-12T09:13:22.649 INFO:teuthology.orchestra.run.trial044.stdout: debug objecter = 0/0 2026-02-12T09:13:22.649 INFO:teuthology.orchestra.run.trial044.stdout: debug optracker = 0/0 2026-02-12T09:13:22.649 INFO:teuthology.orchestra.run.trial044.stdout: debug osd = 0/0 2026-02-12T09:13:22.649 INFO:teuthology.orchestra.run.trial044.stdout: debug paxos = 0/0 2026-02-12T09:13:22.649 INFO:teuthology.orchestra.run.trial044.stdout: debug perfcounter = 0/0 2026-02-12T09:13:22.649 INFO:teuthology.orchestra.run.trial044.stdout: debug rados = 0/0 2026-02-12T09:13:22.649 INFO:teuthology.orchestra.run.trial044.stdout: debug rbd = 0/0 2026-02-12T09:13:22.649 INFO:teuthology.orchestra.run.trial044.stdout: debug rbd mirror = 0/0 2026-02-12T09:13:22.649 INFO:teuthology.orchestra.run.trial044.stdout: debug rbd replay = 0/0 2026-02-12T09:13:22.649 INFO:teuthology.orchestra.run.trial044.stdout: debug refs = 0/0 2026-02-12T09:13:22.650 INFO:teuthology.orchestra.run.trial044.stdout: debug rgw = 0/0 2026-02-12T09:13:22.650 INFO:teuthology.orchestra.run.trial044.stdout: debug rgw lifecycle = 0/0 2026-02-12T09:13:22.650 INFO:teuthology.orchestra.run.trial044.stdout: debug rgw notification = 0/0 2026-02-12T09:13:22.650 INFO:teuthology.orchestra.run.trial044.stdout: debug rgw sync = 0/0 2026-02-12T09:13:22.650 INFO:teuthology.orchestra.run.trial044.stdout: debug rocksdb = 0/0 2026-02-12T09:13:22.650 INFO:teuthology.orchestra.run.trial044.stdout: debug striper = 0/0 2026-02-12T09:13:22.650 INFO:teuthology.orchestra.run.trial044.stdout: debug throttle = 0/0 2026-02-12T09:13:22.650 INFO:teuthology.orchestra.run.trial044.stdout: debug timer = 0/0 2026-02-12T09:13:22.650 INFO:teuthology.orchestra.run.trial044.stdout: debug tp = 0/0 2026-02-12T09:13:22.650 INFO:teuthology.orchestra.run.trial044.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-12T09:13:22.650 INFO:teuthology.orchestra.run.trial044.stdout: ms client mode = crc 2026-02-12T09:13:22.650 INFO:teuthology.orchestra.run.trial044.stdout: ms cluster mode = crc 2026-02-12T09:13:22.650 INFO:teuthology.orchestra.run.trial044.stdout: ms mon client mode = crc 2026-02-12T09:13:22.651 INFO:teuthology.orchestra.run.trial044.stdout: ms mon cluster mode = crc 2026-02-12T09:13:22.651 INFO:teuthology.orchestra.run.trial044.stdout: ms mon service mode = crc 2026-02-12T09:13:22.651 INFO:teuthology.orchestra.run.trial044.stdout: ms service mode = crc 2026-02-12T09:13:22.651 INFO:teuthology.orchestra.run.trial044.stdout: fsid = 74f73dcc-542d-4c1a-9175-f74dd177fad8 2026-02-12T09:13:22.651 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.651 INFO:teuthology.orchestra.run.trial044.stdout:[osd] 2026-02-12T09:13:22.651 INFO:teuthology.orchestra.run.trial044.stdout: osd journal size = 100 2026-02-12T09:13:22.651 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.651 INFO:teuthology.orchestra.run.trial044.stdout: osd scrub load threshold = 5.0 2026-02-12T09:13:22.651 INFO:teuthology.orchestra.run.trial044.stdout: osd scrub max interval = 600 2026-02-12T09:13:22.651 INFO:teuthology.orchestra.run.trial044.stdout: osd mclock profile = high_recovery_ops 2026-02-12T09:13:22.651 INFO:teuthology.orchestra.run.trial044.stdout: osd mclock skip benchmark = true 2026-02-12T09:13:22.652 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.652 INFO:teuthology.orchestra.run.trial044.stdout: osd recover clone overlap = true 2026-02-12T09:13:22.652 INFO:teuthology.orchestra.run.trial044.stdout: osd recovery max chunk = 1048576 2026-02-12T09:13:22.652 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.652 INFO:teuthology.orchestra.run.trial044.stdout: osd debug shutdown = true 2026-02-12T09:13:22.652 INFO:teuthology.orchestra.run.trial044.stdout: osd debug op order = true 2026-02-12T09:13:22.652 INFO:teuthology.orchestra.run.trial044.stdout: osd debug verify stray on activate = true 2026-02-12T09:13:22.652 INFO:teuthology.orchestra.run.trial044.stdout: osd debug trim objects = true 2026-02-12T09:13:22.652 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.652 INFO:teuthology.orchestra.run.trial044.stdout: osd open classes on start = true 2026-02-12T09:13:22.652 INFO:teuthology.orchestra.run.trial044.stdout: osd debug pg log writeout = true 2026-02-12T09:13:22.652 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.653 INFO:teuthology.orchestra.run.trial044.stdout: osd deep scrub update digest min age = 30 2026-02-12T09:13:22.653 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.653 INFO:teuthology.orchestra.run.trial044.stdout: osd map max advance = 10 2026-02-12T09:13:22.653 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.653 INFO:teuthology.orchestra.run.trial044.stdout: journal zero on create = true 2026-02-12T09:13:22.653 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.653 INFO:teuthology.orchestra.run.trial044.stdout: filestore ondisk finisher threads = 3 2026-02-12T09:13:22.653 INFO:teuthology.orchestra.run.trial044.stdout: filestore apply finisher threads = 3 2026-02-12T09:13:22.653 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.653 INFO:teuthology.orchestra.run.trial044.stdout: bdev debug aio = true 2026-02-12T09:13:22.653 INFO:teuthology.orchestra.run.trial044.stdout: osd debug misdirected ops = true 2026-02-12T09:13:22.653 INFO:teuthology.orchestra.run.trial044.stdout: debug monc = 20 2026-02-12T09:13:22.654 INFO:teuthology.orchestra.run.trial044.stdout: crimson cpu num = 4 2026-02-12T09:13:22.654 INFO:teuthology.orchestra.run.trial044.stdout: crimson osd obc lru size = 10 2026-02-12T09:13:22.654 INFO:teuthology.orchestra.run.trial044.stdout: debug filestore = 0/0 2026-02-12T09:13:22.654 INFO:teuthology.orchestra.run.trial044.stdout: debug journal = 0/0 2026-02-12T09:13:22.654 INFO:teuthology.orchestra.run.trial044.stdout: debug ms = 0/0 2026-02-12T09:13:22.654 INFO:teuthology.orchestra.run.trial044.stdout: debug osd = 0/0 2026-02-12T09:13:22.654 INFO:teuthology.orchestra.run.trial044.stdout: debug seastore = 20 2026-02-12T09:13:22.654 INFO:teuthology.orchestra.run.trial044.stdout: debug seastore backref = 20 2026-02-12T09:13:22.654 INFO:teuthology.orchestra.run.trial044.stdout: debug seastore cache = 20 2026-02-12T09:13:22.654 INFO:teuthology.orchestra.run.trial044.stdout: debug seastore cleaner = 20 2026-02-12T09:13:22.654 INFO:teuthology.orchestra.run.trial044.stdout: debug seastore device = 20 2026-02-12T09:13:22.654 INFO:teuthology.orchestra.run.trial044.stdout: debug seastore epm = 20 2026-02-12T09:13:22.654 INFO:teuthology.orchestra.run.trial044.stdout: debug seastore fixedkv tree = 20 2026-02-12T09:13:22.654 INFO:teuthology.orchestra.run.trial044.stdout: debug seastore journal = 20 2026-02-12T09:13:22.655 INFO:teuthology.orchestra.run.trial044.stdout: debug seastore lba = 20 2026-02-12T09:13:22.655 INFO:teuthology.orchestra.run.trial044.stdout: debug seastore odata = 20 2026-02-12T09:13:22.655 INFO:teuthology.orchestra.run.trial044.stdout: debug seastore omap = 20 2026-02-12T09:13:22.655 INFO:teuthology.orchestra.run.trial044.stdout: debug seastore onode = 20 2026-02-12T09:13:22.655 INFO:teuthology.orchestra.run.trial044.stdout: debug seastore t = 20 2026-02-12T09:13:22.655 INFO:teuthology.orchestra.run.trial044.stdout: debug seastore tm = 20 2026-02-12T09:13:22.655 INFO:teuthology.orchestra.run.trial044.stdout: osd objectstore = seastore 2026-02-12T09:13:22.655 INFO:teuthology.orchestra.run.trial044.stdout: osd shutdown pgref assert = True 2026-02-12T09:13:22.655 INFO:teuthology.orchestra.run.trial044.stdout: seastore cachepin size pershard = 64M 2026-02-12T09:13:22.655 INFO:teuthology.orchestra.run.trial044.stdout: seastore cachepin type = 2Q 2026-02-12T09:13:22.655 INFO:teuthology.orchestra.run.trial044.stdout: seastore main device type = RANDOM_BLOCK_SSD 2026-02-12T09:13:22.655 INFO:teuthology.orchestra.run.trial044.stdout: seastore max concurrent transactions = 8 2026-02-12T09:13:22.655 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.656 INFO:teuthology.orchestra.run.trial044.stdout:[mgr] 2026-02-12T09:13:22.656 INFO:teuthology.orchestra.run.trial044.stdout: debug ms = 1 2026-02-12T09:13:22.656 INFO:teuthology.orchestra.run.trial044.stdout: debug mgr = 20 2026-02-12T09:13:22.656 INFO:teuthology.orchestra.run.trial044.stdout: debug mon = 20 2026-02-12T09:13:22.656 INFO:teuthology.orchestra.run.trial044.stdout: debug auth = 20 2026-02-12T09:13:22.656 INFO:teuthology.orchestra.run.trial044.stdout: mon reweight min pgs per osd = 4 2026-02-12T09:13:22.656 INFO:teuthology.orchestra.run.trial044.stdout: mon reweight min bytes per osd = 10 2026-02-12T09:13:22.656 INFO:teuthology.orchestra.run.trial044.stdout: mgr/telemetry/nag = false 2026-02-12T09:13:22.656 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.656 INFO:teuthology.orchestra.run.trial044.stdout:[mon] 2026-02-12T09:13:22.656 INFO:teuthology.orchestra.run.trial044.stdout: debug ms = 0/0 2026-02-12T09:13:22.656 INFO:teuthology.orchestra.run.trial044.stdout: debug mon = 0/0 2026-02-12T09:13:22.657 INFO:teuthology.orchestra.run.trial044.stdout: debug paxos = 0/0 2026-02-12T09:13:22.657 INFO:teuthology.orchestra.run.trial044.stdout: debug auth = 20 2026-02-12T09:13:22.657 INFO:teuthology.orchestra.run.trial044.stdout: mon data avail warn = 5 2026-02-12T09:13:22.657 INFO:teuthology.orchestra.run.trial044.stdout: mon mgr mkfs grace = 240 2026-02-12T09:13:22.657 INFO:teuthology.orchestra.run.trial044.stdout: mon reweight min pgs per osd = 4 2026-02-12T09:13:22.657 INFO:teuthology.orchestra.run.trial044.stdout: mon osd reporter subtree level = osd 2026-02-12T09:13:22.657 INFO:teuthology.orchestra.run.trial044.stdout: mon osd prime pg temp = true 2026-02-12T09:13:22.657 INFO:teuthology.orchestra.run.trial044.stdout: mon reweight min bytes per osd = 10 2026-02-12T09:13:22.657 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.657 INFO:teuthology.orchestra.run.trial044.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-12T09:13:22.657 INFO:teuthology.orchestra.run.trial044.stdout: auth mon ticket ttl = 660 # 11m 2026-02-12T09:13:22.657 INFO:teuthology.orchestra.run.trial044.stdout: auth service ticket ttl = 240 # 4m 2026-02-12T09:13:22.658 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.658 INFO:teuthology.orchestra.run.trial044.stdout: # don't complain about insecure global_id in the test suite 2026-02-12T09:13:22.658 INFO:teuthology.orchestra.run.trial044.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-12T09:13:22.658 INFO:teuthology.orchestra.run.trial044.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-12T09:13:22.658 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.658 INFO:teuthology.orchestra.run.trial044.stdout: # 1m isn't quite enough 2026-02-12T09:13:22.658 INFO:teuthology.orchestra.run.trial044.stdout: mon_down_mkfs_grace = 2m 2026-02-12T09:13:22.658 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.658 INFO:teuthology.orchestra.run.trial044.stdout: mon_warn_on_filestore_osds = false 2026-02-12T09:13:22.658 INFO:teuthology.orchestra.run.trial044.stdout: osd pool default crimson = True 2026-02-12T09:13:22.658 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:22.658 INFO:teuthology.orchestra.run.trial044.stdout:[client] 2026-02-12T09:13:22.658 INFO:teuthology.orchestra.run.trial044.stdout: rgw cache enabled = true 2026-02-12T09:13:22.659 INFO:teuthology.orchestra.run.trial044.stdout: rgw enable ops log = true 2026-02-12T09:13:22.659 INFO:teuthology.orchestra.run.trial044.stdout: rgw enable usage log = true 2026-02-12T09:13:22.659 INFO:teuthology.orchestra.run.trial044.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-12T09:13:22.659 INFO:teuthology.orchestra.run.trial044.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-12T09:13:22.659 INFO:teuthology.orchestra.run.trial044.stdout:[mon.a] 2026-02-12T09:13:22.660 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-12T09:13:22.660 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-12T09:13:22.751 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-12T09:13:22.751 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:13:22.751 DEBUG:teuthology.orchestra.run.trial044:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-12T09:13:22.768 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:13:22.769 DEBUG:teuthology.orchestra.run.trial044:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-12T09:13:22.824 INFO:tasks.ceph:Sending monmap to node ubuntu@trial044.front.sepia.ceph.com 2026-02-12T09:13:22.824 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:13:22.824 DEBUG:teuthology.orchestra.run.trial044:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-12T09:13:22.824 DEBUG:teuthology.orchestra.run.trial044:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-12T09:13:22.897 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:13:22.897 DEBUG:teuthology.orchestra.run.trial044:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-12T09:13:22.954 INFO:tasks.ceph:Setting up mon nodes... 2026-02-12T09:13:22.954 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-12T09:13:22.954 DEBUG:teuthology.orchestra.run.trial044:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-12T09:13:23.043 INFO:teuthology.orchestra.run.trial044.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-12T09:13:23.047 INFO:tasks.ceph:Setting up mds nodes... 2026-02-12T09:13:23.047 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-12T09:13:23.047 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-12T09:13:23.089 INFO:teuthology.orchestra.run.trial044.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-12T09:13:23.101 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-12T09:13:23.102 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial044.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-12T09:13:23.102 DEBUG:teuthology.orchestra.run.trial044:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-12T09:13:23.168 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-12T09:13:23.168 INFO:tasks.ceph:role: osd.0 2026-02-12T09:13:23.168 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial044.front.sepia.ceph.com 2026-02-12T09:13:23.169 DEBUG:teuthology.orchestra.run.trial044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-12T09:13:23.233 INFO:teuthology.orchestra.run.trial044.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-12T09:13:23.233 INFO:teuthology.orchestra.run.trial044.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-12T09:13:23.233 INFO:teuthology.orchestra.run.trial044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-12T09:13:23.234 INFO:teuthology.orchestra.run.trial044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-12T09:13:23.234 INFO:teuthology.orchestra.run.trial044.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-12T09:13:23.234 INFO:teuthology.orchestra.run.trial044.stdout: = sunit=0 swidth=0 blks 2026-02-12T09:13:23.234 INFO:teuthology.orchestra.run.trial044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-12T09:13:23.234 INFO:teuthology.orchestra.run.trial044.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-12T09:13:23.234 INFO:teuthology.orchestra.run.trial044.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-12T09:13:23.234 INFO:teuthology.orchestra.run.trial044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-12T09:13:23.804 INFO:teuthology.orchestra.run.trial044.stdout:Discarding blocks...Done. 2026-02-12T09:13:23.806 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial044.front.sepia.ceph.com -o noatime 2026-02-12T09:13:23.806 DEBUG:teuthology.orchestra.run.trial044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-12T09:13:23.874 DEBUG:teuthology.orchestra.run.trial044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-12T09:13:23.941 DEBUG:teuthology.orchestra.run.trial044:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-12T09:13:24.006 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-12T09:13:24.006 INFO:tasks.ceph:role: osd.1 2026-02-12T09:13:24.006 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial044.front.sepia.ceph.com 2026-02-12T09:13:24.006 DEBUG:teuthology.orchestra.run.trial044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-12T09:13:24.072 INFO:teuthology.orchestra.run.trial044.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-12T09:13:24.072 INFO:teuthology.orchestra.run.trial044.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-12T09:13:24.073 INFO:teuthology.orchestra.run.trial044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-12T09:13:24.073 INFO:teuthology.orchestra.run.trial044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-12T09:13:24.073 INFO:teuthology.orchestra.run.trial044.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-12T09:13:24.073 INFO:teuthology.orchestra.run.trial044.stdout: = sunit=0 swidth=0 blks 2026-02-12T09:13:24.073 INFO:teuthology.orchestra.run.trial044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-12T09:13:24.073 INFO:teuthology.orchestra.run.trial044.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-12T09:13:24.073 INFO:teuthology.orchestra.run.trial044.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-12T09:13:24.073 INFO:teuthology.orchestra.run.trial044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-12T09:13:24.631 INFO:teuthology.orchestra.run.trial044.stdout:Discarding blocks...Done. 2026-02-12T09:13:24.633 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial044.front.sepia.ceph.com -o noatime 2026-02-12T09:13:24.633 DEBUG:teuthology.orchestra.run.trial044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-12T09:13:24.706 DEBUG:teuthology.orchestra.run.trial044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-12T09:13:24.773 DEBUG:teuthology.orchestra.run.trial044:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-12T09:13:24.838 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-12T09:13:24.838 INFO:tasks.ceph:role: osd.2 2026-02-12T09:13:24.838 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial044.front.sepia.ceph.com 2026-02-12T09:13:24.838 DEBUG:teuthology.orchestra.run.trial044:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-12T09:13:24.902 INFO:teuthology.orchestra.run.trial044.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-12T09:13:24.902 INFO:teuthology.orchestra.run.trial044.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-12T09:13:24.902 INFO:teuthology.orchestra.run.trial044.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-12T09:13:24.903 INFO:teuthology.orchestra.run.trial044.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-12T09:13:24.903 INFO:teuthology.orchestra.run.trial044.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-12T09:13:24.903 INFO:teuthology.orchestra.run.trial044.stdout: = sunit=0 swidth=0 blks 2026-02-12T09:13:24.903 INFO:teuthology.orchestra.run.trial044.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-12T09:13:24.903 INFO:teuthology.orchestra.run.trial044.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-12T09:13:24.903 INFO:teuthology.orchestra.run.trial044.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-12T09:13:24.903 INFO:teuthology.orchestra.run.trial044.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-12T09:13:25.458 INFO:teuthology.orchestra.run.trial044.stdout:Discarding blocks...Done. 2026-02-12T09:13:25.460 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial044.front.sepia.ceph.com -o noatime 2026-02-12T09:13:25.460 DEBUG:teuthology.orchestra.run.trial044:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-12T09:13:25.530 DEBUG:teuthology.orchestra.run.trial044:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-12T09:13:25.596 DEBUG:teuthology.orchestra.run.trial044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-12T09:13:25.744 INFO:teuthology.orchestra.run.trial044.stderr:INFO  osd - OSD::main: parsing early config 2026-02-12T09:13:25.805 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:25,802 seastar - Reactor backend: linux-aio 2026-02-12T09:13:25.818 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:25,815 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:13:25.819 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:25,815 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:13:25.820 INFO:teuthology.orchestra.run.trial044.stderr:==20912==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T09:13:25.880 INFO:teuthology.orchestra.run.trial044.stderr:DEBUG 2026-02-12 09:13:25,877 [shard 0:main] seastar - reactor::drain 2026-02-12T09:13:28.000 INFO:teuthology.orchestra.run.trial044.stderr:----------------------------------------------------- 2026-02-12T09:13:28.000 INFO:teuthology.orchestra.run.trial044.stderr:Suppressions used: 2026-02-12T09:13:28.000 INFO:teuthology.orchestra.run.trial044.stderr: count bytes template 2026-02-12T09:13:28.000 INFO:teuthology.orchestra.run.trial044.stderr: 1 8 MallocExtension::Register 2026-02-12T09:13:28.000 INFO:teuthology.orchestra.run.trial044.stderr:----------------------------------------------------- 2026-02-12T09:13:28.000 INFO:teuthology.orchestra.run.trial044.stderr: 2026-02-12T09:13:28.018 INFO:teuthology.orchestra.run.trial044.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-12T09:13:28.019 INFO:teuthology.orchestra.run.trial044.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-12T09:13:28.063 INFO:teuthology.orchestra.run.trial044.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-12T09:13:28.063 INFO:teuthology.orchestra.run.trial044.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-12T09:13:28.070 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:28,067 seastar - Reactor backend: linux-aio 2026-02-12T09:13:28.085 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:28,082 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:13:28.085 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:28,082 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:13:28.089 INFO:teuthology.orchestra.run.trial044.stderr:==20910==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T09:13:28.089 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:28,086 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-12T09:13:28.230 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x9bf2094 0x9d2f72c 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:28.231 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x9c3e11c 0x9c3e7b8 0x9c46c5f 0x9c4733f 0x9d2fe8c 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:28.231 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0xc95b32 0xca6f17 0x9c86b00 0x9d1fed0 0x9d32ab0 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:28.380 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x361f /lib64/libasan.so.8+0x151b7 0x1124fe8 0xf2dc91 0x9c3d803 0x9c3dccd 0x9c3df40 0x9c3e1f6 0x9c3e7b8 0x9c46c5f 0x9c4733f 0x9d2fe8c 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:28.380 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0xb692fc 0x1090438 0xf2dc9c 0x9d208b5 0x9d20bfa 0x9d20f7d 0x9d310d4 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:28.380 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x10905ec 0xf2dc9c 0x9c3d9d0 0x9c3dccd 0x9c3df40 0x9c3fc67 0x9d25bc1 0x9d25e30 0x9d300be 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:30.555 INFO:teuthology.orchestra.run.trial044.stderr:----------------------------------------------------- 2026-02-12T09:13:30.555 INFO:teuthology.orchestra.run.trial044.stderr:Suppressions used: 2026-02-12T09:13:30.555 INFO:teuthology.orchestra.run.trial044.stderr: count bytes template 2026-02-12T09:13:30.555 INFO:teuthology.orchestra.run.trial044.stderr: 1 8 MallocExtension::Register 2026-02-12T09:13:30.555 INFO:teuthology.orchestra.run.trial044.stderr:----------------------------------------------------- 2026-02-12T09:13:30.555 INFO:teuthology.orchestra.run.trial044.stderr: 2026-02-12T09:13:30.587 DEBUG:teuthology.orchestra.run.trial044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-12T09:13:30.612 DEBUG:teuthology.orchestra.run.trial044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-12T09:13:30.763 INFO:teuthology.orchestra.run.trial044.stderr:INFO  osd - OSD::main: parsing early config 2026-02-12T09:13:30.814 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:30,811 seastar - Reactor backend: linux-aio 2026-02-12T09:13:30.831 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:30,828 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:13:30.831 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:30,828 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:13:30.833 INFO:teuthology.orchestra.run.trial044.stderr:==20977==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T09:13:30.894 INFO:teuthology.orchestra.run.trial044.stderr:DEBUG 2026-02-12 09:13:30,891 [shard 0:main] seastar - reactor::drain 2026-02-12T09:13:33.001 INFO:teuthology.orchestra.run.trial044.stderr:----------------------------------------------------- 2026-02-12T09:13:33.001 INFO:teuthology.orchestra.run.trial044.stderr:Suppressions used: 2026-02-12T09:13:33.001 INFO:teuthology.orchestra.run.trial044.stderr: count bytes template 2026-02-12T09:13:33.002 INFO:teuthology.orchestra.run.trial044.stderr: 1 8 MallocExtension::Register 2026-02-12T09:13:33.002 INFO:teuthology.orchestra.run.trial044.stderr:----------------------------------------------------- 2026-02-12T09:13:33.002 INFO:teuthology.orchestra.run.trial044.stderr: 2026-02-12T09:13:33.021 INFO:teuthology.orchestra.run.trial044.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-12T09:13:33.021 INFO:teuthology.orchestra.run.trial044.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-12T09:13:33.072 INFO:teuthology.orchestra.run.trial044.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-12T09:13:33.072 INFO:teuthology.orchestra.run.trial044.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-12T09:13:33.079 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:33,076 seastar - Reactor backend: linux-aio 2026-02-12T09:13:33.090 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:33,087 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:13:33.090 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:33,087 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:13:33.094 INFO:teuthology.orchestra.run.trial044.stderr:==20975==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T09:13:33.094 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:33,091 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-12T09:13:33.236 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15812 0x35e89ab 0x35e8b2b 0x36049a8 0x9c4fecb 0x9c4ffd3 0x9d0fd7b 0x9d0ff10 0x9d102a2 0x9d11889 0x9d12ba4 0x9d12fc4 0x9d2f84f 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:33.236 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1582a 0x9cf8efa 0x9cf906b 0x9cf92db 0x9cf93e0 0x9d2b7eb 0x9d2bf8e 0x9d36884 0x788188f 0x7bd6cd5 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:33.236 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0xa5243 0xb693e5 0xb756af 0xbb8eaa 0xe05b40 0x18e9658 0x9d12c1f 0x9d12fc4 0x9d2f84f 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:33.384 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0xcb01e3 0x109056c 0xf2dc9c 0x9c3d9d0 0x9c3dccd 0x9c3df40 0x9c3e1f6 0x9c3e7b8 0x9c46c5f 0x9c4733f 0x9d2fe8c 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:33.384 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x9c3d857 0x9c3dccd 0x9c3df40 0x9c3fc67 0x9c401b4 0x9c403b3 0x9d256d5 0x9d25e30 0x9d300be 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:33.384 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x10905ec 0xf2dc9c 0x9c3d803 0x9c3dccd 0x9c3df40 0x9c3fc67 0x9c401b4 0x9c403b3 0x9d256d5 0x9d25e30 0x9d300be 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:35.596 INFO:teuthology.orchestra.run.trial044.stderr:----------------------------------------------------- 2026-02-12T09:13:35.596 INFO:teuthology.orchestra.run.trial044.stderr:Suppressions used: 2026-02-12T09:13:35.596 INFO:teuthology.orchestra.run.trial044.stderr: count bytes template 2026-02-12T09:13:35.596 INFO:teuthology.orchestra.run.trial044.stderr: 1 8 MallocExtension::Register 2026-02-12T09:13:35.596 INFO:teuthology.orchestra.run.trial044.stderr:----------------------------------------------------- 2026-02-12T09:13:35.596 INFO:teuthology.orchestra.run.trial044.stderr: 2026-02-12T09:13:35.628 DEBUG:teuthology.orchestra.run.trial044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-12T09:13:35.652 DEBUG:teuthology.orchestra.run.trial044:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-12T09:13:35.802 INFO:teuthology.orchestra.run.trial044.stderr:INFO  osd - OSD::main: parsing early config 2026-02-12T09:13:35.857 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:35,854 seastar - Reactor backend: linux-aio 2026-02-12T09:13:35.870 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:35,867 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:13:35.870 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:35,867 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:13:35.872 INFO:teuthology.orchestra.run.trial044.stderr:==21040==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T09:13:35.932 INFO:teuthology.orchestra.run.trial044.stderr:DEBUG 2026-02-12 09:13:35,929 [shard 0:main] seastar - reactor::drain 2026-02-12T09:13:38.050 INFO:teuthology.orchestra.run.trial044.stderr:----------------------------------------------------- 2026-02-12T09:13:38.051 INFO:teuthology.orchestra.run.trial044.stderr:Suppressions used: 2026-02-12T09:13:38.051 INFO:teuthology.orchestra.run.trial044.stderr: count bytes template 2026-02-12T09:13:38.051 INFO:teuthology.orchestra.run.trial044.stderr: 1 8 MallocExtension::Register 2026-02-12T09:13:38.051 INFO:teuthology.orchestra.run.trial044.stderr:----------------------------------------------------- 2026-02-12T09:13:38.051 INFO:teuthology.orchestra.run.trial044.stderr: 2026-02-12T09:13:38.070 INFO:teuthology.orchestra.run.trial044.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-12T09:13:38.070 INFO:teuthology.orchestra.run.trial044.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-12T09:13:38.115 INFO:teuthology.orchestra.run.trial044.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-12T09:13:38.115 INFO:teuthology.orchestra.run.trial044.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-12T09:13:38.122 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:38,119 seastar - Reactor backend: linux-aio 2026-02-12T09:13:38.136 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:38,133 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:13:38.136 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:38,133 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-12T09:13:38.140 INFO:teuthology.orchestra.run.trial044.stderr:==21038==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-12T09:13:38.141 INFO:teuthology.orchestra.run.trial044.stderr:INFO 2026-02-12 09:13:38,138 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-12T09:13:38.285 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x361f /lib64/libasan.so.8+0x151b7 0x9c3d86c 0x9c3dccd 0x9c3df40 0x9c3e1f6 0x9c3eca5 0x9c3f1a3 0x9c3f7d3 0x9c3f93d 0x9d1fcad 0x9d32ab0 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:38.285 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc08 /lib64/libasan.so.8+0x1006bc /lib64/libasan.so.8+0xc6619 /lib64/libasan.so.8+0x151b7 0x1124fe8 0xf2dc91 0x9c3d9d0 0x9c3dccd 0x9c3df40 0x9c3fc67 0x9c401b4 0x9c403b3 0x9d256d5 0x9d25e30 0x9d300be 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:38.285 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc65 0xe1d15b 0x9d35c7b 0x788188f 0x7bd6cd5 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f906da 0x6f944db 0x6f9a085 0x7408a8a 0x742b492 0x727f2b9 0x727f3b1 0x7351548 0x7351601 0x73517b8 0x742b513 0x727f42e 0x727f4b3 0x72ab251 0x741391a 0x741399f 0x7413a7d 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:38.285 INFO:teuthology.orchestra.run.trial044.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff83e61060 0xffffffff83e62473 0xffffffff83dc95c6 0xffffffff83dd3b7b 0xffffffff83dd3fdb 0xffffffff83a910d2 0xffffffff8474d451 0xffffffff84800ba2 2026-02-12T09:13:38.433 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0x316a758 0x9c3df2e 0x9c3e1f6 0x9c3eca5 0x9c3f1a3 0x9c3f7d3 0x9c3f93d 0x9d1fcad 0x9d32ab0 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:38.433 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16b377 /lib64/libasan.so.8+0xb0581 /lib64/libasan.so.8+0xb1477 0xb69317 0x1090438 0xf2dc9c 0x9d2060d 0x9d20bfa 0x9d20f7d 0x9d310d4 0x9d34537 0x9cf4b05 0x9cf3b05 0x7bd73a3 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:38.433 INFO:teuthology.orchestra.run.trial044.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54ed3 0x9e14eec 0x9e151d5 0x9e75b73 0x9e76366 0x9e765d4 0x9e7671c 0x9e76bdb /lib64/libc.so.6+0x3fc2f 0xb6d692 0x9d35f60 0x788188f 0x7bd72da 0x7be0f12 0x7a3027b 0x7a6d86f 0x710f04d 0x6f336f2 0x6f35a5a 0x7473988 0x729685f 0x7296961 0x7296a59 0x7350714 0x73507cd 0x7350980 0x7350ab8 0x7350b33 0x7350bb8 0x7350d13 0x7413397 0x7473a11 0x727dcf4 0x7296bd6 0x73a49b2 0x741fcb0 0x741fd2b 0x741fda6 0x74e35b8 0x75d8e38 0x9ecfa3b 0x9e23d10 0x9efd4ab 0x9f65c8e 0x9f8071e 0x9f80d9a 0x26e4fbb 0xa58eca0 /lib64/libasan.so.8+0x2a589 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-12T09:13:40.614 INFO:teuthology.orchestra.run.trial044.stderr:----------------------------------------------------- 2026-02-12T09:13:40.614 INFO:teuthology.orchestra.run.trial044.stderr:Suppressions used: 2026-02-12T09:13:40.614 INFO:teuthology.orchestra.run.trial044.stderr: count bytes template 2026-02-12T09:13:40.615 INFO:teuthology.orchestra.run.trial044.stderr: 1 8 MallocExtension::Register 2026-02-12T09:13:40.615 INFO:teuthology.orchestra.run.trial044.stderr:----------------------------------------------------- 2026-02-12T09:13:40.615 INFO:teuthology.orchestra.run.trial044.stderr: 2026-02-12T09:13:40.646 DEBUG:teuthology.orchestra.run.trial044:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-12T09:13:40.671 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-12T09:13:40.672 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:13:40.672 DEBUG:teuthology.orchestra.run.trial044:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-12T09:13:40.736 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:13:40.737 DEBUG:teuthology.orchestra.run.trial044:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-12T09:13:40.801 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:13:40.801 DEBUG:teuthology.orchestra.run.trial044:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-12T09:13:40.866 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:13:40.866 DEBUG:teuthology.orchestra.run.trial044:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-12T09:13:40.932 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:13:40.932 DEBUG:teuthology.orchestra.run.trial044:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-12T09:13:40.988 INFO:tasks.ceph:Adding keys to all mons... 2026-02-12T09:13:40.988 DEBUG:teuthology.orchestra.run.trial044:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-12T09:13:41.050 INFO:teuthology.orchestra.run.trial044.stdout:[mgr.x] 2026-02-12T09:13:41.050 INFO:teuthology.orchestra.run.trial044.stdout: key = AQCzmY1pbFxxAhAA+MEeLgz1jBwVruNaIn3tig== 2026-02-12T09:13:41.050 INFO:teuthology.orchestra.run.trial044.stdout:[osd.0] 2026-02-12T09:13:41.050 INFO:teuthology.orchestra.run.trial044.stdout: key = AQC4mY1pffIYCRAA3asHUcIl8KE27Zihy1UUGg== 2026-02-12T09:13:41.050 INFO:teuthology.orchestra.run.trial044.stdout:[osd.1] 2026-02-12T09:13:41.051 INFO:teuthology.orchestra.run.trial044.stdout: key = AQC9mY1pDC9kCRAAmoQ5BaX4icCqMKzPWLdDdw== 2026-02-12T09:13:41.051 INFO:teuthology.orchestra.run.trial044.stdout:[osd.2] 2026-02-12T09:13:41.051 INFO:teuthology.orchestra.run.trial044.stdout: key = AQDCmY1puK5UDBAACoTZDwguATZVNRMJZw1KGA== 2026-02-12T09:13:41.051 INFO:teuthology.orchestra.run.trial044.stdout:[client.0] 2026-02-12T09:13:41.051 INFO:teuthology.orchestra.run.trial044.stdout: key = AQCzmY1pdAchBRAAJ4CVHgC48krpS8V4xUb00w== 2026-02-12T09:13:41.052 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-12T09:13:41.139 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-12T09:13:41.188 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-12T09:13:41.237 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-12T09:13:41.285 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-12T09:13:41.336 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-12T09:13:41.337 DEBUG:teuthology.orchestra.run.trial044:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-12T09:13:41.362 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-12T09:13:41.447 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:41.443+0000 7fb465c8ac00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:41.447 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:41.443+0000 7fb465c8ac00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:41.454 DEBUG:teuthology.orchestra.run.trial044:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-12T09:13:41.479 DEBUG:teuthology.orchestra.run.trial044:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-12T09:13:41.535 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-12T09:13:41.535 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-12T09:13:41.535 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-12T09:13:41.577 INFO:tasks.ceph.mon.a:Started 2026-02-12T09:13:41.578 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-12T09:13:41.578 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-12T09:13:41.578 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-12T09:13:41.581 INFO:tasks.ceph.mgr.x:Started 2026-02-12T09:13:41.581 DEBUG:tasks.ceph:set 0 configs 2026-02-12T09:13:41.581 DEBUG:teuthology.orchestra.run.trial044:> sudo ceph --cluster ceph config dump 2026-02-12T09:13:41.641 INFO:tasks.ceph.mon.a.trial044.stderr:2026-02-12T09:13:41.637+0000 7f68d8c97c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:41.641 INFO:tasks.ceph.mon.a.trial044.stderr:2026-02-12T09:13:41.637+0000 7f68d8c97c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:41.641 INFO:tasks.ceph.mon.a.trial044.stderr:2026-02-12T09:13:41.637+0000 7f68d8c97c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:41.645 INFO:tasks.ceph.mgr.x.trial044.stderr:2026-02-12T09:13:41.641+0000 7f7f7be2ffc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:41.659 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:41.652+0000 7f2f0ef9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:41.660 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:41.656+0000 7f2f0ef9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:41.691 INFO:teuthology.orchestra.run.trial044.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-12T09:13:41.699 INFO:tasks.ceph:Setting crush tunables to default 2026-02-12T09:13:41.699 DEBUG:teuthology.orchestra.run.trial044:> sudo ceph --cluster ceph osd crush tunables default 2026-02-12T09:13:41.775 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:41.767+0000 7fc79223d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:41.776 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:41.772+0000 7fc79223d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:41.810 INFO:teuthology.orchestra.run.trial044.stderr:adjusted tunables profile to default 2026-02-12T09:13:41.818 INFO:tasks.ceph:check_enable_crimson: True 2026-02-12T09:13:41.818 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-12T09:13:41.818 DEBUG:teuthology.orchestra.run.trial044:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-12T09:13:41.848 INFO:tasks.ceph.mgr.x.trial044.stderr:2026-02-12T09:13:41.844+0000 7f7f7be2ffc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:41.849 INFO:tasks.ceph.mgr.x.trial044.stderr:2026-02-12T09:13:41.844+0000 7f7f7be2ffc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:41.894 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:41.888+0000 7fd4ba1c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:41.895 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:41.891+0000 7fd4ba1c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:41.933 INFO:tasks.ceph.mon.a.trial044.stderr:2026-02-12T09:13:41.929+0000 7f68d03f1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-12T09:13:41.933 INFO:tasks.ceph.mon.a.trial044.stderr:Please be aware that this feature is experimental, untested, 2026-02-12T09:13:41.933 INFO:tasks.ceph.mon.a.trial044.stderr:unsupported, and may result in data corruption, data loss, 2026-02-12T09:13:41.933 INFO:tasks.ceph.mon.a.trial044.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-12T09:13:41.934 INFO:tasks.ceph.mon.a.trial044.stderr:feature with important data. 2026-02-12T09:13:41.934 INFO:tasks.ceph.mon.a.trial044.stderr: 2026-02-12T09:13:42.687 INFO:tasks.ceph.mgr.x.trial044.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-12T09:13:42.688 INFO:tasks.ceph.mgr.x.trial044.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-12T09:13:42.688 INFO:tasks.ceph.mgr.x.trial044.stderr: from numpy import show_config as show_numpy_config 2026-02-12T09:13:42.820 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-12T09:13:42.820 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:13:42.821 DEBUG:teuthology.orchestra.run.trial044:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-12T09:13:42.885 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:13:42.885 DEBUG:teuthology.orchestra.run.trial044:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-12T09:13:42.948 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:13:42.949 DEBUG:teuthology.orchestra.run.trial044:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-12T09:13:43.013 DEBUG:teuthology.orchestra.run.trial044:> sudo ceph --cluster ceph osd new 8220c66b-e1a8-4e69-bea3-aaede23eb114 0 2026-02-12T09:13:43.127 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:43.120+0000 7f831a25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:43.128 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:43.124+0000 7f831a25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:43.165 INFO:teuthology.orchestra.run.trial044.stdout:0 2026-02-12T09:13:43.174 DEBUG:teuthology.orchestra.run.trial044:> sudo ceph --cluster ceph osd new f84815ec-7cf2-408a-9d65-9863e4a0aedf 1 2026-02-12T09:13:43.248 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:43.241+0000 7f28824c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:43.249 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:43.245+0000 7f28824c2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:43.286 INFO:teuthology.orchestra.run.trial044.stdout:1 2026-02-12T09:13:43.294 DEBUG:teuthology.orchestra.run.trial044:> sudo ceph --cluster ceph osd new 7c20d125-d252-446b-89de-fec38807a87d 2 2026-02-12T09:13:43.369 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:43.362+0000 7f6ee2cc2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:43.371 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:43.367+0000 7f6ee2cc2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:43.406 INFO:teuthology.orchestra.run.trial044.stdout:2 2026-02-12T09:13:43.415 DEBUG:teuthology.orchestra.run.trial044:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-12T09:13:43.443 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-12T09:13:43.444 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-12T09:13:43.486 INFO:tasks.ceph.osd.0:Started 2026-02-12T09:13:43.486 DEBUG:teuthology.orchestra.run.trial044:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-12T09:13:43.516 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-12T09:13:43.516 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-12T09:13:43.558 INFO:tasks.ceph.osd.1:Started 2026-02-12T09:13:43.558 DEBUG:teuthology.orchestra.run.trial044:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-12T09:13:43.587 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-12T09:13:43.588 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-12T09:13:43.631 INFO:tasks.ceph.osd.2:Started 2026-02-12T09:13:43.631 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-12T09:13:43.707 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:43.701+0000 7feb453c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:43.708 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:43.704+0000 7feb453c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:43.743 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:13:43.743 INFO:teuthology.orchestra.run.trial044.stdout:{"epoch":6,"fsid":"74f73dcc-542d-4c1a-9175-f74dd177fad8","created":"2026-02-12T09:13:41.648752+0000","modified":"2026-02-12T09:13:43.403255+0000","last_up_change":"0.000000","last_in_change":"2026-02-12T09:13:43.403255+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"8220c66b-e1a8-4e69-bea3-aaede23eb114","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f84815ec-7cf2-408a-9d65-9863e4a0aedf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7c20d125-d252-446b-89de-fec38807a87d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T09:13:43.753 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-12T09:13:43.753 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-12T09:13:44.055 DEBUG:teuthology.orchestra.run.trial044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-12T09:13:44.124 INFO:teuthology.misc.health.trial044.stderr:2026-02-12T09:13:44.119+0000 7f1f68851640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:44.126 INFO:teuthology.misc.health.trial044.stderr:2026-02-12T09:13:44.122+0000 7f1f68851640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:44.160 INFO:teuthology.misc.health.trial044.stdout: 2026-02-12T09:13:44.160 INFO:teuthology.misc.health.trial044.stdout:{"epoch":6,"fsid":"74f73dcc-542d-4c1a-9175-f74dd177fad8","created":"2026-02-12T09:13:41.648752+0000","modified":"2026-02-12T09:13:43.403255+0000","last_up_change":"0.000000","last_in_change":"2026-02-12T09:13:43.403255+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"8220c66b-e1a8-4e69-bea3-aaede23eb114","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f84815ec-7cf2-408a-9d65-9863e4a0aedf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7c20d125-d252-446b-89de-fec38807a87d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T09:13:44.169 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-12T09:13:45.111 INFO:tasks.ceph.mgr.x.trial044.stderr:2026-02-12T09:13:45.107+0000 7f7e9692a640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-12T09:13:50.471 DEBUG:teuthology.orchestra.run.trial044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-12T09:13:50.541 INFO:teuthology.misc.health.trial044.stderr:2026-02-12T09:13:50.535+0000 7fd2e6d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:50.542 INFO:teuthology.misc.health.trial044.stderr:2026-02-12T09:13:50.538+0000 7fd2e6d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:50.652 INFO:teuthology.misc.health.trial044.stdout: 2026-02-12T09:13:50.652 INFO:teuthology.misc.health.trial044.stdout:{"epoch":8,"fsid":"74f73dcc-542d-4c1a-9175-f74dd177fad8","created":"2026-02-12T09:13:41.648752+0000","modified":"2026-02-12T09:13:50.206602+0000","last_up_change":"0.000000","last_in_change":"2026-02-12T09:13:43.403255+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"8220c66b-e1a8-4e69-bea3-aaede23eb114","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"f84815ec-7cf2-408a-9d65-9863e4a0aedf","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"7c20d125-d252-446b-89de-fec38807a87d","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T09:13:50.660 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-12T09:13:56.963 DEBUG:teuthology.orchestra.run.trial044:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-12T09:13:57.035 INFO:teuthology.misc.health.trial044.stderr:2026-02-12T09:13:57.028+0000 7f9cba25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:57.036 INFO:teuthology.misc.health.trial044.stderr:2026-02-12T09:13:57.031+0000 7f9cba25a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:57.149 INFO:teuthology.misc.health.trial044.stdout: 2026-02-12T09:13:57.150 INFO:teuthology.misc.health.trial044.stdout:{"epoch":14,"fsid":"74f73dcc-542d-4c1a-9175-f74dd177fad8","created":"2026-02-12T09:13:41.648752+0000","modified":"2026-02-12T09:13:56.226040+0000","last_up_change":"2026-02-12T09:13:53.213676+0000","last_in_change":"2026-02-12T09:13:43.403255+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-12T09:13:54.119695+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8220c66b-e1a8-4e69-bea3-aaede23eb114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6802","nonce":4197462059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6809","nonce":4197462059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6813","nonce":4197462059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6812","nonce":4197462059}]},"public_addr":"10.20.193.44:6802/4197462059","cluster_addr":"10.20.193.44:6809/4197462059","heartbeat_back_addr":"10.20.193.44:6813/4197462059","heartbeat_front_addr":"10.20.193.44:6812/4197462059","state":["exists","up"]},{"osd":1,"uuid":"f84815ec-7cf2-408a-9d65-9863e4a0aedf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6805","nonce":3319702322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6806","nonce":3319702322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6810","nonce":3319702322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6811","nonce":3319702322}]},"public_addr":"10.20.193.44:6805/3319702322","cluster_addr":"10.20.193.44:6806/3319702322","heartbeat_back_addr":"10.20.193.44:6810/3319702322","heartbeat_front_addr":"10.20.193.44:6811/3319702322","state":["exists","up"]},{"osd":2,"uuid":"7c20d125-d252-446b-89de-fec38807a87d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6804","nonce":4204775452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6803","nonce":4204775452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6807","nonce":4204775452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6808","nonce":4204775452}]},"public_addr":"10.20.193.44:6804/4204775452","cluster_addr":"10.20.193.44:6803/4204775452","heartbeat_back_addr":"10.20.193.44:6807/4204775452","heartbeat_front_addr":"10.20.193.44:6808/4204775452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T09:13:57.158 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-12T09:13:57.159 INFO:tasks.ceph:Creating RBD pool 2026-02-12T09:13:57.159 DEBUG:teuthology.orchestra.run.trial044:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-12T09:13:57.233 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:57.225+0000 7f60a49a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:57.234 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:57.229+0000 7f60a49a4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:58.238 INFO:teuthology.orchestra.run.trial044.stderr:pool 'rbd' created 2026-02-12T09:13:58.251 DEBUG:teuthology.orchestra.run.trial044:> rbd --cluster ceph pool init rbd 2026-02-12T09:13:58.293 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:58.288+0000 7f1d0fc14240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:58.293 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:58.288+0000 7f1d0fc14240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:13:58.294 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:13:58.289+0000 7f1d0fc14240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:01.274 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-12T09:14:01.275 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-12T09:14:01.275 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-12T09:14:01.391 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:01.385+0000 7f8b27964640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:01.392 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:01.388+0000 7f8b27964640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:01.501 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:14:01.511 INFO:teuthology.orchestra.run.trial044.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-12T09:14:01.512 INFO:tasks.ceph_manager:config epoch is 1 2026-02-12T09:14:01.512 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-12T09:14:01.512 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-12T09:14:01.512 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-12T09:14:01.588 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:01.582+0000 7fdd2c79c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:01.589 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:01.585+0000 7fdd2c79c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:01.728 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:14:01.737 INFO:teuthology.orchestra.run.trial044.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6800","nonce":1430653267},{"type":"v1","addr":"10.20.193.44:6801","nonce":1430653267}]},"active_addr":"10.20.193.44:6801/1430653267","active_change":"2026-02-12T09:13:44.103041+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.44:0","nonce":3354555629}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.44:0","nonce":1462352795}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.44:0","nonce":2289392728}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.44:0","nonce":539983711}]}]} 2026-02-12T09:14:01.740 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-12T09:14:01.741 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-12T09:14:01.741 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-12T09:14:01.818 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:01.812+0000 7f682ca1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:01.819 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:01.815+0000 7f682ca1a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:01.931 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:14:01.931 INFO:teuthology.orchestra.run.trial044.stdout:{"epoch":18,"fsid":"74f73dcc-542d-4c1a-9175-f74dd177fad8","created":"2026-02-12T09:13:41.648752+0000","modified":"2026-02-12T09:14:01.242422+0000","last_up_change":"2026-02-12T09:13:53.213676+0000","last_in_change":"2026-02-12T09:13:43.403255+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-12T09:13:54.119695+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-12T09:13:57.343884+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8220c66b-e1a8-4e69-bea3-aaede23eb114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6802","nonce":4197462059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6809","nonce":4197462059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6813","nonce":4197462059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6812","nonce":4197462059}]},"public_addr":"10.20.193.44:6802/4197462059","cluster_addr":"10.20.193.44:6809/4197462059","heartbeat_back_addr":"10.20.193.44:6813/4197462059","heartbeat_front_addr":"10.20.193.44:6812/4197462059","state":["exists","up"]},{"osd":1,"uuid":"f84815ec-7cf2-408a-9d65-9863e4a0aedf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6805","nonce":3319702322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6806","nonce":3319702322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6810","nonce":3319702322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6811","nonce":3319702322}]},"public_addr":"10.20.193.44:6805/3319702322","cluster_addr":"10.20.193.44:6806/3319702322","heartbeat_back_addr":"10.20.193.44:6810/3319702322","heartbeat_front_addr":"10.20.193.44:6811/3319702322","state":["exists","up"]},{"osd":2,"uuid":"7c20d125-d252-446b-89de-fec38807a87d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6804","nonce":4204775452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6803","nonce":4204775452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6807","nonce":4204775452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6808","nonce":4204775452}]},"public_addr":"10.20.193.44:6804/4204775452","cluster_addr":"10.20.193.44:6803/4204775452","heartbeat_back_addr":"10.20.193.44:6807/4204775452","heartbeat_front_addr":"10.20.193.44:6808/4204775452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T09:14:01.941 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-12T09:14:01.941 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-12T09:14:02.017 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:02.011+0000 7fc3faacb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:02.018 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:02.014+0000 7fc3faacb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:02.128 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:14:02.128 INFO:teuthology.orchestra.run.trial044.stdout:{"epoch":18,"fsid":"74f73dcc-542d-4c1a-9175-f74dd177fad8","created":"2026-02-12T09:13:41.648752+0000","modified":"2026-02-12T09:14:01.242422+0000","last_up_change":"2026-02-12T09:13:53.213676+0000","last_in_change":"2026-02-12T09:13:43.403255+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-12T09:13:54.119695+0000","flags":262177,"flags_names":"hashpspool,nopgchange,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-12T09:13:57.343884+0000","flags":270369,"flags_names":"hashpspool,nopgchange,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"8220c66b-e1a8-4e69-bea3-aaede23eb114","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6802","nonce":4197462059}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6809","nonce":4197462059}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6813","nonce":4197462059}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6812","nonce":4197462059}]},"public_addr":"10.20.193.44:6802/4197462059","cluster_addr":"10.20.193.44:6809/4197462059","heartbeat_back_addr":"10.20.193.44:6813/4197462059","heartbeat_front_addr":"10.20.193.44:6812/4197462059","state":["exists","up"]},{"osd":1,"uuid":"f84815ec-7cf2-408a-9d65-9863e4a0aedf","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6805","nonce":3319702322}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6806","nonce":3319702322}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6810","nonce":3319702322}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6811","nonce":3319702322}]},"public_addr":"10.20.193.44:6805/3319702322","cluster_addr":"10.20.193.44:6806/3319702322","heartbeat_back_addr":"10.20.193.44:6810/3319702322","heartbeat_front_addr":"10.20.193.44:6811/3319702322","state":["exists","up"]},{"osd":2,"uuid":"7c20d125-d252-446b-89de-fec38807a87d","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6804","nonce":4204775452}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6803","nonce":4204775452}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6807","nonce":4204775452}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.44:6808","nonce":4204775452}]},"public_addr":"10.20.193.44:6804/4204775452","cluster_addr":"10.20.193.44:6803/4204775452","heartbeat_back_addr":"10.20.193.44:6807/4204775452","heartbeat_front_addr":"10.20.193.44:6808/4204775452","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-12T09:14:02.138 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-12T09:14:02.139 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-12T09:14:02.139 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-12T09:14:02.218 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:02.212+0000 7f46c9dad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:02.218 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:02.212+0000 7f57b28cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:02.220 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:02.216+0000 7f57b28cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:02.220 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:02.216+0000 7f46c9dad640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:02.221 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:02.215+0000 7fb4dfc51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:02.223 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:02.219+0000 7fb4dfc51640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:02.236 INFO:teuthology.orchestra.run.trial044.stdout:42949672969 2026-02-12T09:14:02.236 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-12T09:14:02.236 INFO:teuthology.orchestra.run.trial044.stdout:47244640264 2026-02-12T09:14:02.236 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-12T09:14:02.236 INFO:teuthology.orchestra.run.trial044.stdout:38654705674 2026-02-12T09:14:02.237 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-12T09:14:02.356 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:02.350+0000 7fcec1ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:02.357 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:02.350+0000 7fed9259c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:02.357 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:02.350+0000 7f44d24cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:02.358 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:02.354+0000 7fed9259c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:02.358 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:02.354+0000 7fcec1ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:02.359 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:02.354+0000 7f44d24cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:02.478 INFO:teuthology.orchestra.run.trial044.stdout:42949672969 2026-02-12T09:14:02.479 INFO:teuthology.orchestra.run.trial044.stdout:47244640264 2026-02-12T09:14:02.482 INFO:teuthology.orchestra.run.trial044.stdout:38654705673 2026-02-12T09:14:02.489 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672969 for osd.1 2026-02-12T09:14:02.490 DEBUG:teuthology.parallel:result is None 2026-02-12T09:14:02.491 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640264 got 47244640264 for osd.0 2026-02-12T09:14:02.491 DEBUG:teuthology.parallel:result is None 2026-02-12T09:14:02.494 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705673 for osd.2 2026-02-12T09:14:03.495 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-12T09:14:03.575 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:03.569+0000 7f186d2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:03.576 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:03.572+0000 7f186d2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:03.685 INFO:teuthology.orchestra.run.trial044.stdout:38654705673 2026-02-12T09:14:03.694 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705673 for osd.2 2026-02-12T09:14:04.695 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-12T09:14:04.774 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:04.767+0000 7ff816ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:04.775 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:04.771+0000 7ff816ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:04.888 INFO:teuthology.orchestra.run.trial044.stdout:38654705674 2026-02-12T09:14:04.897 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705674 for osd.2 2026-02-12T09:14:04.897 DEBUG:teuthology.parallel:result is None 2026-02-12T09:14:04.898 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-12T09:14:04.898 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-12T09:14:04.973 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:04.967+0000 7fb9359c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:04.974 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:04.970+0000 7fb9359c1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:05.082 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:14:05.083 INFO:teuthology.orchestra.run.trial044.stderr:dumped all 2026-02-12T09:14:05.092 INFO:teuthology.orchestra.run.trial044.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2026-02-12T09:14:04.107171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62917084,"kb_used_data":62917084,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94369268,"statfs":{"total":161061224448,"available":96634130432,"internally_reserved":0,"allocated":64427094016,"data_stored":64427094016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.881956"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.253913+0000","last_change":"2026-02-12T09:14:01.258444+0000","last_active":"2026-02-12T09:14:01.253913+0000","last_peered":"2026-02-12T09:14:01.253913+0000","last_clean":"2026-02-12T09:14:01.253913+0000","last_became_active":"2026-02-12T09:13:59.271234+0000","last_became_peered":"2026-02-12T09:13:59.271234+0000","last_unstale":"2026-02-12T09:14:01.253913+0000","last_undegraded":"2026-02-12T09:14:01.253913+0000","last_fullsized":"2026-02-12T09:14:01.253913+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_clean_scrub_stamp":"2026-02-12T09:13:58.233173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.255600+0000","last_change":"2026-02-12T09:14:01.257989+0000","last_active":"2026-02-12T09:14:01.255600+0000","last_peered":"2026-02-12T09:14:01.255600+0000","last_clean":"2026-02-12T09:14:01.255600+0000","last_became_active":"2026-02-12T09:13:59.284873+0000","last_became_peered":"2026-02-12T09:13:59.284873+0000","last_unstale":"2026-02-12T09:14:01.255600+0000","last_undegraded":"2026-02-12T09:14:01.255600+0000","last_fullsized":"2026-02-12T09:14:01.255600+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_clean_scrub_stamp":"2026-02-12T09:13:58.233173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.255552+0000","last_change":"2026-02-12T09:14:01.258028+0000","last_active":"2026-02-12T09:14:01.255552+0000","last_peered":"2026-02-12T09:14:01.255552+0000","last_clean":"2026-02-12T09:14:01.255552+0000","last_became_active":"2026-02-12T09:13:59.273669+0000","last_became_peered":"2026-02-12T09:13:59.273669+0000","last_unstale":"2026-02-12T09:14:01.255552+0000","last_undegraded":"2026-02-12T09:14:01.255552+0000","last_fullsized":"2026-02-12T09:14:01.255552+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_clean_scrub_stamp":"2026-02-12T09:13:58.233173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.253702+0000","last_change":"2026-02-12T09:14:01.258631+0000","last_active":"2026-02-12T09:14:01.253702+0000","last_peered":"2026-02-12T09:14:01.253702+0000","last_clean":"2026-02-12T09:14:01.253702+0000","last_became_active":"2026-02-12T09:13:59.275839+0000","last_became_peered":"2026-02-12T09:13:59.275839+0000","last_unstale":"2026-02-12T09:14:01.253702+0000","last_undegraded":"2026-02-12T09:14:01.253702+0000","last_fullsized":"2026-02-12T09:14:01.253702+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_clean_scrub_stamp":"2026-02-12T09:13:58.233173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'1","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.259477+0000","last_change":"2026-02-12T09:14:01.259477+0000","last_active":"2026-02-12T09:14:01.259477+0000","last_peered":"2026-02-12T09:14:01.259477+0000","last_clean":"2026-02-12T09:14:01.259477+0000","last_became_active":"2026-02-12T09:13:59.282565+0000","last_became_peered":"2026-02-12T09:13:59.282565+0000","last_unstale":"2026-02-12T09:14:01.259477+0000","last_undegraded":"2026-02-12T09:14:01.259477+0000","last_fullsized":"2026-02-12T09:14:01.259477+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_clean_scrub_stamp":"2026-02-12T09:13:58.233173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.254285+0000","last_change":"2026-02-12T09:14:01.256099+0000","last_active":"2026-02-12T09:14:01.254285+0000","last_peered":"2026-02-12T09:14:01.254285+0000","last_clean":"2026-02-12T09:14:01.254285+0000","last_became_active":"2026-02-12T09:13:59.286149+0000","last_became_peered":"2026-02-12T09:13:59.286149+0000","last_unstale":"2026-02-12T09:14:01.254285+0000","last_undegraded":"2026-02-12T09:14:01.254285+0000","last_fullsized":"2026-02-12T09:14:01.254285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_clean_scrub_stamp":"2026-02-12T09:13:58.233173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.252551+0000","last_change":"2026-02-12T09:14:01.254501+0000","last_active":"2026-02-12T09:14:01.252551+0000","last_peered":"2026-02-12T09:14:01.252551+0000","last_clean":"2026-02-12T09:14:01.252551+0000","last_became_active":"2026-02-12T09:13:59.284565+0000","last_became_peered":"2026-02-12T09:13:59.284565+0000","last_unstale":"2026-02-12T09:14:01.252551+0000","last_undegraded":"2026-02-12T09:14:01.252551+0000","last_fullsized":"2026-02-12T09:14:01.252551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_clean_scrub_stamp":"2026-02-12T09:13:58.233173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.255571+0000","last_change":"2026-02-12T09:14:01.257799+0000","last_active":"2026-02-12T09:14:01.255571+0000","last_peered":"2026-02-12T09:14:01.255571+0000","last_clean":"2026-02-12T09:14:01.255571+0000","last_became_active":"2026-02-12T09:13:59.270512+0000","last_became_peered":"2026-02-12T09:13:59.270512+0000","last_unstale":"2026-02-12T09:14:01.255571+0000","last_undegraded":"2026-02-12T09:14:01.255571+0000","last_fullsized":"2026-02-12T09:14:01.255571+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_clean_scrub_stamp":"2026-02-12T09:13:58.233173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":90,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.253793+0000","last_change":"2026-02-12T09:13:55.255372+0000","last_active":"2026-02-12T09:14:01.253793+0000","last_peered":"2026-02-12T09:14:01.253793+0000","last_clean":"2026-02-12T09:14:01.253793+0000","last_became_active":"2026-02-12T09:13:55.248261+0000","last_became_peered":"2026-02-12T09:13:55.248261+0000","last_unstale":"2026-02-12T09:14:01.253793+0000","last_undegraded":"2026-02-12T09:14:01.253793+0000","last_fullsized":"2026-02-12T09:14:01.253793+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:54.218077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:54.218077+0000","last_clean_scrub_stamp":"2026-02-12T09:13:54.218077+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634114048,"internally_reserved":0,"allocated":64427110400,"data_stored":64427110400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211070976,"internally_reserved":0,"allocated":21476003840,"data_stored":21476003840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972652,"kb_used_data":20972652,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456132,"statfs":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640265,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972612,"kb_used_data":20972612,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456172,"statfs":{"total":53687074816,"available":32211120128,"internally_reserved":0,"allocated":21475954688,"data_stored":21475954688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971820,"kb_used_data":20971820,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456964,"statfs":{"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211070976,"internally_reserved":0,"allocated":21476003840,"data_stored":21476003840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211120128,"internally_reserved":0,"allocated":21475954688,"data_stored":21475954688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211070976,"internally_reserved":0,"allocated":21476003840,"data_stored":21476003840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211922944,"internally_reserved":0,"allocated":21475151872,"data_stored":21475151872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-12T09:14:05.093 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-12T09:14:05.170 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:05.164+0000 7fa38ae5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:05.171 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:05.167+0000 7fa38ae5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:05.278 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:14:05.279 INFO:teuthology.orchestra.run.trial044.stderr:dumped all 2026-02-12T09:14:05.288 INFO:teuthology.orchestra.run.trial044.stdout:{"pg_ready":true,"pg_map":{"version":16,"stamp":"2026-02-12T09:14:04.107171+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62917084,"kb_used_data":62917084,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94369268,"statfs":{"total":161061224448,"available":96634130432,"internally_reserved":0,"allocated":64427094016,"data_stored":64427094016,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.881956"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.253913+0000","last_change":"2026-02-12T09:14:01.258444+0000","last_active":"2026-02-12T09:14:01.253913+0000","last_peered":"2026-02-12T09:14:01.253913+0000","last_clean":"2026-02-12T09:14:01.253913+0000","last_became_active":"2026-02-12T09:13:59.271234+0000","last_became_peered":"2026-02-12T09:13:59.271234+0000","last_unstale":"2026-02-12T09:14:01.253913+0000","last_undegraded":"2026-02-12T09:14:01.253913+0000","last_fullsized":"2026-02-12T09:14:01.253913+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_clean_scrub_stamp":"2026-02-12T09:13:58.233173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.255600+0000","last_change":"2026-02-12T09:14:01.257989+0000","last_active":"2026-02-12T09:14:01.255600+0000","last_peered":"2026-02-12T09:14:01.255600+0000","last_clean":"2026-02-12T09:14:01.255600+0000","last_became_active":"2026-02-12T09:13:59.284873+0000","last_became_peered":"2026-02-12T09:13:59.284873+0000","last_unstale":"2026-02-12T09:14:01.255600+0000","last_undegraded":"2026-02-12T09:14:01.255600+0000","last_fullsized":"2026-02-12T09:14:01.255600+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_clean_scrub_stamp":"2026-02-12T09:13:58.233173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.255552+0000","last_change":"2026-02-12T09:14:01.258028+0000","last_active":"2026-02-12T09:14:01.255552+0000","last_peered":"2026-02-12T09:14:01.255552+0000","last_clean":"2026-02-12T09:14:01.255552+0000","last_became_active":"2026-02-12T09:13:59.273669+0000","last_became_peered":"2026-02-12T09:13:59.273669+0000","last_unstale":"2026-02-12T09:14:01.255552+0000","last_undegraded":"2026-02-12T09:14:01.255552+0000","last_fullsized":"2026-02-12T09:14:01.255552+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_clean_scrub_stamp":"2026-02-12T09:13:58.233173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.253702+0000","last_change":"2026-02-12T09:14:01.258631+0000","last_active":"2026-02-12T09:14:01.253702+0000","last_peered":"2026-02-12T09:14:01.253702+0000","last_clean":"2026-02-12T09:14:01.253702+0000","last_became_active":"2026-02-12T09:13:59.275839+0000","last_became_peered":"2026-02-12T09:13:59.275839+0000","last_unstale":"2026-02-12T09:14:01.253702+0000","last_undegraded":"2026-02-12T09:14:01.253702+0000","last_fullsized":"2026-02-12T09:14:01.253702+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_clean_scrub_stamp":"2026-02-12T09:13:58.233173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'1","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.259477+0000","last_change":"2026-02-12T09:14:01.259477+0000","last_active":"2026-02-12T09:14:01.259477+0000","last_peered":"2026-02-12T09:14:01.259477+0000","last_clean":"2026-02-12T09:14:01.259477+0000","last_became_active":"2026-02-12T09:13:59.282565+0000","last_became_peered":"2026-02-12T09:13:59.282565+0000","last_unstale":"2026-02-12T09:14:01.259477+0000","last_undegraded":"2026-02-12T09:14:01.259477+0000","last_fullsized":"2026-02-12T09:14:01.259477+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_clean_scrub_stamp":"2026-02-12T09:13:58.233173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.254285+0000","last_change":"2026-02-12T09:14:01.256099+0000","last_active":"2026-02-12T09:14:01.254285+0000","last_peered":"2026-02-12T09:14:01.254285+0000","last_clean":"2026-02-12T09:14:01.254285+0000","last_became_active":"2026-02-12T09:13:59.286149+0000","last_became_peered":"2026-02-12T09:13:59.286149+0000","last_unstale":"2026-02-12T09:14:01.254285+0000","last_undegraded":"2026-02-12T09:14:01.254285+0000","last_fullsized":"2026-02-12T09:14:01.254285+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_clean_scrub_stamp":"2026-02-12T09:13:58.233173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.252551+0000","last_change":"2026-02-12T09:14:01.254501+0000","last_active":"2026-02-12T09:14:01.252551+0000","last_peered":"2026-02-12T09:14:01.252551+0000","last_clean":"2026-02-12T09:14:01.252551+0000","last_became_active":"2026-02-12T09:13:59.284565+0000","last_became_peered":"2026-02-12T09:13:59.284565+0000","last_unstale":"2026-02-12T09:14:01.252551+0000","last_undegraded":"2026-02-12T09:14:01.252551+0000","last_fullsized":"2026-02-12T09:14:01.252551+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_clean_scrub_stamp":"2026-02-12T09:13:58.233173+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.255571+0000","last_change":"2026-02-12T09:14:01.257799+0000","last_active":"2026-02-12T09:14:01.255571+0000","last_peered":"2026-02-12T09:14:01.255571+0000","last_clean":"2026-02-12T09:14:01.255571+0000","last_became_active":"2026-02-12T09:13:59.270512+0000","last_became_peered":"2026-02-12T09:13:59.270512+0000","last_unstale":"2026-02-12T09:14:01.255571+0000","last_undegraded":"2026-02-12T09:14:01.255571+0000","last_fullsized":"2026-02-12T09:14:01.255571+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:58.233173+0000","last_clean_scrub_stamp":"2026-02-12T09:13:58.233173+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":90,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-12T09:14:01.253793+0000","last_change":"2026-02-12T09:13:55.255372+0000","last_active":"2026-02-12T09:14:01.253793+0000","last_peered":"2026-02-12T09:14:01.253793+0000","last_clean":"2026-02-12T09:14:01.253793+0000","last_became_active":"2026-02-12T09:13:55.248261+0000","last_became_peered":"2026-02-12T09:13:55.248261+0000","last_unstale":"2026-02-12T09:14:01.253793+0000","last_undegraded":"2026-02-12T09:14:01.253793+0000","last_fullsized":"2026-02-12T09:14:01.253793+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-12T09:13:54.218077+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-12T09:13:54.218077+0000","last_clean_scrub_stamp":"2026-02-12T09:13:54.218077+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634114048,"internally_reserved":0,"allocated":64427110400,"data_stored":64427110400,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211070976,"internally_reserved":0,"allocated":21476003840,"data_stored":21476003840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":1,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972652,"kb_used_data":20972652,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456132,"statfs":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640265,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972612,"kb_used_data":20972612,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456172,"statfs":{"total":53687074816,"available":32211120128,"internally_reserved":0,"allocated":21475954688,"data_stored":21475954688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971820,"kb_used_data":20971820,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456964,"statfs":{"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211070976,"internally_reserved":0,"allocated":21476003840,"data_stored":21476003840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211120128,"internally_reserved":0,"allocated":21475954688,"data_stored":21475954688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211070976,"internally_reserved":0,"allocated":21476003840,"data_stored":21476003840,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211922944,"internally_reserved":0,"allocated":21475151872,"data_stored":21475151872,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-12T09:14:05.290 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-12T09:14:05.290 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-12T09:14:05.290 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-12T09:14:05.290 DEBUG:teuthology.orchestra.run.trial044:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-12T09:14:05.366 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:05.360+0000 7f749b983640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:05.367 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:14:05.363+0000 7f749b983640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:14:05.502 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:14:05.502 INFO:teuthology.orchestra.run.trial044.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-12T09:14:05.512 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-12T09:14:05.512 INFO:teuthology.run_tasks:Running task ssh_keys... 2026-02-12T09:14:05.581 DEBUG:teuthology.orchestra.run.trial044:> echo 'StrictHostKeyChecking no 2026-02-12T09:14:05.582 DEBUG:teuthology.orchestra.run.trial044:> ' > /home/ubuntu/.ssh/config && echo 'UserKnownHostsFile ' /dev/null >> /home/ubuntu/.ssh/config && chmod 600 /home/ubuntu/.ssh/config 2026-02-12T09:14:05.598 INFO:teuthology.task.ssh_keys:generated public key AAAAB3NzaC1yc2EAAAADAQABAAABAQC3xMaF7YZJogSJbFAKZvR+eL5I2RZAEkQBbY62qOil9GquT7gMgaFTAmKhnR8x/qwVZq6Axh+yylOMNVnU23ySB0Qc+BgwcYEn/LHTR9+GENOH1hsJ0vUVO5id8MlK2OOJwb+HTFGF/QipV7n6tPB9M/FhE3SqLi75UBJLsPbQ+G2mUMwtfJ6LfbaVdA2DN3O1wKqMtxMmID0hw5zpyshTdydF7IkgzxQ+K9epYCuSFpeXF2vR/+JHbYjlKWwWJ9bT4jYWdFlNT1rMStI+mn+HuuT90TSV6ZxCNmIuBNpe0VpdfBK6/akiC4HcObPkHSlZw5xmtgdBup9zuZvLHeUl 2026-02-12T09:14:05.598 INFO:teuthology.task.ssh_keys:pushing keys to trial044.front.sepia.ceph.com for ubuntu 2026-02-12T09:14:05.598 DEBUG:teuthology.orchestra.run.trial044:> rm -f -- /home/ubuntu/.ssh/id_rsa 2026-02-12T09:14:05.654 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:14:05.654 DEBUG:teuthology.orchestra.run.trial044:> dd of=/home/ubuntu/.ssh/id_rsa 2026-02-12T09:14:05.654 DEBUG:teuthology.orchestra.run.trial044:> chmod 0500 /home/ubuntu/.ssh/id_rsa 2026-02-12T09:14:05.711 DEBUG:teuthology.orchestra.run.trial044:> rm -f -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-12T09:14:05.767 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:14:05.767 DEBUG:teuthology.orchestra.run.trial044:> dd of=/home/ubuntu/.ssh/id_rsa.pub 2026-02-12T09:14:05.823 DEBUG:teuthology.orchestra.run.trial044:> cp -v -a /home/ubuntu/.ssh/authorized_keys /home/ubuntu/.ssh/authorized_keys_2026-02-12_09:14:05:823172 2026-02-12T09:14:05.879 INFO:teuthology.orchestra.run.trial044.stdout:'/home/ubuntu/.ssh/authorized_keys' -> '/home/ubuntu/.ssh/authorized_keys_2026-02-12_09:14:05:823172' 2026-02-12T09:14:05.880 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:14:05.880 DEBUG:teuthology.orchestra.run.trial044:> dd of=/home/ubuntu/.ssh/authorized_keys conv=notrunc oflag=append 2026-02-12T09:14:05.936 INFO:teuthology.run_tasks:Running task cbt... 2026-02-12T09:14:05.960 INFO:tasks.cbt:cbt configuration is {'cluster': {'user': 'ubuntu', 'head': 'ubuntu@trial044.front.sepia.ceph.com', 'osds': ['ubuntu@trial044.front.sepia.ceph.com'], 'mons': ['ubuntu@trial044.front.sepia.ceph.com'], 'clients': ['ubuntu@trial044.front.sepia.ceph.com'], 'rgws': {'ubuntu@trial044.front.sepia.ceph.com': None}, 'osds_per_node': 3, 'rebuild_every_test': False, 'use_existing': True, 'is_teuthology': True, 'iterations': 1, 'tmp_dir': '/tmp/cbt', 'pool_profiles': {'replicated': {'pg_size': 256, 'pgp_size': 256, 'replication': 'replicated'}}, 'pid_dir': '/var/run/ceph'}, 'benchmarks': {'radosbench': {'concurrent_ops': 4, 'concurrent_procs': 2, 'op_size': [4096], 'pool_monitoring_list': ['collectl'], 'pool_profile': 'replicated', 'run_monitoring_list': ['collectl'], 'time': 60, 'write_only': True}}, 'client_endpoints': None, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}} 2026-02-12T09:14:05.961 DEBUG:teuthology.orchestra.run.trial044:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/cbt 2026-02-12T09:14:05.995 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:14:05.996 DEBUG:teuthology.orchestra.run.trial044:> dd of=/home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2026-02-12T09:14:06.048 DEBUG:teuthology.orchestra.run.trial044:> git clone --depth 1 -b master https://github.com/ceph/cbt.git /home/ubuntu/cephtest/cbt 2026-02-12T09:14:06.110 INFO:teuthology.orchestra.run.trial044.stderr:Cloning into '/home/ubuntu/cephtest/cbt'... 2026-02-12T09:14:06.344 INFO:tasks.cbt:Installing collectl 2026-02-12T09:14:06.344 DEBUG:teuthology.orchestra.run.trial044:> sudo mkdir collectl && cd collectl && sudo wget https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download -O collectl.tar.gz && sudo tar -xvf collectl.tar.gz && cd collectl-4.3.1 && sudo ./INSTALL 2026-02-12T09:14:06.419 INFO:teuthology.orchestra.run.trial044.stderr:--2026-02-12 09:14:06-- https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download 2026-02-12T09:14:11.484 INFO:teuthology.orchestra.run.trial044.stderr:Resolving sourceforge.net (sourceforge.net)... 104.18.12.149, 104.18.13.149, 2606:4700::6812:c95, ... 2026-02-12T09:14:11.493 INFO:teuthology.orchestra.run.trial044.stderr:Connecting to sourceforge.net (sourceforge.net)|104.18.12.149|:443... connected. 2026-02-12T09:14:11.691 INFO:teuthology.orchestra.run.trial044.stderr:HTTP request sent, awaiting response... 302 Found 2026-02-12T09:14:11.691 INFO:teuthology.orchestra.run.trial044.stderr:Location: https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABpjZnjyHH5l3RzDkB93YbryQT3iglOSFSiDdbeP8gTeEEbCJnnvFMeiDM-qEzg2D-B4EqWD73oFhyA_nvZECGeeLsadw%3D%3D&use_mirror=psychz&r= [following] 2026-02-12T09:14:11.692 INFO:teuthology.orchestra.run.trial044.stderr:--2026-02-12 09:14:11-- https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABpjZnjyHH5l3RzDkB93YbryQT3iglOSFSiDdbeP8gTeEEbCJnnvFMeiDM-qEzg2D-B4EqWD73oFhyA_nvZECGeeLsadw%3D%3D&use_mirror=psychz&r= 2026-02-12T09:14:11.705 INFO:teuthology.orchestra.run.trial044.stderr:Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 104.18.12.149, 104.18.13.149, 2606:4700::6812:c95, ... 2026-02-12T09:14:11.718 INFO:teuthology.orchestra.run.trial044.stderr:Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|104.18.12.149|:443... connected. 2026-02-12T09:14:11.908 INFO:teuthology.orchestra.run.trial044.stderr:HTTP request sent, awaiting response... 302 Found 2026-02-12T09:14:11.908 INFO:teuthology.orchestra.run.trial044.stderr:Location: https://psychz.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 [following] 2026-02-12T09:14:11.908 INFO:teuthology.orchestra.run.trial044.stderr:--2026-02-12 09:14:11-- https://psychz.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 2026-02-12T09:14:11.920 INFO:teuthology.orchestra.run.trial044.stderr:Resolving psychz.dl.sourceforge.net (psychz.dl.sourceforge.net)... 208.87.241.191 2026-02-12T09:14:11.985 INFO:teuthology.orchestra.run.trial044.stderr:Connecting to psychz.dl.sourceforge.net (psychz.dl.sourceforge.net)|208.87.241.191|:443... connected. 2026-02-12T09:14:12.282 INFO:teuthology.orchestra.run.trial044.stderr:HTTP request sent, awaiting response... 200 OK 2026-02-12T09:14:12.282 INFO:teuthology.orchestra.run.trial044.stderr:Length: 620693 (606K) [application/x-gzip] 2026-02-12T09:14:12.282 INFO:teuthology.orchestra.run.trial044.stderr:Saving to: ‘collectl.tar.gz’ 2026-02-12T09:14:12.282 INFO:teuthology.orchestra.run.trial044.stderr: 2026-02-12T09:14:12.350 INFO:teuthology.orchestra.run.trial044.stderr: 0K .......... .......... .......... .......... .......... 8% 737K 1s 2026-02-12T09:14:12.409 INFO:teuthology.orchestra.run.trial044.stderr: 50K .......... .......... .......... .......... .......... 16% 838K 1s 2026-02-12T09:14:12.415 INFO:teuthology.orchestra.run.trial044.stderr: 100K .......... .......... .......... .......... .......... 24% 9.57M 0s 2026-02-12T09:14:12.473 INFO:teuthology.orchestra.run.trial044.stderr: 150K .......... .......... .......... .......... .......... 32% 855K 0s 2026-02-12T09:14:12.482 INFO:teuthology.orchestra.run.trial044.stderr: 200K .......... .......... .......... .......... .......... 41% 7.69M 0s 2026-02-12T09:14:12.482 INFO:teuthology.orchestra.run.trial044.stderr: 250K .......... .......... .......... .......... .......... 49% 259M 0s 2026-02-12T09:14:12.482 INFO:teuthology.orchestra.run.trial044.stderr: 300K .......... .......... .......... .......... .......... 57% 139M 0s 2026-02-12T09:14:12.482 INFO:teuthology.orchestra.run.trial044.stderr: 350K .......... .......... .......... .......... .......... 65% 204M 0s 2026-02-12T09:14:12.542 INFO:teuthology.orchestra.run.trial044.stderr: 400K .......... .......... .......... .......... .......... 74% 818K 0s 2026-02-12T09:14:12.542 INFO:teuthology.orchestra.run.trial044.stderr: 450K .......... .......... .......... .......... .......... 82% 243M 0s 2026-02-12T09:14:12.546 INFO:teuthology.orchestra.run.trial044.stderr: 500K .......... .......... .......... .......... .......... 90% 16.7M 0s 2026-02-12T09:14:12.546 INFO:teuthology.orchestra.run.trial044.stderr: 550K .......... .......... .......... .......... .......... 98% 258M 0s 2026-02-12T09:14:12.546 INFO:teuthology.orchestra.run.trial044.stderr: 600K ...... 100% 11.4T=0.3s 2026-02-12T09:14:12.546 INFO:teuthology.orchestra.run.trial044.stderr: 2026-02-12T09:14:12.546 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12 09:14:12 (2.25 MB/s) - ‘collectl.tar.gz’ saved [620693/620693] 2026-02-12T09:14:12.546 INFO:teuthology.orchestra.run.trial044.stderr: 2026-02-12T09:14:12.567 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/ 2026-02-12T09:14:12.567 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/collectl.conf 2026-02-12T09:14:12.567 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/ 2026-02-12T09:14:12.567 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/SlowProc.html 2026-02-12T09:14:12.567 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/BuddyInfo.html 2026-02-12T09:14:12.567 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/WhySummary.html 2026-02-12T09:14:12.567 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Socket.html 2026-02-12T09:14:12.568 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/colmux.html 2026-02-12T09:14:12.568 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Tutorial.html 2026-02-12T09:14:12.568 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Gexpr.html 2026-02-12T09:14:12.568 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Import.html 2026-02-12T09:14:12.568 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/style.css 2026-02-12T09:14:12.568 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Ganglia3.jpg 2026-02-12T09:14:12.568 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Ganglia2.jpg 2026-02-12T09:14:12.568 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/FileNaming.html 2026-02-12T09:14:12.568 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/TheMath.html 2026-02-12T09:14:12.568 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Environmental.html 2026-02-12T09:14:12.568 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Process.html 2026-02-12T09:14:12.568 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/OutputFiles.html 2026-02-12T09:14:12.568 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/HiResTime.html 2026-02-12T09:14:12.569 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/OutputFormats.html 2026-02-12T09:14:12.569 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/slub.jpg 2026-02-12T09:14:12.569 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Lustre.html 2026-02-12T09:14:12.569 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Documentation.html 2026-02-12T09:14:12.569 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Inodes.html 2026-02-12T09:14:12.569 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Graphite.html 2026-02-12T09:14:12.569 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/InputFiles.html 2026-02-12T09:14:12.569 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Export.html 2026-02-12T09:14:12.569 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Interrupts.html 2026-02-12T09:14:12.569 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Startup.html 2026-02-12T09:14:12.569 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Performance.html 2026-02-12T09:14:12.569 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Messages.html 2026-02-12T09:14:12.569 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Logging.html 2026-02-12T09:14:12.570 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Data-detail.html 2026-02-12T09:14:12.570 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/CPUs.html 2026-02-12T09:14:12.570 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Matrix.html 2026-02-12T09:14:12.570 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Data-brief.html 2026-02-12T09:14:12.570 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/SlabInfo.html 2026-02-12T09:14:12.570 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Misc.html 2026-02-12T09:14:12.570 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Examples.html 2026-02-12T09:14:12.570 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/OpenStack.html 2026-02-12T09:14:12.570 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Plotfiles.html 2026-02-12T09:14:12.570 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/ColmuxCPU.jpg 2026-02-12T09:14:12.570 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Tutorial-Lustre.html 2026-02-12T09:14:12.570 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Data-verbose.html 2026-02-12T09:14:12.570 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Data.html 2026-02-12T09:14:12.571 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Exceptions.html 2026-02-12T09:14:12.571 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/RunningAsAService.html 2026-02-12T09:14:12.571 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/FAQ-collectl.html 2026-02-12T09:14:12.571 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Ganglia.jpg 2026-02-12T09:14:12.571 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/OperationalModes.html 2026-02-12T09:14:12.571 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Playback.html 2026-02-12T09:14:12.571 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Hello.html 2026-02-12T09:14:12.571 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Sockets.html 2026-02-12T09:14:12.571 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Network.html 2026-02-12T09:14:12.571 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Infiniband.html 2026-02-12T09:14:12.571 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Memory.html 2026-02-12T09:14:12.571 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/ColmuxLustre.jpg 2026-02-12T09:14:12.571 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/Disks.html 2026-02-12T09:14:12.571 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/docs/NfsInfo.html 2026-02-12T09:14:12.572 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/service/ 2026-02-12T09:14:12.572 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/service/collectl.service 2026-02-12T09:14:12.572 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/initd/ 2026-02-12T09:14:12.572 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/initd/collectl-suse 2026-02-12T09:14:12.572 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/initd/collectl-debian 2026-02-12T09:14:12.572 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/initd/collectl-generic 2026-02-12T09:14:12.572 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/initd/collectl 2026-02-12T09:14:12.572 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/misc.ph 2026-02-12T09:14:12.572 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/gexpr.ph 2026-02-12T09:14:12.572 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/statsd.ph 2026-02-12T09:14:12.572 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/envrules.std 2026-02-12T09:14:12.572 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/hello.ph 2026-02-12T09:14:12.572 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/vmsum.ph 2026-02-12T09:14:12.573 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/proctree.ph 2026-02-12T09:14:12.573 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/README-WINDOWS 2026-02-12T09:14:12.573 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/formatit.ph 2026-02-12T09:14:12.573 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/colmux 2026-02-12T09:14:12.573 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/INSTALL 2026-02-12T09:14:12.573 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/graphite.ph 2026-02-12T09:14:12.573 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/RELEASE-collectl 2026-02-12T09:14:12.573 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/vmstat.ph 2026-02-12T09:14:12.573 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/README 2026-02-12T09:14:12.573 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/man1/ 2026-02-12T09:14:12.573 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/man1/collectl.1 2026-02-12T09:14:12.573 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/man1/colmux.1 2026-02-12T09:14:12.573 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/lexpr.ph 2026-02-12T09:14:12.573 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/client.pl 2026-02-12T09:14:12.574 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/collectl 2026-02-12T09:14:12.574 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/COPYING 2026-02-12T09:14:12.574 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/vnet.ph 2026-02-12T09:14:12.574 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/GPL 2026-02-12T09:14:12.574 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/UNINSTALL 2026-02-12T09:14:12.574 INFO:teuthology.orchestra.run.trial044.stdout:collectl-4.3.1/ARTISTIC 2026-02-12T09:14:12.609 INFO:teuthology.orchestra.run.trial044.stderr:./INSTALL: line 90: chkconfig: command not found 2026-02-12T09:14:12.611 DEBUG:teuthology.orchestra.run.trial044:> sudo yum -y install librbd-devel pdsh pdsh-rcmd-ssh perf 2026-02-12T09:14:12.955 INFO:teuthology.orchestra.run.trial044.stdout:Last metadata expiration check: 0:01:04 ago on Thu 12 Feb 2026 09:13:08 AM UTC. 2026-02-12T09:14:13.039 INFO:teuthology.orchestra.run.trial044.stdout:Dependencies resolved. 2026-02-12T09:14:13.040 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:14:13.040 INFO:teuthology.orchestra.run.trial044.stdout: Package Arch Version Repository Size 2026-02-12T09:14:13.041 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:14:13.041 INFO:teuthology.orchestra.run.trial044.stdout:Installing: 2026-02-12T09:14:13.041 INFO:teuthology.orchestra.run.trial044.stdout: librbd-devel x86_64 2:20.3.0-5161.g2401a109.el9 ceph 24 k 2026-02-12T09:14:13.041 INFO:teuthology.orchestra.run.trial044.stdout: pdsh x86_64 2.34-7.el9 epel 118 k 2026-02-12T09:14:13.041 INFO:teuthology.orchestra.run.trial044.stdout: pdsh-rcmd-ssh x86_64 2.34-7.el9 epel 13 k 2026-02-12T09:14:13.041 INFO:teuthology.orchestra.run.trial044.stdout: perf x86_64 5.14.0-677.el9 appstream 3.2 M 2026-02-12T09:14:13.041 INFO:teuthology.orchestra.run.trial044.stdout:Installing dependencies: 2026-02-12T09:14:13.041 INFO:teuthology.orchestra.run.trial044.stdout: bzip2 x86_64 1.0.8-10.el9 baseos 56 k 2026-02-12T09:14:13.041 INFO:teuthology.orchestra.run.trial044.stdout: libradospp-devel x86_64 2:20.3.0-5161.g2401a109.el9 ceph 31 k 2026-02-12T09:14:13.041 INFO:teuthology.orchestra.run.trial044.stdout: libtraceevent x86_64 1.8.4-2.el9 baseos 290 k 2026-02-12T09:14:13.041 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:14:13.041 INFO:teuthology.orchestra.run.trial044.stdout:Transaction Summary 2026-02-12T09:14:13.042 INFO:teuthology.orchestra.run.trial044.stdout:================================================================================ 2026-02-12T09:14:13.042 INFO:teuthology.orchestra.run.trial044.stdout:Install 7 Packages 2026-02-12T09:14:13.042 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:14:13.042 INFO:teuthology.orchestra.run.trial044.stdout:Total download size: 3.7 M 2026-02-12T09:14:13.042 INFO:teuthology.orchestra.run.trial044.stdout:Installed size: 14 M 2026-02-12T09:14:13.042 INFO:teuthology.orchestra.run.trial044.stdout:Downloading Packages: 2026-02-12T09:14:13.755 INFO:teuthology.orchestra.run.trial044.stdout:(1/7): librbd-devel-20.3.0-5161.g2401a109.el9.x 82 kB/s | 24 kB 00:00 2026-02-12T09:14:13.763 INFO:teuthology.orchestra.run.trial044.stdout:(2/7): libradospp-devel-20.3.0-5161.g2401a109.e 105 kB/s | 31 kB 00:00 2026-02-12T09:14:14.072 INFO:teuthology.orchestra.run.trial044.stdout:(3/7): bzip2-1.0.8-10.el9.x86_64.rpm 92 kB/s | 56 kB 00:00 2026-02-12T09:14:14.350 INFO:teuthology.orchestra.run.trial044.stdout:(4/7): pdsh-2.34-7.el9.x86_64.rpm 427 kB/s | 118 kB 00:00 2026-02-12T09:14:14.397 INFO:teuthology.orchestra.run.trial044.stdout:(5/7): pdsh-rcmd-ssh-2.34-7.el9.x86_64.rpm 283 kB/s | 13 kB 00:00 2026-02-12T09:14:14.563 INFO:teuthology.orchestra.run.trial044.stdout:(6/7): libtraceevent-1.8.4-2.el9.x86_64.rpm 359 kB/s | 290 kB 00:00 2026-02-12T09:14:14.826 INFO:teuthology.orchestra.run.trial044.stdout:(7/7): perf-5.14.0-677.el9.x86_64.rpm 3.0 MB/s | 3.2 MB 00:01 2026-02-12T09:14:14.827 INFO:teuthology.orchestra.run.trial044.stdout:-------------------------------------------------------------------------------- 2026-02-12T09:14:14.827 INFO:teuthology.orchestra.run.trial044.stdout:Total 2.1 MB/s | 3.7 MB 00:01 2026-02-12T09:14:14.860 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction check 2026-02-12T09:14:14.870 INFO:teuthology.orchestra.run.trial044.stdout:Transaction check succeeded. 2026-02-12T09:14:14.870 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction test 2026-02-12T09:14:14.928 INFO:teuthology.orchestra.run.trial044.stdout:Transaction test succeeded. 2026-02-12T09:14:14.928 INFO:teuthology.orchestra.run.trial044.stdout:Running transaction 2026-02-12T09:14:15.003 INFO:teuthology.orchestra.run.trial044.stdout: Preparing : 1/1 2026-02-12T09:14:15.008 INFO:teuthology.orchestra.run.trial044.stdout: Installing : pdsh-rcmd-ssh-2.34-7.el9.x86_64 1/7 2026-02-12T09:14:15.048 INFO:teuthology.orchestra.run.trial044.stdout: Installing : pdsh-2.34-7.el9.x86_64 2/7 2026-02-12T09:14:15.053 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libtraceevent-1.8.4-2.el9.x86_64 3/7 2026-02-12T09:14:15.057 INFO:teuthology.orchestra.run.trial044.stdout: Installing : bzip2-1.0.8-10.el9.x86_64 4/7 2026-02-12T09:14:15.059 INFO:teuthology.orchestra.run.trial044.stdout: Installing : libradospp-devel-2:20.3.0-5161.g2401a109.el9.x86_64 5/7 2026-02-12T09:14:15.222 INFO:teuthology.orchestra.run.trial044.stdout: Installing : librbd-devel-2:20.3.0-5161.g2401a109.el9.x86_64 6/7 2026-02-12T09:14:15.243 INFO:teuthology.orchestra.run.trial044.stdout: Installing : perf-5.14.0-677.el9.x86_64 7/7 2026-02-12T09:14:15.357 INFO:teuthology.orchestra.run.trial044.stdout: Running scriptlet: perf-5.14.0-677.el9.x86_64 7/7 2026-02-12T09:14:15.357 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libradospp-devel-2:20.3.0-5161.g2401a109.el9.x86_64 1/7 2026-02-12T09:14:15.357 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : librbd-devel-2:20.3.0-5161.g2401a109.el9.x86_64 2/7 2026-02-12T09:14:15.357 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : bzip2-1.0.8-10.el9.x86_64 3/7 2026-02-12T09:14:15.357 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : libtraceevent-1.8.4-2.el9.x86_64 4/7 2026-02-12T09:14:15.357 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : perf-5.14.0-677.el9.x86_64 5/7 2026-02-12T09:14:15.357 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : pdsh-2.34-7.el9.x86_64 6/7 2026-02-12T09:14:15.418 INFO:teuthology.orchestra.run.trial044.stdout: Verifying : pdsh-rcmd-ssh-2.34-7.el9.x86_64 7/7 2026-02-12T09:14:15.418 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:14:15.418 INFO:teuthology.orchestra.run.trial044.stdout:Installed: 2026-02-12T09:14:15.418 INFO:teuthology.orchestra.run.trial044.stdout: bzip2-1.0.8-10.el9.x86_64 2026-02-12T09:14:15.418 INFO:teuthology.orchestra.run.trial044.stdout: libradospp-devel-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:14:15.418 INFO:teuthology.orchestra.run.trial044.stdout: librbd-devel-2:20.3.0-5161.g2401a109.el9.x86_64 2026-02-12T09:14:15.418 INFO:teuthology.orchestra.run.trial044.stdout: libtraceevent-1.8.4-2.el9.x86_64 2026-02-12T09:14:15.418 INFO:teuthology.orchestra.run.trial044.stdout: pdsh-2.34-7.el9.x86_64 2026-02-12T09:14:15.419 INFO:teuthology.orchestra.run.trial044.stdout: pdsh-rcmd-ssh-2.34-7.el9.x86_64 2026-02-12T09:14:15.419 INFO:teuthology.orchestra.run.trial044.stdout: perf-5.14.0-677.el9.x86_64 2026-02-12T09:14:15.419 INFO:teuthology.orchestra.run.trial044.stdout: 2026-02-12T09:14:15.419 INFO:teuthology.orchestra.run.trial044.stdout:Complete! 2026-02-12T09:14:15.449 DEBUG:teuthology.orchestra.run.trial044:> python3 -m pip install -r /home/ubuntu/cephtest/cbt/requirements.txt 2026-02-12T09:14:15.586 INFO:teuthology.orchestra.run.trial044.stdout:Defaulting to user installation because normal site-packages is not writeable 2026-02-12T09:14:15.597 INFO:teuthology.orchestra.run.trial044.stdout:Requirement already satisfied: pyyaml in /usr/lib64/python3.9/site-packages (from -r /home/ubuntu/cephtest/cbt/requirements.txt (line 1)) (5.4.1) 2026-02-12T09:14:15.597 INFO:teuthology.orchestra.run.trial044.stdout:Requirement already satisfied: lxml in /usr/lib64/python3.9/site-packages (from -r /home/ubuntu/cephtest/cbt/requirements.txt (line 2)) (4.6.5) 2026-02-12T09:14:15.928 INFO:teuthology.orchestra.run.trial044.stdout:Collecting matplotlib 2026-02-12T09:14:15.958 INFO:teuthology.orchestra.run.trial044.stdout: Downloading matplotlib-3.9.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.3 MB) 2026-02-12T09:14:16.015 INFO:teuthology.orchestra.run.trial044.stdout:Requirement already satisfied: packaging>=20.0 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (20.9) 2026-02-12T09:14:16.049 INFO:teuthology.orchestra.run.trial044.stdout:Collecting importlib-resources>=3.2.0 2026-02-12T09:14:16.052 INFO:teuthology.orchestra.run.trial044.stdout: Downloading importlib_resources-6.5.2-py3-none-any.whl (37 kB) 2026-02-12T09:14:16.065 INFO:teuthology.orchestra.run.trial044.stdout:Collecting cycler>=0.10 2026-02-12T09:14:16.069 INFO:teuthology.orchestra.run.trial044.stdout: Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB) 2026-02-12T09:14:16.449 INFO:teuthology.orchestra.run.trial044.stdout:Collecting pillow>=8 2026-02-12T09:14:16.455 INFO:teuthology.orchestra.run.trial044.stdout: Downloading pillow-11.3.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (6.6 MB) 2026-02-12T09:14:16.489 INFO:teuthology.orchestra.run.trial044.stdout:Requirement already satisfied: pyparsing>=2.3.1 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (2.4.7) 2026-02-12T09:14:16.626 INFO:teuthology.orchestra.run.trial044.stdout:Collecting contourpy>=1.0.1 2026-02-12T09:14:16.631 INFO:teuthology.orchestra.run.trial044.stdout: Downloading contourpy-1.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (321 kB) 2026-02-12T09:14:16.952 INFO:teuthology.orchestra.run.trial044.stdout:Collecting fonttools>=4.22.0 2026-02-12T09:14:16.958 INFO:teuthology.orchestra.run.trial044.stdout: Downloading fonttools-4.60.2-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.8 MB) 2026-02-12T09:14:16.982 INFO:teuthology.orchestra.run.trial044.stdout:Requirement already satisfied: python-dateutil>=2.7 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (2.9.0.post0) 2026-02-12T09:14:16.982 INFO:teuthology.orchestra.run.trial044.stdout:Requirement already satisfied: numpy>=1.23 in /usr/lib64/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (1.23.5) 2026-02-12T09:14:17.131 INFO:teuthology.orchestra.run.trial044.stdout:Collecting kiwisolver>=1.3.1 2026-02-12T09:14:17.138 INFO:teuthology.orchestra.run.trial044.stdout: Downloading kiwisolver-1.4.7-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB) 2026-02-12T09:14:17.195 INFO:teuthology.orchestra.run.trial044.stdout:Collecting zipp>=3.1.0 2026-02-12T09:14:17.198 INFO:teuthology.orchestra.run.trial044.stdout: Downloading zipp-3.23.0-py3-none-any.whl (10 kB) 2026-02-12T09:14:17.209 INFO:teuthology.orchestra.run.trial044.stdout:Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (1.15.0) 2026-02-12T09:14:17.344 INFO:teuthology.orchestra.run.trial044.stdout:Installing collected packages: zipp, pillow, kiwisolver, importlib-resources, fonttools, cycler, contourpy, matplotlib 2026-02-12T09:14:18.386 INFO:teuthology.orchestra.run.trial044.stdout:Successfully installed contourpy-1.3.0 cycler-0.12.1 fonttools-4.60.2 importlib-resources-6.5.2 kiwisolver-1.4.7 matplotlib-3.9.4 pillow-11.3.0 zipp-3.23.0 2026-02-12T09:14:18.497 INFO:tasks.cbt:benchmark: radosbench 2026-02-12T09:14:18.497 DEBUG:teuthology.orchestra.run.trial044:> sudo /sbin/sysctl -q -w kernel.perf_event_paranoid=0 2026-02-12T09:14:18.522 DEBUG:teuthology.orchestra.run.trial044:> /home/ubuntu/cephtest/cbt/cbt.py -a /home/ubuntu/cephtest/archive/cbt /home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2026-02-12T09:14:18.634 INFO:teuthology.orchestra.run.trial044.stderr:09:14:18 - DEBUG - cbt - Settings.cluster: 2026-02-12T09:14:18.634 INFO:teuthology.orchestra.run.trial044.stderr: {'archive_dir': '/home/ubuntu/cephtest/archive/cbt', 2026-02-12T09:14:18.634 INFO:teuthology.orchestra.run.trial044.stderr: 'clients': ['ubuntu@trial044.front.sepia.ceph.com'], 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr: 'conf_file': '/etc/ceph/ceph.conf', 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr: 'head': 'ubuntu@trial044.front.sepia.ceph.com', 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr: 'is_teuthology': True, 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr: 'iterations': 1, 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr: 'mons': ['ubuntu@trial044.front.sepia.ceph.com'], 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr: 'osds': ['ubuntu@trial044.front.sepia.ceph.com'], 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr: 'osds_per_node': 3, 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr: 'pid_dir': '/var/run/ceph', 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr: 'pool_profiles': {'replicated': {'pg_size': 256, 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr: 'pgp_size': 256, 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr: 'replication': 'replicated'}}, 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr: 'rebuild_every_test': False, 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr: 'rgws': {'ubuntu@trial044.front.sepia.ceph.com': None}, 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr: 'tmp_dir': '/tmp/cbt', 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr: 'use_existing': True, 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr: 'user': 'ubuntu'} 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr:09:14:18 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-8b8185cb 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr:09:14:18 - INFO - cbt - Pausing for 60s for idle monitoring. 2026-02-12T09:14:18.635 INFO:teuthology.orchestra.run.trial044.stderr:09:14:18 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/collectl 2026-02-12T09:14:18.804 INFO:teuthology.orchestra.run.trial044.stderr:09:14:18 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-12T09:14:18.806 INFO:teuthology.orchestra.run.trial044.stderr:09:14:18 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/perf 2026-02-12T09:14:18.970 INFO:teuthology.orchestra.run.trial044.stderr:09:14:18 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-12T09:14:18.970 INFO:teuthology.orchestra.run.trial044.stderr:09:14:18 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/perf/perf_stat.${pid} & done 2026-02-12T09:15:19.030 INFO:teuthology.orchestra.run.trial044.stderr:09:15:19 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-12T09:15:19.221 INFO:teuthology.orchestra.run.trial044.stderr:09:15:19 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-12T09:15:19.403 INFO:teuthology.orchestra.run.trial044.stderr:09:15:19 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/perf/perf.data 2026-02-12T09:15:19.404 INFO:teuthology.orchestra.run.trial044.stderr:09:15:19 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/perf/perf_stat.* 2026-02-12T09:15:19.406 INFO:teuthology.orchestra.run.trial044.stderr:09:15:19 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/* 2026-02-12T09:15:19.590 INFO:teuthology.orchestra.run.trial044.stderr:09:15:19 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial044.front.sepia.ceph.com -r /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-8b8185cb 2026-02-12T09:15:19.751 INFO:teuthology.orchestra.run.trial044.stderr:09:15:19 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo killall -9 rados 2026-02-12T09:15:19.930 INFO:teuthology.orchestra.run.trial044.stderr:09:15:19 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-8b8185cb 2026-02-12T09:15:19.930 INFO:teuthology.orchestra.run.trial044.stderr:09:15:19 - INFO - cbt - Skipping existing test in /home/ubuntu/cephtest/archive/cbt/results/00000000/id-8b8185cb. 2026-02-12T09:15:19.930 INFO:teuthology.orchestra.run.trial044.stderr:09:15:19 - INFO - cbt - Running benchmark /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004 2026-02-12T09:15:19.930 INFO:teuthology.orchestra.run.trial044.stderr:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-8b8185cb 2026-02-12T09:15:19.931 INFO:teuthology.orchestra.run.trial044.stderr:{'concurrent_ops': 4, 'concurrent_procs': 2, 'pool_profile': 'replicated', 'time': 60, 'write_only': True, 'op_size': 4096, 'pool_monitoring_list': 'collectl', 'run_monitoring_list': 'collectl', 'benchmark': 'radosbench', 'iteration': 0} == iteration 0 == 2026-02-12T09:15:19.931 INFO:teuthology.orchestra.run.trial044.stderr:09:15:19 - INFO - cbt - Setting OSD Read Ahead to: 0 2026-02-12T09:15:19.931 INFO:teuthology.orchestra.run.trial044.stderr:09:15:19 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com find /dev/disk/by-partlabel/osd-device-*data -exec readlink {} \; | cut -d"/" -f 3 | sed "s/[0-9]$//" | xargs -I{} sudo sh -c "echo 0 > /sys/block/'{}'/queue/read_ahead_kb" 2026-02-12T09:15:19.931 INFO:teuthology.orchestra.run.trial044.stderr:09:15:19 - DEBUG - cbt - Cleaning existing temporary run directory: /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004 2026-02-12T09:15:19.932 INFO:teuthology.orchestra.run.trial044.stderr:09:15:19 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo rm -rf /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004 2026-02-12T09:15:20.108 INFO:teuthology.orchestra.run.trial044.stderr:09:15:20 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/collectl 2026-02-12T09:15:20.275 INFO:teuthology.orchestra.run.trial044.stderr:09:15:20 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-12T09:15:20.277 INFO:teuthology.orchestra.run.trial044.stderr:09:15:20 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/perf 2026-02-12T09:15:20.440 INFO:teuthology.orchestra.run.trial044.stderr:09:15:20 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-12T09:15:20.440 INFO:teuthology.orchestra.run.trial044.stderr:09:15:20 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/perf/perf_stat.${pid} & done 2026-02-12T09:15:20.442 INFO:teuthology.orchestra.run.trial044.stderr:09:15:20 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool delete rados-bench-cbt rados-bench-cbt --yes-i-really-really-mean-it 2026-02-12T09:15:20.784 INFO:teuthology.orchestra.run.trial044.stderr:09:15:20 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool create rados-bench-cbt 256 256 2026-02-12T09:15:21.309 INFO:teuthology.orchestra.run.trial044.stderr:09:15:21 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool application enable rados-bench-cbt radosbench 2026-02-12T09:15:22.313 INFO:teuthology.orchestra.run.trial044.stderr:09:15:22 - INFO - cbt - Checking Health after pool creation. 2026-02-12T09:15:27.313 INFO:teuthology.orchestra.run.trial044.stderr:09:15:27 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com /usr/bin/ceph -c /etc/ceph/ceph.conf health 2026-02-12T09:15:27.672 INFO:teuthology.orchestra.run.trial044.stderr:09:15:27 - INFO - cbt - Final Pool Health Check. 2026-02-12T09:15:32.678 INFO:teuthology.orchestra.run.trial044.stderr:09:15:32 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com /usr/bin/ceph -c /etc/ceph/ceph.conf health 2026-02-12T09:15:33.047 INFO:teuthology.orchestra.run.trial044.stderr:09:15:33 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-12T09:15:33.221 INFO:teuthology.orchestra.run.trial044.stderr:09:15:33 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-12T09:15:33.406 INFO:teuthology.orchestra.run.trial044.stderr:09:15:33 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/perf/perf.data 2026-02-12T09:15:33.407 INFO:teuthology.orchestra.run.trial044.stderr:09:15:33 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/perf/perf_stat.* 2026-02-12T09:15:33.409 INFO:teuthology.orchestra.run.trial044.stderr:09:15:33 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sync 2026-02-12T09:15:33.578 INFO:teuthology.orchestra.run.trial044.stderr:09:15:33 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com echo 3 | sudo tee /proc/sys/vm/drop_caches 2026-02-12T09:15:34.575 INFO:teuthology.orchestra.run.trial044.stderr:09:15:34 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -v 2026-02-12T09:15:34.783 INFO:teuthology.orchestra.run.trial044.stderr:09:15:34 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write 2026-02-12T09:15:34.945 INFO:teuthology.orchestra.run.trial044.stderr:09:15:34 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf daemon osd.0 config show > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/ceph_settings.out 2026-02-12T09:15:35.231 INFO:teuthology.orchestra.run.trial044.stderr:09:15:35 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/collectl 2026-02-12T09:15:35.400 INFO:teuthology.orchestra.run.trial044.stderr:09:15:35 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-12T09:15:35.402 INFO:teuthology.orchestra.run.trial044.stderr:09:15:35 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/perf 2026-02-12T09:15:35.571 INFO:teuthology.orchestra.run.trial044.stderr:09:15:35 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-12T09:15:35.572 INFO:teuthology.orchestra.run.trial044.stderr:09:15:35 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/perf/perf_stat.${pid} & done 2026-02-12T09:15:35.572 INFO:teuthology.orchestra.run.trial044.stderr:09:15:35 - INFO - cbt - Running radosbench write test. 2026-02-12T09:15:35.573 INFO:teuthology.orchestra.run.trial044.stderr:09:15:35 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench -b 4096 60 write --concurrent-ios 4 --run-name `hostname -f`-0 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/objecter.0.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/output.0 2026-02-12T09:15:35.574 INFO:teuthology.orchestra.run.trial044.stderr:09:15:35 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench -b 4096 60 write --concurrent-ios 4 --run-name `hostname -f`-1 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/objecter.1.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/output.1 2026-02-12T09:16:36.796 INFO:teuthology.orchestra.run.trial044.stderr:09:16:36 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-12T09:16:36.975 INFO:teuthology.orchestra.run.trial044.stderr:09:16:36 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-12T09:16:37.155 INFO:teuthology.orchestra.run.trial044.stderr:09:16:37 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/perf/perf.data 2026-02-12T09:16:37.157 INFO:teuthology.orchestra.run.trial044.stderr:09:16:37 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/perf/perf_stat.* 2026-02-12T09:16:37.159 INFO:teuthology.orchestra.run.trial044.stderr:09:16:37 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com find /var/run/ceph/ceph-osd*.asok -maxdepth 1 -exec sudo /usr/bin/ceph --admin-daemon {} dump_historic_ops \; > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/historic_ops.out 2026-02-12T09:16:37.521 INFO:teuthology.orchestra.run.trial044.stderr:09:16:37 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial044.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/* 2026-02-12T09:16:37.697 INFO:teuthology.orchestra.run.trial044.stderr:09:16:37 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial044.front.sepia.ceph.com -r /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-8b8185cb/write 2026-02-12T09:16:37.866 INFO:teuthology.orchestra.run.trial044.stderr:09:16:37 - INFO - cbt - Convert results to json format. 2026-02-12T09:16:37.877 DEBUG:teuthology.run_tasks:Unwinding manager cbt 2026-02-12T09:16:37.891 DEBUG:teuthology.orchestra.run.trial044:> rm --one-file-system -rf -- /home/ubuntu/cephtest/cbt 2026-02-12T09:16:37.919 INFO:tasks.cbt_performance:Collecting CBT performance data config 2026-02-12T09:16:37.919 INFO:tasks.cbt_performance:reading cbt preferences 2026-02-12T09:16:37.919 DEBUG:teuthology.orchestra.run.trial044:> find /home/ubuntu/cephtest -name 'json_output.*' 2026-02-12T09:16:37.977 INFO:teuthology.orchestra.run.trial044.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-8b8185cb/write/json_output.1.trial044.front.sepia.ceph.com 2026-02-12T09:16:37.977 INFO:teuthology.orchestra.run.trial044.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-8b8185cb/write/json_output.0.trial044.front.sepia.ceph.com 2026-02-12T09:16:37.978 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:16:37.978 DEBUG:teuthology.orchestra.run.trial044:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-8b8185cb/write/json_output.1.trial044.front.sepia.ceph.com of=/dev/stdout 2026-02-12T09:16:38.036 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-12T09:16:38.036 DEBUG:teuthology.orchestra.run.trial044:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-8b8185cb/write -name 'perf_stat.*' 2026-02-12T09:16:38.092 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-12T09:16:38.092 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-12T09:16:38.092 DEBUG:teuthology.orchestra.run.trial044:> set -ex 2026-02-12T09:16:38.092 DEBUG:teuthology.orchestra.run.trial044:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-8b8185cb/write/json_output.0.trial044.front.sepia.ceph.com of=/dev/stdout 2026-02-12T09:16:38.148 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-12T09:16:38.149 DEBUG:teuthology.orchestra.run.trial044:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-8b8185cb/write -name 'perf_stat.*' 2026-02-12T09:16:38.204 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-12T09:16:38.204 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-12T09:16:38.209 WARNING:tasks.cbt_performance:Could not send performance data to http://mira118.front.sepia.ceph.com:4000/cbt_performance: HTTPConnectionPool(host='mira118.front.sepia.ceph.com', port=4000): Max retries exceeded with url: /cbt_performance (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known')) 2026-02-12T09:16:38.209 WARNING:tasks.cbt_performance:Continuing without performance data collection 2026-02-12T09:16:38.209 DEBUG:teuthology.run_tasks:Unwinding manager ssh_keys 2026-02-12T09:16:38.225 INFO:teuthology.task.ssh_keys:Cleaning up SSH keys 2026-02-12T09:16:38.225 INFO:teuthology.task.ssh_keys:cleaning up keys added for testing 2026-02-12T09:16:38.225 INFO:teuthology.task.ssh_keys: cleaning up keys for user ubuntu on trial044.front.sepia.ceph.com 2026-02-12T09:16:38.225 DEBUG:teuthology.orchestra.run.trial044:> rm -- /home/ubuntu/.ssh/id_rsa 2026-02-12T09:16:38.262 DEBUG:teuthology.orchestra.run.trial044:> rm -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-12T09:16:38.317 DEBUG:teuthology.orchestra.run.trial044:> stat -c '"%a"' /home/ubuntu/.ssh/authorized_keys 2026-02-12T09:16:38.374 INFO:teuthology.orchestra.run.trial044.stdout:"600" 2026-02-12T09:16:38.375 DEBUG:teuthology.orchestra.run.trial044:> mv -- /home/ubuntu/.ssh/authorized_keys_2026-02-12_09:14:05:823172 /home/ubuntu/.ssh/authorized_keys 2026-02-12T09:16:38.430 DEBUG:teuthology.orchestra.run.trial044:> chmod 600 /home/ubuntu/.ssh/authorized_keys 2026-02-12T09:16:38.487 DEBUG:teuthology.orchestra.run.trial044:> rm /home/ubuntu/.ssh/config 2026-02-12T09:16:38.542 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-12T09:16:38.553 DEBUG:teuthology.orchestra.run.trial044:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-12T09:16:38.658 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:16:38.651+0000 7fb529964640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:16:38.660 INFO:teuthology.orchestra.run.trial044.stderr:2026-02-12T09:16:38.656+0000 7fb529964640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-12T09:16:38.783 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-12T09:16:38.784 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-12T09:16:38.784 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-12T09:16:38.784 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T09:16:38.865 INFO:tasks.ceph.osd.0:Stopped 2026-02-12T09:16:38.866 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-12T09:16:38.866 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T09:16:38.954 INFO:tasks.ceph.osd.1:Stopped 2026-02-12T09:16:38.954 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-12T09:16:38.954 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T09:16:39.039 INFO:tasks.ceph.osd.2:Stopped 2026-02-12T09:16:39.039 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-12T09:16:39.040 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-12T09:16:39.040 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T09:16:39.075 INFO:tasks.ceph.mgr.x:Stopped 2026-02-12T09:16:39.075 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-12T09:16:39.075 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-12T09:16:39.075 INFO:teuthology.orchestra.run:waiting for 300 2026-02-12T09:16:39.124 INFO:tasks.ceph.mon.a:Stopped 2026-02-12T09:16:39.124 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-12T09:16:39.124 DEBUG:teuthology.orchestra.run.trial044:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v '\(PG_' | grep -E -v '\(OSD_' | grep -E -v '\(OBJECT_' | grep -E -v 'overall HEALTH' | head -n 1 2026-02-12T09:16:39.191 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial044.front.sepia.ceph.com 2026-02-12T09:16:39.191 DEBUG:teuthology.orchestra.run.trial044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-12T09:16:39.506 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial044.front.sepia.ceph.com 2026-02-12T09:16:39.506 DEBUG:teuthology.orchestra.run.trial044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-12T09:16:39.534 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial044.front.sepia.ceph.com 2026-02-12T09:16:39.534 DEBUG:teuthology.orchestra.run.trial044:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-12T09:16:39.608 INFO:tasks.ceph:Archiving mon data... 2026-02-12T09:16:39.609 DEBUG:teuthology.misc:Transferring archived files from trial044:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41608/data/mon.a.tgz 2026-02-12T09:16:39.609 DEBUG:teuthology.orchestra.run.trial044:> mktemp 2026-02-12T09:16:39.664 INFO:teuthology.orchestra.run.trial044.stdout:/tmp/tmp.8XTRr9iMlN 2026-02-12T09:16:39.664 DEBUG:teuthology.orchestra.run.trial044:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.8XTRr9iMlN 2026-02-12T09:16:39.753 DEBUG:teuthology.orchestra.run.trial044:> sudo chmod 0666 /tmp/tmp.8XTRr9iMlN 2026-02-12T09:16:39.836 DEBUG:teuthology.orchestra.remote:trial044:/tmp/tmp.8XTRr9iMlN is 197KB 2026-02-12T09:16:39.888 DEBUG:teuthology.orchestra.run.trial044:> rm -fr /tmp/tmp.8XTRr9iMlN 2026-02-12T09:16:39.903 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-12T09:16:39.904 DEBUG:teuthology.orchestra.run.trial044:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-12T09:16:40.024 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-12T09:16:40.024 INFO:tasks.ceph:Archiving crash dumps... 2026-02-12T09:16:40.027 DEBUG:teuthology.misc:Transferring archived files from trial044:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41608/remote/trial044/crash 2026-02-12T09:16:40.028 DEBUG:teuthology.orchestra.run.trial044:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-12T09:16:40.090 INFO:tasks.ceph:Compressing logs... 2026-02-12T09:16:40.090 DEBUG:teuthology.orchestra.run.trial044:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-12T09:16:40.155 INFO:teuthology.orchestra.run.trial044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-12T09:16:40.167 INFO:teuthology.orchestra.run.trial044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-12T09:16:40.167 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22470.log 2026-02-12T09:16:40.167 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23148.log 2026-02-12T09:16:40.167 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.22470.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-12T09:16:40.168 INFO:teuthology.orchestra.run.trial044.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22470.log.gz 2026-02-12T09:16:40.168 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.23148.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22560.log 2026-02-12T09:16:40.168 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22658.log 2026-02-12T09:16:40.168 INFO:teuthology.orchestra.run.trial044.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.23148.log.gz 2026-02-12T09:16:40.168 INFO:teuthology.orchestra.run.trial044.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-12T09:16:40.168 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.22560.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22707.log 2026-02-12T09:16:40.168 INFO:teuthology.orchestra.run.trial044.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22560.log.gz 2026-02-12T09:16:40.168 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.22658.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22400.log 2026-02-12T09:16:40.168 INFO:teuthology.orchestra.run.trial044.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22658.log.gz 2026-02-12T09:16:40.168 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.22707.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23246.log 2026-02-12T09:16:40.168 INFO:teuthology.orchestra.run.trial044.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22707.log.gz 2026-02-12T09:16:40.169 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.22400.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21889.log 2026-02-12T09:16:40.169 INFO:teuthology.orchestra.run.trial044.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.22400.log.gz 2026-02-12T09:16:40.169 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.23246.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26648.log 2026-02-12T09:16:40.169 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.21889.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.23246.log.gz 2026-02-12T09:16:40.169 INFO:teuthology.orchestra.run.trial044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22332.log 2026-02-12T09:16:40.169 INFO:teuthology.orchestra.run.trial044.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.21889.log.gz 2026-02-12T09:16:40.169 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.26648.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25943.log 2026-02-12T09:16:40.169 INFO:teuthology.orchestra.run.trial044.stderr: 66.4% -- replaced with /var/log/ceph/ceph-client.admin.26648.log.gz 2026-02-12T09:16:40.169 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.22332.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.25887.log 2026-02-12T09:16:40.169 INFO:teuthology.orchestra.run.trial044.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22332.log.gz 2026-02-12T09:16:40.169 INFO:teuthology.orchestra.run.trial044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23197.log 2026-02-12T09:16:40.169 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.25943.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.25943.log.gz 2026-02-12T09:16:40.169 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.25887.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26039.log 2026-02-12T09:16:40.170 INFO:teuthology.orchestra.run.trial044.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.25887.log.gz/var/log/ceph/ceph-client.admin.23197.log: 2026-02-12T09:16:40.170 INFO:teuthology.orchestra.run.trial044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21705.log 2026-02-12T09:16:40.170 INFO:teuthology.orchestra.run.trial044.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.23197.log.gz 2026-02-12T09:16:40.170 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.26039.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21842.log 2026-02-12T09:16:40.170 INFO:teuthology.orchestra.run.trial044.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.26039.log.gz 2026-02-12T09:16:40.170 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.21705.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.27275.log 2026-02-12T09:16:40.170 INFO:teuthology.orchestra.run.trial044.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.21705.log.gz 2026-02-12T09:16:40.170 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.21842.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22955.log 2026-02-12T09:16:40.170 INFO:teuthology.orchestra.run.trial044.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.21842.log.gz 2026-02-12T09:16:40.170 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.27275.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26647.log 2026-02-12T09:16:40.170 INFO:teuthology.orchestra.run.trial044.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.27275.log.gz 2026-02-12T09:16:40.170 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.22955.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21563.log 2026-02-12T09:16:40.170 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.26647.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22955.log.gz 2026-02-12T09:16:40.171 INFO:teuthology.orchestra.run.trial044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22495.log 2026-02-12T09:16:40.171 INFO:teuthology.orchestra.run.trial044.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.26647.log.gz 2026-02-12T09:16:40.171 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.21563.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22186.log 2026-02-12T09:16:40.171 INFO:teuthology.orchestra.run.trial044.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.21563.log.gz 2026-02-12T09:16:40.171 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.22495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22808.log 2026-02-12T09:16:40.171 INFO:teuthology.orchestra.run.trial044.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.22495.log.gz 2026-02-12T09:16:40.171 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.22186.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-12T09:16:40.171 INFO:teuthology.orchestra.run.trial044.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22186.log.gz 2026-02-12T09:16:40.171 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.22808.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-12T09:16:40.171 INFO:teuthology.orchestra.run.trial044.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22808.log.gz 2026-02-12T09:16:40.171 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22956.log 2026-02-12T09:16:40.171 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26093.log 2026-02-12T09:16:40.171 INFO:teuthology.orchestra.run.trial044.stderr: 87.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-12T09:16:40.172 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.22956.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22809.log 2026-02-12T09:16:40.172 INFO:teuthology.orchestra.run.trial044.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22956.log.gz 2026-02-12T09:16:40.172 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.26093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.26002.log 2026-02-12T09:16:40.172 INFO:teuthology.orchestra.run.trial044.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.26093.log.gz 2026-02-12T09:16:40.172 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.22809.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22609.log 2026-02-12T09:16:40.172 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.26002.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22957.log 2026-02-12T09:16:40.172 INFO:teuthology.orchestra.run.trial044.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22809.log.gz 2026-02-12T09:16:40.172 INFO:teuthology.orchestra.run.trial044.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.26002.log.gz 2026-02-12T09:16:40.172 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.22609.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21658.log 2026-02-12T09:16:40.172 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.22957.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22609.log.gz 2026-02-12T09:16:40.172 INFO:teuthology.orchestra.run.trial044.stderr:gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.20277.log 2026-02-12T09:16:40.172 INFO:teuthology.orchestra.run.trial044.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.22957.log.gz 2026-02-12T09:16:40.172 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.21658.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-12T09:16:40.173 INFO:teuthology.orchestra.run.trial044.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.21658.log.gz 2026-02-12T09:16:40.173 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph.tmp-client.admin.20277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.21936.log 2026-02-12T09:16:40.173 INFO:teuthology.orchestra.run.trial044.stderr: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.20277.log.gz 2026-02-12T09:16:40.173 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-12T09:16:40.173 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.21936.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22810.log 2026-02-12T09:16:40.173 INFO:teuthology.orchestra.run.trial044.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.21936.log.gz 2026-02-12T09:16:40.173 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.22155.log 2026-02-12T09:16:40.173 INFO:teuthology.orchestra.run.trial044.stderr: 85.1% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-12T09:16:40.173 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.22810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23050.log 2026-02-12T09:16:40.173 INFO:teuthology.orchestra.run.trial044.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.22810.log.gz 2026-02-12T09:16:40.173 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.22155.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.23099.log 2026-02-12T09:16:40.173 INFO:teuthology.orchestra.run.trial044.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.22155.log.gz 2026-02-12T09:16:40.174 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.23050.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.23050.log.gz 2026-02-12T09:16:40.174 INFO:teuthology.orchestra.run.trial044.stderr:/var/log/ceph/ceph-client.admin.23099.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.23099.log.gz 2026-02-12T09:16:40.175 INFO:teuthology.orchestra.run.trial044.stderr: 92.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-12T09:16:44.280 INFO:teuthology.orchestra.run.trial044.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-12T09:16:44.495 INFO:teuthology.orchestra.run.trial044.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-12T09:16:44.578 INFO:teuthology.orchestra.run.trial044.stderr: 91.9% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-12T09:16:44.579 INFO:teuthology.orchestra.run.trial044.stderr: 2026-02-12T09:16:44.579 INFO:teuthology.orchestra.run.trial044.stderr:real 0m4.433s 2026-02-12T09:16:44.579 INFO:teuthology.orchestra.run.trial044.stderr:user 0m12.255s 2026-02-12T09:16:44.580 INFO:teuthology.orchestra.run.trial044.stderr:sys 0m0.649s 2026-02-12T09:16:44.581 INFO:tasks.ceph:Archiving logs... 2026-02-12T09:16:44.581 DEBUG:teuthology.misc:Transferring archived files from trial044:/var/log/ceph to /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41608/remote/trial044/log 2026-02-12T09:16:44.583 DEBUG:teuthology.orchestra.run.trial044:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-12T09:16:45.767 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-12T09:16:45.837 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-12T09:16:45.848 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-12T09:16:45.848 DEBUG:teuthology.orchestra.run.trial044:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-12T09:16:45.871 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-12T09:16:45.882 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-12T09:16:45.882 DEBUG:teuthology.orchestra.run.trial044:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-12T09:16:45.928 INFO:teuthology.orchestra.run.trial044.stderr:bash: line 1: ntpq: command not found 2026-02-12T09:16:45.954 INFO:teuthology.orchestra.run.trial044.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-12T09:16:45.954 INFO:teuthology.orchestra.run.trial044.stdout:=============================================================================== 2026-02-12T09:16:45.954 INFO:teuthology.orchestra.run.trial044.stdout:^- 23.157.160.168 2 6 377 12 +2408us[+2408us] +/- 49ms 2026-02-12T09:16:45.954 INFO:teuthology.orchestra.run.trial044.stdout:^- ntp.maxhost.io 2 6 377 13 -4729us[-4761us] +/- 62ms 2026-02-12T09:16:45.955 INFO:teuthology.orchestra.run.trial044.stdout:^- ntp1.glypnod.com 2 6 377 13 -578us[ -611us] +/- 42ms 2026-02-12T09:16:45.955 INFO:teuthology.orchestra.run.trial044.stdout:^* pool-96-231-54-40.washdc> 1 6 377 12 -236us[ -269us] +/- 7265us 2026-02-12T09:16:45.956 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-12T09:16:45.970 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-12T09:16:45.970 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-12T09:16:45.985 DEBUG:teuthology.orchestra.run.trial044:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-12T09:16:46.044 DEBUG:teuthology.orchestra.run.trial044:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-12T09:16:46.069 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T09:16:46.069 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-12T09:16:46.086 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-12T09:16:46.102 INFO:teuthology.task.internal:Duration was 540.101332 seconds 2026-02-12T09:16:46.102 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-12T09:16:46.118 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-12T09:16:46.118 DEBUG:teuthology.orchestra.run.trial044:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-12T09:16:46.157 INFO:teuthology.orchestra.run.trial044.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-12T09:16:46.441 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-12T09:16:46.442 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial044.front.sepia.ceph.com 2026-02-12T09:16:46.442 DEBUG:teuthology.orchestra.run.trial044:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-12T09:16:46.500 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-12T09:16:46.501 DEBUG:teuthology.orchestra.run.trial044:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-12T09:16:46.585 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-12T09:16:46.585 DEBUG:teuthology.orchestra.run.trial044:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-12T09:16:46.608 INFO:teuthology.orchestra.run.trial044.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-12T09:16:46.609 INFO:teuthology.orchestra.run.trial044.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-12T09:16:46.609 INFO:teuthology.orchestra.run.trial044.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-12T09:16:46.610 INFO:teuthology.orchestra.run.trial044.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-12T09:16:46.610 INFO:teuthology.orchestra.run.trial044.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-12T09:16:46.613 INFO:teuthology.orchestra.run.trial044.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 88.1% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-12T09:16:46.615 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-12T09:16:46.632 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-12T09:16:46.633 DEBUG:teuthology.orchestra.run.trial044:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-12T09:16:46.677 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-12T09:16:46.695 DEBUG:teuthology.orchestra.run.trial044:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-12T09:16:46.741 INFO:teuthology.orchestra.run.trial044.stdout:kernel.core_pattern = core 2026-02-12T09:16:46.755 DEBUG:teuthology.orchestra.run.trial044:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-12T09:16:46.810 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-12T09:16:46.811 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-12T09:16:46.829 INFO:teuthology.task.internal:Transferring archived files... 2026-02-12T09:16:46.830 DEBUG:teuthology.misc:Transferring archived files from trial044:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/matan-2026-02-09_15:50:17-crimson-rados-main-distro-debug-trial/41608/remote/trial044 2026-02-12T09:16:46.830 DEBUG:teuthology.orchestra.run.trial044:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-12T09:16:46.906 INFO:teuthology.task.internal:Removing archive directory... 2026-02-12T09:16:46.906 DEBUG:teuthology.orchestra.run.trial044:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-12T09:16:46.936 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-12T09:16:46.957 INFO:teuthology.task.internal:Not uploading archives. 2026-02-12T09:16:46.957 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-12T09:16:46.973 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-12T09:16:46.974 DEBUG:teuthology.orchestra.run.trial044:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-12T09:16:46.992 INFO:teuthology.orchestra.run.trial044.stdout: 786525 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 12 09:16 /home/ubuntu/cephtest 2026-02-12T09:16:46.994 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-12T09:16:47.012 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-12T09:16:47.050 INFO:teuthology.run:Summary data: description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} perf_ignorelist settings/optimized workloads/sample_radosbench} duration: 540.1013317108154 flavor: debug owner: scheduled_matan@soko04.front.sepia.ceph.com success: true 2026-02-12T09:16:47.050 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-12T09:16:47.177 INFO:teuthology.run:pass