2026-02-20T12:03:26.324 INFO:root:teuthology version: 1.2.4.dev11+ge6d9099ce 2026-02-20T12:03:26.339 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-20T12:03:26.424 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:29:19-crimson-rados-wip-perezjos-crimson-only-12-02-2026-PR66245-distro-debug-trial/60625 branch: wip-perezjos-crimson-only-12-02-2026-PR66245 description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} perf_ignorelist settings/optimized workloads/fio_4K_rand_rw} email: null first_in_suite: false flavor: debug job_id: '60625' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: jjperez-2026-02-20_10:29:19-crimson-rados-wip-perezjos-crimson-only-12-02-2026-PR66245-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-perezjos-crimson-only-12-02-2026-PR66245 ceph: conf: global: auth client required: none auth cluster required: none auth service required: none auth supported: none debug asok: 0/0 debug auth: 0/0 debug bdev: 0/0 debug bluefs: 0/0 debug bluestore: 0/0 debug buffer: 0/0 debug civetweb: 0/0 debug client: 0/0 debug compressor: 0/0 debug context: 0/0 debug crush: 0/0 debug crypto: 0/0 debug dpdk: 0/0 debug eventtrace: 0/0 debug filer: 0/0 debug filestore: 0/0 debug finisher: 0/0 debug fuse: 0/0 debug heartbeatmap: 0/0 debug javaclient: 0/0 debug journal: 0/0 debug journaler: 0/0 debug kstore: 0/0 debug leveldb: 0/0 debug lockdep: 0/0 debug mds: 0/0 debug mds balancer: 0/0 debug mds locker: 0/0 debug mds log: 0/0 debug mds log expire: 0/0 debug mds migrator: 0/0 debug memdb: 0/0 debug mgr: 0/0 debug mgrc: 0/0 debug mon: 0/0 debug monc: 0/0 debug ms: 0/0 debug objclass: 0/0 debug objectcacher: 0/0 debug objecter: 0/0 debug optracker: 0/0 debug osd: 0/0 debug paxos: 0/0 debug perfcounter: 0/0 debug rados: 0/0 debug rbd: 0/0 debug rbd mirror: 0/0 debug rbd replay: 0/0 debug refs: 0/0 debug rgw: 0/0 debug rgw lifecycle: 0/0 debug rgw notification: 0/0 debug rgw sync: 0/0 debug rocksdb: 0/0 debug striper: 0/0 debug throttle: 0/0 debug timer: 0/0 debug tp: 0/0 enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 0/0 debug ms: 0/0 debug paxos: 0/0 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug filestore: 0/0 debug journal: 0/0 debug ms: 0/0 debug osd: 0/0 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore main device type: RANDOM_BLOCK_SSD seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - \(PG_ - \(OSD_ - \(OBJECT_ - overall HEALTH sha1: 26bc97655ae0056b78c14ab91d1307332bd6dfca ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: flavor: debug sha1: 26bc97655ae0056b78c14ab91d1307332bd6dfca workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: 445d5c9ba18a4040e70e25303230c36c77773acb owner: scheduled_jjperez@soko04.front.sepia.ceph.com priority: 101 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 5547 sha1: 26bc97655ae0056b78c14ab91d1307332bd6dfca sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: 445d5c9ba18a4040e70e25303230c36c77773acb targets: trial131.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGlv7HAgmqSscJYcergW8kdG4gz3ZdM0bq6HeVpKKAxyt9ZwUXgHoVxkFPk+TGY9aydadUJcGuF9bCKsCgZDRyA= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: osd: debug monc: 20 wait-for-scrub: false - ssh_keys: null - cbt: benchmarks: librbdfio: iodepth: - 32 log_avg_msec: 100 mode: - randrw norandommap: true op_size: - 4096 osd_ra: - 4096 pool_profile: rbd procs_per_volume: - 1 time: 60 vol_size: 4096 volumes_per_client: - 2 cluster: iterations: 1 osds_per_node: 3 pool_profiles: rbd: pg_size: 128 pgp_size: 128 replication: 3 user: ubuntu monitoring_profiles: perf: args: stat -p {pid} -o {perf_dir}/perf_stat.{pid} nodes: - osds perf_cmd: perf pid_glob: crimson-osd.*.pid teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: reimage-ssh-down teuthology_sha1: e6d9099ce03dc21193fda653159ec82465c3a94a timestamp: 2026-02-20_10:29:19 tube: trial user: jjperez verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-20T12:03:26.424 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa; will attempt to use it 2026-02-20T12:03:26.425 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks 2026-02-20T12:03:26.425 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-20T12:03:26.426 INFO:teuthology.task.internal:Checking packages... 2026-02-20T12:03:26.426 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash '26bc97655ae0056b78c14ab91d1307332bd6dfca' 2026-02-20T12:03:26.426 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-20T12:03:26.426 INFO:teuthology.packaging:ref: None 2026-02-20T12:03:26.426 INFO:teuthology.packaging:tag: None 2026-02-20T12:03:26.426 INFO:teuthology.packaging:branch: wip-perezjos-crimson-only-12-02-2026-PR66245 2026-02-20T12:03:26.426 INFO:teuthology.packaging:sha1: 26bc97655ae0056b78c14ab91d1307332bd6dfca 2026-02-20T12:03:26.426 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=wip-perezjos-crimson-only-12-02-2026-PR66245 2026-02-20T12:03:26.628 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5312.g26bc9765 2026-02-20T12:03:26.629 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-20T12:03:26.641 INFO:teuthology.task.internal:no buildpackages task found 2026-02-20T12:03:26.641 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-20T12:03:26.654 INFO:teuthology.task.internal:Saving configuration 2026-02-20T12:03:26.670 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-20T12:03:26.682 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-20T12:03:26.761 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial131.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:29:19-crimson-rados-wip-perezjos-crimson-only-12-02-2026-PR66245-distro-debug-trial/60625', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-20 12:00:00.516831', 'locked_by': 'scheduled_jjperez@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGlv7HAgmqSscJYcergW8kdG4gz3ZdM0bq6HeVpKKAxyt9ZwUXgHoVxkFPk+TGY9aydadUJcGuF9bCKsCgZDRyA='} 2026-02-20T12:03:26.761 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-20T12:03:26.772 INFO:teuthology.task.internal:roles: ubuntu@trial131.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-20T12:03:26.772 INFO:teuthology.run_tasks:Running task console_log... 2026-02-20T12:03:26.855 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x77a6f25ffec0>, signals=[15]) 2026-02-20T12:03:26.855 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-20T12:03:26.869 INFO:teuthology.task.internal:Opening connections... 2026-02-20T12:03:26.869 DEBUG:teuthology.task.internal:connecting to ubuntu@trial131.front.sepia.ceph.com 2026-02-20T12:03:26.870 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T12:03:26.930 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-20T12:03:26.942 DEBUG:teuthology.orchestra.run.trial131:> uname -m 2026-02-20T12:03:27.004 INFO:teuthology.orchestra.run.trial131.stdout:x86_64 2026-02-20T12:03:27.005 DEBUG:teuthology.orchestra.run.trial131:> cat /etc/os-release 2026-02-20T12:03:27.060 INFO:teuthology.orchestra.run.trial131.stdout:NAME="CentOS Stream" 2026-02-20T12:03:27.061 INFO:teuthology.orchestra.run.trial131.stdout:VERSION="9" 2026-02-20T12:03:27.061 INFO:teuthology.orchestra.run.trial131.stdout:ID="centos" 2026-02-20T12:03:27.061 INFO:teuthology.orchestra.run.trial131.stdout:ID_LIKE="rhel fedora" 2026-02-20T12:03:27.061 INFO:teuthology.orchestra.run.trial131.stdout:VERSION_ID="9" 2026-02-20T12:03:27.061 INFO:teuthology.orchestra.run.trial131.stdout:PLATFORM_ID="platform:el9" 2026-02-20T12:03:27.061 INFO:teuthology.orchestra.run.trial131.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-20T12:03:27.061 INFO:teuthology.orchestra.run.trial131.stdout:ANSI_COLOR="0;31" 2026-02-20T12:03:27.061 INFO:teuthology.orchestra.run.trial131.stdout:LOGO="fedora-logo-icon" 2026-02-20T12:03:27.061 INFO:teuthology.orchestra.run.trial131.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-20T12:03:27.061 INFO:teuthology.orchestra.run.trial131.stdout:HOME_URL="https://centos.org/" 2026-02-20T12:03:27.061 INFO:teuthology.orchestra.run.trial131.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-20T12:03:27.061 INFO:teuthology.orchestra.run.trial131.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-20T12:03:27.061 INFO:teuthology.orchestra.run.trial131.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-20T12:03:27.062 INFO:teuthology.lock.ops:Updating trial131.front.sepia.ceph.com on lock server 2026-02-20T12:03:27.148 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-20T12:03:27.159 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-20T12:03:27.176 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-20T12:03:27.176 DEBUG:teuthology.orchestra.run.trial131:> test '!' -e /home/ubuntu/cephtest 2026-02-20T12:03:27.191 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-20T12:03:27.202 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-20T12:03:27.202 DEBUG:teuthology.orchestra.run.trial131:> test -z $(ls -A /var/lib/ceph) 2026-02-20T12:03:27.247 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-20T12:03:27.349 INFO:teuthology.run_tasks:Running task kernel... 2026-02-20T12:03:27.367 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-20T12:03:27.368 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-20T12:03:27.368 DEBUG:teuthology.orchestra.run.trial131:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-20T12:03:27.382 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T12:03:27.382 DEBUG:teuthology.orchestra.run.trial131:> uname -r 2026-02-20T12:03:27.438 INFO:teuthology.orchestra.run.trial131.stdout:5.14.0-681.el9.x86_64 2026-02-20T12:03:27.438 INFO:teuthology.task.kernel:Running kernel on trial131: 5.14.0-681.el9.x86_64 2026-02-20T12:03:27.438 DEBUG:teuthology.orchestra.run.trial131:> sudo yum install -y kernel 2026-02-20T12:03:28.488 INFO:teuthology.orchestra.run.trial131.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.9 MB 00:00 2026-02-20T12:03:32.247 INFO:teuthology.orchestra.run.trial131.stdout:CentOS Stream 9 - AppStream 8.3 MB/s | 27 MB 00:03 2026-02-20T12:03:36.216 INFO:teuthology.orchestra.run.trial131.stdout:CentOS Stream 9 - CRB 4.9 MB/s | 7.8 MB 00:01 2026-02-20T12:03:37.051 INFO:teuthology.orchestra.run.trial131.stdout:CentOS Stream 9 - Extras packages 179 kB/s | 20 kB 00:00 2026-02-20T12:03:37.736 INFO:teuthology.orchestra.run.trial131.stdout:Extra Packages for Enterprise Linux 33 MB/s | 20 MB 00:00 2026-02-20T12:03:41.114 INFO:teuthology.orchestra.run.trial131.stdout:lab-extras 274 kB/s | 50 kB 00:00 2026-02-20T12:03:42.123 INFO:teuthology.orchestra.run.trial131.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-20T12:03:42.123 INFO:teuthology.orchestra.run.trial131.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-20T12:03:42.141 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:03:42.142 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:03:42.142 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:03:42.167 DEBUG:teuthology.orchestra.run.trial131:> echo no | sudo yum reinstall kernel || true 2026-02-20T12:03:42.470 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:01 ago on Fri 20 Feb 2026 12:03:41 PM UTC. 2026-02-20T12:03:42.551 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:03:42.552 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:03:42.552 INFO:teuthology.orchestra.run.trial131.stdout: Package Architecture Version Repository Size 2026-02-20T12:03:42.552 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:03:42.552 INFO:teuthology.orchestra.run.trial131.stdout:Reinstalling: 2026-02-20T12:03:42.552 INFO:teuthology.orchestra.run.trial131.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T12:03:42.552 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:03:42.552 INFO:teuthology.orchestra.run.trial131.stdout:Transaction Summary 2026-02-20T12:03:42.552 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:03:42.552 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:03:42.552 INFO:teuthology.orchestra.run.trial131.stdout:Total download size: 803 k 2026-02-20T12:03:42.553 INFO:teuthology.orchestra.run.trial131.stdout:Installed size: 0 2026-02-20T12:03:42.553 INFO:teuthology.orchestra.run.trial131.stderr:Operation aborted. 2026-02-20T12:03:42.572 INFO:teuthology.orchestra.run.trial131.stdout:Is this ok [y/N]: 2026-02-20T12:03:42.572 DEBUG:teuthology.orchestra.run.trial131:> sudo yum reinstall -y kernel || true 2026-02-20T12:03:42.865 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:01 ago on Fri 20 Feb 2026 12:03:41 PM UTC. 2026-02-20T12:03:42.946 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:03:42.946 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:03:42.946 INFO:teuthology.orchestra.run.trial131.stdout: Package Architecture Version Repository Size 2026-02-20T12:03:42.946 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:03:42.946 INFO:teuthology.orchestra.run.trial131.stdout:Reinstalling: 2026-02-20T12:03:42.946 INFO:teuthology.orchestra.run.trial131.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T12:03:42.946 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:03:42.947 INFO:teuthology.orchestra.run.trial131.stdout:Transaction Summary 2026-02-20T12:03:42.947 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:03:42.947 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:03:42.947 INFO:teuthology.orchestra.run.trial131.stdout:Total download size: 803 k 2026-02-20T12:03:42.947 INFO:teuthology.orchestra.run.trial131.stdout:Installed size: 0 2026-02-20T12:03:42.947 INFO:teuthology.orchestra.run.trial131.stdout:Downloading Packages: 2026-02-20T12:03:43.073 INFO:teuthology.orchestra.run.trial131.stdout:kernel-5.14.0-681.el9.x86_64.rpm 12 MB/s | 803 kB 00:00 2026-02-20T12:03:43.073 INFO:teuthology.orchestra.run.trial131.stdout:-------------------------------------------------------------------------------- 2026-02-20T12:03:43.073 INFO:teuthology.orchestra.run.trial131.stdout:Total 6.2 MB/s | 803 kB 00:00 2026-02-20T12:03:43.083 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction check 2026-02-20T12:03:43.116 INFO:teuthology.orchestra.run.trial131.stdout:Transaction check succeeded. 2026-02-20T12:03:43.116 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction test 2026-02-20T12:03:43.120 INFO:teuthology.orchestra.run.trial131.stdout:Transaction test succeeded. 2026-02-20T12:03:43.121 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction 2026-02-20T12:03:43.144 INFO:teuthology.orchestra.run.trial131.stdout: Preparing : 1/1 2026-02-20T12:03:43.148 INFO:teuthology.orchestra.run.trial131.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T12:03:43.149 INFO:teuthology.orchestra.run.trial131.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T12:03:43.260 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T12:03:43.261 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T12:03:43.317 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T12:03:43.317 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:03:43.318 INFO:teuthology.orchestra.run.trial131.stdout:Reinstalled: 2026-02-20T12:03:43.318 INFO:teuthology.orchestra.run.trial131.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-20T12:03:43.318 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:03:43.318 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:03:43.344 DEBUG:teuthology.orchestra.run.trial131:> rpm -q kernel | sort -rV | head -n 1 2026-02-20T12:03:43.367 INFO:teuthology.orchestra.run.trial131.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-20T12:03:43.368 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-20T12:03:43.368 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-20T12:03:43.368 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-20T12:03:43.369 DEBUG:teuthology.orchestra.run.trial131:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-20T12:03:43.432 INFO:teuthology.orchestra.run.trial131.stdout:ttyS1 2026-02-20T12:03:43.461 DEBUG:teuthology.parallel:result is None 2026-02-20T12:03:43.462 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-20T12:03:43.475 INFO:teuthology.task.internal:Creating test directory... 2026-02-20T12:03:43.475 DEBUG:teuthology.orchestra.run.trial131:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-20T12:03:43.517 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-20T12:03:43.529 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-20T12:03:43.540 INFO:teuthology.task.internal:Creating archive directory... 2026-02-20T12:03:43.540 DEBUG:teuthology.orchestra.run.trial131:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-20T12:03:43.578 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-20T12:03:43.590 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-20T12:03:43.590 DEBUG:teuthology.orchestra.run.trial131:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-20T12:03:43.637 INFO:teuthology.orchestra.run.trial131.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T12:03:43.646 INFO:teuthology.orchestra.run.trial131.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T12:03:43.647 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-20T12:03:43.659 INFO:teuthology.task.internal:Configuring sudo... 2026-02-20T12:03:43.659 DEBUG:teuthology.orchestra.run.trial131:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-20T12:03:43.711 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-20T12:03:43.727 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-20T12:03:43.727 DEBUG:teuthology.orchestra.run.trial131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-20T12:03:43.766 DEBUG:teuthology.orchestra.run.trial131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T12:03:43.927 DEBUG:teuthology.orchestra.run.trial131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T12:03:43.950 DEBUG:teuthology.orchestra.run.trial131:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T12:03:44.097 DEBUG:teuthology.orchestra.run.trial131:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T12:03:44.120 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:03:44.120 DEBUG:teuthology.orchestra.run.trial131:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-20T12:03:44.183 DEBUG:teuthology.orchestra.run.trial131:> sudo service rsyslog restart 2026-02-20T12:03:44.252 INFO:teuthology.orchestra.run.trial131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-20T12:03:44.770 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-20T12:03:44.783 INFO:teuthology.task.internal:Starting timer... 2026-02-20T12:03:44.783 INFO:teuthology.run_tasks:Running task pcp... 2026-02-20T12:03:44.804 INFO:teuthology.run_tasks:Running task selinux... 2026-02-20T12:03:44.822 DEBUG:teuthology.orchestra.run.trial131:> sudo service auditd rotate 2026-02-20T12:03:44.851 INFO:teuthology.orchestra.run.trial131.stdout:Rotating logs: 2026-02-20T12:03:44.853 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-20T12:03:44.853 DEBUG:teuthology.orchestra.run.trial131:> /usr/sbin/getenforce 2026-02-20T12:03:44.909 INFO:teuthology.orchestra.run.trial131.stdout:Permissive 2026-02-20T12:03:44.909 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial131.front.sepia.ceph.com': 'permissive'} 2026-02-20T12:03:44.910 DEBUG:teuthology.orchestra.run.trial131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-20T12:03:44.974 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T12:03:44.974 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-20T12:03:44.975 DEBUG:teuthology.orchestra.run.trial131:> sudo /usr/sbin/setenforce permissive 2026-02-20T12:03:45.039 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-20T12:03:45.056 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-20T12:03:45.058 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-20T12:03:45.263 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-20T12:03:45.275 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-20T12:03:45.277 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial131.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-20T12:06:31.772 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial131.front.sepia.ceph.com')] 2026-02-20T12:06:31.774 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial131.front.sepia.ceph.com' 2026-02-20T12:06:31.776 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial131.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T12:06:31.881 DEBUG:teuthology.orchestra.run.trial131:> true 2026-02-20T12:06:31.957 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial131.front.sepia.ceph.com' 2026-02-20T12:06:31.958 INFO:teuthology.run_tasks:Running task clock... 2026-02-20T12:06:31.970 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-20T12:06:31.971 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-20T12:06:31.971 DEBUG:teuthology.orchestra.run.trial131:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-20T12:06:32.024 INFO:teuthology.orchestra.run.trial131.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-20T12:06:32.036 INFO:teuthology.orchestra.run.trial131.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-20T12:06:32.056 INFO:teuthology.orchestra.run.trial131.stderr:sudo: ntpd: command not found 2026-02-20T12:06:32.066 INFO:teuthology.orchestra.run.trial131.stdout:506 Cannot talk to daemon 2026-02-20T12:06:32.078 INFO:teuthology.orchestra.run.trial131.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-20T12:06:32.090 INFO:teuthology.orchestra.run.trial131.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-20T12:06:32.137 INFO:teuthology.orchestra.run.trial131.stderr:bash: line 1: ntpq: command not found 2026-02-20T12:06:32.138 INFO:teuthology.orchestra.run.trial131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-20T12:06:32.138 INFO:teuthology.orchestra.run.trial131.stdout:=============================================================================== 2026-02-20T12:06:32.139 INFO:teuthology.run_tasks:Running task install... 2026-02-20T12:06:32.156 DEBUG:teuthology.task.install:project ceph 2026-02-20T12:06:32.157 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '26bc97655ae0056b78c14ab91d1307332bd6dfca'}} 2026-02-20T12:06:32.157 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '26bc97655ae0056b78c14ab91d1307332bd6dfca'} 2026-02-20T12:06:32.157 INFO:teuthology.task.install:Using flavor: debug 2026-02-20T12:06:32.166 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-20T12:06:32.166 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-20T12:06:32.167 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '26bc97655ae0056b78c14ab91d1307332bd6dfca', 'tag': None, 'wait_for_package': False} 2026-02-20T12:06:32.168 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=26bc97655ae0056b78c14ab91d1307332bd6dfca 2026-02-20T12:06:32.333 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-12-02-2026-PR66245/26bc97655ae0056b78c14ab91d1307332bd6dfca/centos/9/flavors/debug/ 2026-02-20T12:06:32.333 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5312.g26bc9765 2026-02-20T12:06:32.626 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-12-02-2026-PR66245/26bc97655ae0056b78c14ab91d1307332bd6dfca/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-12-02-2026-PR66245/26bc97655ae0056b78c14ab91d1307332bd6dfca/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-12-02-2026-PR66245/26bc97655ae0056b78c14ab91d1307332bd6dfca/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-20T12:06:32.627 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:06:32.627 DEBUG:teuthology.orchestra.run.trial131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-20T12:06:32.653 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-20T12:06:32.653 DEBUG:teuthology.orchestra.run.trial131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/26bc97655ae0056b78c14ab91d1307332bd6dfca/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-20T12:06:32.720 DEBUG:teuthology.orchestra.run.trial131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-20T12:06:32.794 DEBUG:teuthology.orchestra.run.trial131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-20T12:06:32.858 INFO:teuthology.orchestra.run.trial131.stdout:check_obsoletes = 1 2026-02-20T12:06:32.859 DEBUG:teuthology.orchestra.run.trial131:> sudo yum clean all 2026-02-20T12:06:33.016 INFO:teuthology.orchestra.run.trial131.stdout:41 files removed 2026-02-20T12:06:33.031 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-radosgw 2026-02-20T12:06:33.487 INFO:teuthology.orchestra.run.trial131.stdout:ceph packages for x86_64 299 kB/s | 94 kB 00:00 2026-02-20T12:06:33.626 INFO:teuthology.orchestra.run.trial131.stdout:ceph noarch packages 215 kB/s | 24 kB 00:00 2026-02-20T12:06:33.753 INFO:teuthology.orchestra.run.trial131.stdout:ceph source packages 20 kB/s | 2.1 kB 00:00 2026-02-20T12:06:34.951 INFO:teuthology.orchestra.run.trial131.stdout:CentOS Stream 9 - BaseOS 7.5 MB/s | 8.9 MB 00:01 2026-02-20T12:06:37.208 INFO:teuthology.orchestra.run.trial131.stdout:CentOS Stream 9 - AppStream 15 MB/s | 27 MB 00:01 2026-02-20T12:06:40.541 INFO:teuthology.orchestra.run.trial131.stdout:CentOS Stream 9 - CRB 8.4 MB/s | 7.8 MB 00:00 2026-02-20T12:06:41.724 INFO:teuthology.orchestra.run.trial131.stdout:CentOS Stream 9 - Extras packages 44 kB/s | 20 kB 00:00 2026-02-20T12:06:43.440 INFO:teuthology.orchestra.run.trial131.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2026-02-20T12:06:46.614 INFO:teuthology.orchestra.run.trial131.stdout:lab-extras 2.9 MB/s | 50 kB 00:00 2026-02-20T12:06:47.807 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:06:47.808 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:06:47.808 INFO:teuthology.orchestra.run.trial131.stdout: Package Arch Version Repository Size 2026-02-20T12:06:47.808 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:06:47.808 INFO:teuthology.orchestra.run.trial131.stdout:Installing: 2026-02-20T12:06:47.808 INFO:teuthology.orchestra.run.trial131.stdout: ceph-radosgw x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 11 M 2026-02-20T12:06:47.808 INFO:teuthology.orchestra.run.trial131.stdout:Upgrading: 2026-02-20T12:06:47.808 INFO:teuthology.orchestra.run.trial131.stdout: librados2 x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 3.3 M 2026-02-20T12:06:47.809 INFO:teuthology.orchestra.run.trial131.stdout: librbd1 x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 2.7 M 2026-02-20T12:06:47.809 INFO:teuthology.orchestra.run.trial131.stdout:Installing dependencies: 2026-02-20T12:06:47.809 INFO:teuthology.orchestra.run.trial131.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-20T12:06:47.809 INFO:teuthology.orchestra.run.trial131.stdout: ceph-base x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 5.1 M 2026-02-20T12:06:47.809 INFO:teuthology.orchestra.run.trial131.stdout: ceph-common x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 21 M 2026-02-20T12:06:47.809 INFO:teuthology.orchestra.run.trial131.stdout: ceph-selinux x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 25 k 2026-02-20T12:06:47.809 INFO:teuthology.orchestra.run.trial131.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-20T12:06:47.809 INFO:teuthology.orchestra.run.trial131.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-20T12:06:47.809 INFO:teuthology.orchestra.run.trial131.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-20T12:06:47.809 INFO:teuthology.orchestra.run.trial131.stdout: libcephfs2 x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 860 k 2026-02-20T12:06:47.809 INFO:teuthology.orchestra.run.trial131.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-20T12:06:47.810 INFO:teuthology.orchestra.run.trial131.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-20T12:06:47.810 INFO:teuthology.orchestra.run.trial131.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-20T12:06:47.810 INFO:teuthology.orchestra.run.trial131.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-20T12:06:47.810 INFO:teuthology.orchestra.run.trial131.stdout: libradosstriper1 x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 263 k 2026-02-20T12:06:47.810 INFO:teuthology.orchestra.run.trial131.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-20T12:06:47.810 INFO:teuthology.orchestra.run.trial131.stdout: librgw2 x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 5.6 M 2026-02-20T12:06:47.810 INFO:teuthology.orchestra.run.trial131.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-20T12:06:47.810 INFO:teuthology.orchestra.run.trial131.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-20T12:06:47.810 INFO:teuthology.orchestra.run.trial131.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-20T12:06:47.810 INFO:teuthology.orchestra.run.trial131.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-20T12:06:47.810 INFO:teuthology.orchestra.run.trial131.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-20T12:06:47.810 INFO:teuthology.orchestra.run.trial131.stdout: python3-ceph-argparse x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 45 k 2026-02-20T12:06:47.811 INFO:teuthology.orchestra.run.trial131.stdout: python3-ceph-common x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 185 k 2026-02-20T12:06:47.811 INFO:teuthology.orchestra.run.trial131.stdout: python3-cephfs x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 203 k 2026-02-20T12:06:47.811 INFO:teuthology.orchestra.run.trial131.stdout: python3-rados x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 324 k 2026-02-20T12:06:47.811 INFO:teuthology.orchestra.run.trial131.stdout: python3-rbd x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 305 k 2026-02-20T12:06:47.811 INFO:teuthology.orchestra.run.trial131.stdout: python3-rgw x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 100 k 2026-02-20T12:06:47.811 INFO:teuthology.orchestra.run.trial131.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-20T12:06:47.811 INFO:teuthology.orchestra.run.trial131.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-20T12:06:47.811 INFO:teuthology.orchestra.run.trial131.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-20T12:06:47.811 INFO:teuthology.orchestra.run.trial131.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-20T12:06:47.811 INFO:teuthology.orchestra.run.trial131.stdout:Installing weak dependencies: 2026-02-20T12:06:47.811 INFO:teuthology.orchestra.run.trial131.stdout: libcephfs-daemon x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 36 k 2026-02-20T12:06:47.811 INFO:teuthology.orchestra.run.trial131.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 25 k 2026-02-20T12:06:47.811 INFO:teuthology.orchestra.run.trial131.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-20T12:06:47.812 INFO:teuthology.orchestra.run.trial131.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-20T12:06:47.812 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:06:47.812 INFO:teuthology.orchestra.run.trial131.stdout:Transaction Summary 2026-02-20T12:06:47.812 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:06:47.812 INFO:teuthology.orchestra.run.trial131.stdout:Install 35 Packages 2026-02-20T12:06:47.812 INFO:teuthology.orchestra.run.trial131.stdout:Upgrade 2 Packages 2026-02-20T12:06:47.812 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:06:47.813 INFO:teuthology.orchestra.run.trial131.stdout:Total download size: 61 M 2026-02-20T12:06:47.813 INFO:teuthology.orchestra.run.trial131.stdout:Downloading Packages: 2026-02-20T12:06:48.331 INFO:teuthology.orchestra.run.trial131.stdout:(1/37): ceph-base-20.3.0-5312.g26bc9765.el9.x86 18 MB/s | 5.1 MB 00:00 2026-02-20T12:06:48.366 INFO:teuthology.orchestra.run.trial131.stdout:(2/37): ceph-selinux-20.3.0-5312.g26bc9765.el9. 706 kB/s | 25 kB 00:00 2026-02-20T12:06:48.404 INFO:teuthology.orchestra.run.trial131.stdout:(3/37): libcephfs-daemon-20.3.0-5312.g26bc9765. 955 kB/s | 36 kB 00:00 2026-02-20T12:06:48.444 INFO:teuthology.orchestra.run.trial131.stdout:(4/37): libcephfs-proxy2-20.3.0-5312.g26bc9765. 634 kB/s | 25 kB 00:00 2026-02-20T12:06:48.502 INFO:teuthology.orchestra.run.trial131.stdout:(5/37): libcephfs2-20.3.0-5312.g26bc9765.el9.x8 14 MB/s | 860 kB 00:00 2026-02-20T12:06:48.550 INFO:teuthology.orchestra.run.trial131.stdout:(6/37): libradosstriper1-20.3.0-5312.g26bc9765. 5.3 MB/s | 263 kB 00:00 2026-02-20T12:06:48.715 INFO:teuthology.orchestra.run.trial131.stdout:(7/37): ceph-radosgw-20.3.0-5312.g26bc9765.el9. 17 MB/s | 11 MB 00:00 2026-02-20T12:06:48.756 INFO:teuthology.orchestra.run.trial131.stdout:(8/37): python3-ceph-argparse-20.3.0-5312.g26bc 1.1 MB/s | 45 kB 00:00 2026-02-20T12:06:48.806 INFO:teuthology.orchestra.run.trial131.stdout:(9/37): python3-ceph-common-20.3.0-5312.g26bc97 3.6 MB/s | 185 kB 00:00 2026-02-20T12:06:48.838 INFO:teuthology.orchestra.run.trial131.stdout:(10/37): librgw2-20.3.0-5312.g26bc9765.el9.x86_ 20 MB/s | 5.6 MB 00:00 2026-02-20T12:06:48.855 INFO:teuthology.orchestra.run.trial131.stdout:(11/37): python3-cephfs-20.3.0-5312.g26bc9765.e 4.0 MB/s | 203 kB 00:00 2026-02-20T12:06:48.880 INFO:teuthology.orchestra.run.trial131.stdout:(12/37): python3-rados-20.3.0-5312.g26bc9765.el 7.5 MB/s | 324 kB 00:00 2026-02-20T12:06:48.906 INFO:teuthology.orchestra.run.trial131.stdout:(13/37): ceph-common-20.3.0-5312.g26bc9765.el9. 24 MB/s | 21 MB 00:00 2026-02-20T12:06:48.907 INFO:teuthology.orchestra.run.trial131.stdout:(14/37): python3-rgw-20.3.0-5312.g26bc9765.el9. 3.6 MB/s | 100 kB 00:00 2026-02-20T12:06:48.908 INFO:teuthology.orchestra.run.trial131.stdout:(15/37): python3-rbd-20.3.0-5312.g26bc9765.el9. 5.6 MB/s | 305 kB 00:00 2026-02-20T12:06:49.471 INFO:teuthology.orchestra.run.trial131.stdout:(16/37): mailcap-2.1.49-5.el9.noarch.rpm 59 kB/s | 33 kB 00:00 2026-02-20T12:06:49.517 INFO:teuthology.orchestra.run.trial131.stdout:(17/37): boost-program-options-1.75.0-13.el9.x8 2.2 MB/s | 104 kB 00:00 2026-02-20T12:06:49.537 INFO:teuthology.orchestra.run.trial131.stdout:(18/37): libnbd-1.20.3-4.el9.x86_64.rpm 8.0 MB/s | 164 kB 00:00 2026-02-20T12:06:49.550 INFO:teuthology.orchestra.run.trial131.stdout:(19/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 12 MB/s | 160 kB 00:00 2026-02-20T12:06:49.560 INFO:teuthology.orchestra.run.trial131.stdout:(20/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 4.4 MB/s | 45 kB 00:00 2026-02-20T12:06:49.586 INFO:teuthology.orchestra.run.trial131.stdout:(21/37): librdkafka-1.6.1-102.el9.x86_64.rpm 25 MB/s | 662 kB 00:00 2026-02-20T12:06:49.599 INFO:teuthology.orchestra.run.trial131.stdout:(22/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 24 MB/s | 292 kB 00:00 2026-02-20T12:06:49.610 INFO:teuthology.orchestra.run.trial131.stdout:(23/37): qatlib-25.08.0-2.el9.x86_64.rpm 20 MB/s | 240 kB 00:00 2026-02-20T12:06:49.621 INFO:teuthology.orchestra.run.trial131.stdout:(24/37): qatlib-service-25.08.0-2.el9.x86_64.rp 3.7 MB/s | 37 kB 00:00 2026-02-20T12:06:49.624 INFO:teuthology.orchestra.run.trial131.stdout:(25/37): pciutils-3.7.0-7.el9.x86_64.rpm 130 kB/s | 93 kB 00:00 2026-02-20T12:06:49.631 INFO:teuthology.orchestra.run.trial131.stdout:(26/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 6.2 MB/s | 66 kB 00:00 2026-02-20T12:06:50.030 INFO:teuthology.orchestra.run.trial131.stdout:(27/37): nvme-cli-2.16-1.el9.x86_64.rpm 1.0 MB/s | 1.2 MB 00:01 2026-02-20T12:06:50.271 INFO:teuthology.orchestra.run.trial131.stdout:(28/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 475 kB/s | 308 kB 00:00 2026-02-20T12:06:50.351 INFO:teuthology.orchestra.run.trial131.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 615 kB/s | 49 kB 00:00 2026-02-20T12:06:50.427 INFO:teuthology.orchestra.run.trial131.stdout:(30/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 62 kB/s | 25 kB 00:00 2026-02-20T12:06:50.430 INFO:teuthology.orchestra.run.trial131.stdout:(31/37): libunwind-1.6.2-1.el9.x86_64.rpm 847 kB/s | 67 kB 00:00 2026-02-20T12:06:50.511 INFO:teuthology.orchestra.run.trial131.stdout:(32/37): re2-20211101-20.el9.x86_64.rpm 2.3 MB/s | 191 kB 00:00 2026-02-20T12:06:50.593 INFO:teuthology.orchestra.run.trial131.stdout:(33/37): libarrow-9.0.0-15.el9.x86_64.rpm 4.6 MB/s | 4.4 MB 00:00 2026-02-20T12:06:50.705 INFO:teuthology.orchestra.run.trial131.stdout:(34/37): librados2-20.3.0-5312.g26bc9765.el9.x8 29 MB/s | 3.3 MB 00:00 2026-02-20T12:06:50.755 INFO:teuthology.orchestra.run.trial131.stdout:(35/37): librbd1-20.3.0-5312.g26bc9765.el9.x86_ 53 MB/s | 2.7 MB 00:00 2026-02-20T12:06:50.832 INFO:teuthology.orchestra.run.trial131.stdout:(36/37): thrift-0.15.0-4.el9.x86_64.rpm 4.9 MB/s | 1.6 MB 00:00 2026-02-20T12:06:51.285 INFO:teuthology.orchestra.run.trial131.stdout:(37/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 977 kB/s | 838 kB 00:00 2026-02-20T12:06:51.292 INFO:teuthology.orchestra.run.trial131.stdout:-------------------------------------------------------------------------------- 2026-02-20T12:06:51.293 INFO:teuthology.orchestra.run.trial131.stdout:Total 18 MB/s | 61 MB 00:03 2026-02-20T12:06:51.389 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction check 2026-02-20T12:06:51.409 INFO:teuthology.orchestra.run.trial131.stdout:Transaction check succeeded. 2026-02-20T12:06:51.409 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction test 2026-02-20T12:06:51.662 INFO:teuthology.orchestra.run.trial131.stdout:Transaction test succeeded. 2026-02-20T12:06:51.662 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction 2026-02-20T12:06:51.952 INFO:teuthology.orchestra.run.trial131.stdout: Preparing : 1/1 2026-02-20T12:06:52.092 INFO:teuthology.orchestra.run.trial131.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-20T12:06:52.096 INFO:teuthology.orchestra.run.trial131.stdout: Upgrading : librados2-2:20.3.0-5312.g26bc9765.el9.x86_64 2/39 2026-02-20T12:06:52.146 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: librados2-2:20.3.0-5312.g26bc9765.el9.x86_64 2/39 2026-02-20T12:06:52.147 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libcephfs2-2:20.3.0-5312.g26bc9765.el9.x86_64 3/39 2026-02-20T12:06:52.160 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: libcephfs2-2:20.3.0-5312.g26bc9765.el9.x86_64 3/39 2026-02-20T12:06:52.162 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libcephfs-daemon-2:20.3.0-5312.g26bc9765.el9.x86_6 4/39 2026-02-20T12:06:52.163 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libcephfs-proxy2-2:20.3.0-5312.g26bc9765.el9.x86_6 5/39 2026-02-20T12:06:52.190 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5312.g26bc9765.el9.x86_6 5/39 2026-02-20T12:06:52.193 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-rados-2:20.3.0-5312.g26bc9765.el9.x86_64 6/39 2026-02-20T12:06:52.201 INFO:teuthology.orchestra.run.trial131.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-20T12:06:52.203 INFO:teuthology.orchestra.run.trial131.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-20T12:06:52.206 INFO:teuthology.orchestra.run.trial131.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-20T12:06:52.210 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-20T12:06:52.221 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-ceph-argparse-2:20.3.0-5312.g26bc9765.el9. 11/39 2026-02-20T12:06:52.234 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-cephfs-2:20.3.0-5312.g26bc9765.el9.x86_64 12/39 2026-02-20T12:06:52.234 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libradosstriper1-2:20.3.0-5312.g26bc9765.el9.x86_6 13/39 2026-02-20T12:06:52.256 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5312.g26bc9765.el9.x86_6 13/39 2026-02-20T12:06:52.260 INFO:teuthology.orchestra.run.trial131.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-20T12:06:52.291 INFO:teuthology.orchestra.run.trial131.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-20T12:06:52.296 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-20T12:06:52.302 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-20T12:06:52.304 INFO:teuthology.orchestra.run.trial131.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-20T12:06:52.309 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-20T12:06:52.423 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-20T12:06:52.424 INFO:teuthology.orchestra.run.trial131.stdout: Upgrading : librbd1-2:20.3.0-5312.g26bc9765.el9.x86_64 21/39 2026-02-20T12:06:52.450 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: librbd1-2:20.3.0-5312.g26bc9765.el9.x86_64 21/39 2026-02-20T12:06:52.453 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-rbd-2:20.3.0-5312.g26bc9765.el9.x86_64 22/39 2026-02-20T12:06:52.458 INFO:teuthology.orchestra.run.trial131.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-20T12:06:52.707 INFO:teuthology.orchestra.run.trial131.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-20T12:06:52.708 INFO:teuthology.orchestra.run.trial131.stdout: Installing : librgw2-2:20.3.0-5312.g26bc9765.el9.x86_64 25/39 2026-02-20T12:06:52.725 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: librgw2-2:20.3.0-5312.g26bc9765.el9.x86_64 25/39 2026-02-20T12:06:52.730 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-rgw-2:20.3.0-5312.g26bc9765.el9.x86_64 26/39 2026-02-20T12:06:52.735 INFO:teuthology.orchestra.run.trial131.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-20T12:06:52.737 INFO:teuthology.orchestra.run.trial131.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-20T12:06:52.738 INFO:teuthology.orchestra.run.trial131.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-20T12:06:52.756 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-20T12:06:52.873 INFO:teuthology.orchestra.run.trial131.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-20T12:06:52.876 INFO:teuthology.orchestra.run.trial131.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-20T12:06:53.160 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-20T12:06:53.160 INFO:teuthology.orchestra.run.trial131.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-20T12:06:53.160 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:06:53.204 INFO:teuthology.orchestra.run.trial131.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-20T12:06:53.206 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-ceph-common-2:20.3.0-5312.g26bc9765.el9.x8 33/39 2026-02-20T12:06:54.143 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-common-2:20.3.0-5312.g26bc9765.el9.x86_64 34/39 2026-02-20T12:06:54.145 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-common-2:20.3.0-5312.g26bc9765.el9.x86_64 34/39 2026-02-20T12:06:54.408 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-common-2:20.3.0-5312.g26bc9765.el9.x86_64 34/39 2026-02-20T12:06:54.411 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-base-2:20.3.0-5312.g26bc9765.el9.x86_64 35/39 2026-02-20T12:06:54.456 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-base-2:20.3.0-5312.g26bc9765.el9.x86_64 35/39 2026-02-20T12:06:54.456 INFO:teuthology.orchestra.run.trial131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-20T12:06:54.456 INFO:teuthology.orchestra.run.trial131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-20T12:06:54.456 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:06:54.460 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-selinux-2:20.3.0-5312.g26bc9765.el9.x86_64 36/39 2026-02-20T12:06:58.978 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5312.g26bc9765.el9.x86_64 36/39 2026-02-20T12:06:58.978 INFO:teuthology.orchestra.run.trial131.stdout:skipping the directory /sys 2026-02-20T12:06:58.979 INFO:teuthology.orchestra.run.trial131.stdout:skipping the directory /proc 2026-02-20T12:06:58.979 INFO:teuthology.orchestra.run.trial131.stdout:skipping the directory /mnt 2026-02-20T12:06:58.979 INFO:teuthology.orchestra.run.trial131.stdout:skipping the directory /var/tmp 2026-02-20T12:06:58.979 INFO:teuthology.orchestra.run.trial131.stdout:skipping the directory /home 2026-02-20T12:06:58.979 INFO:teuthology.orchestra.run.trial131.stdout:skipping the directory /root 2026-02-20T12:06:58.979 INFO:teuthology.orchestra.run.trial131.stdout:skipping the directory /tmp 2026-02-20T12:06:58.979 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:06:59.472 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-radosgw-2:20.3.0-5312.g26bc9765.el9.x86_64 37/39 2026-02-20T12:06:59.496 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5312.g26bc9765.el9.x86_64 37/39 2026-02-20T12:06:59.496 INFO:teuthology.orchestra.run.trial131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T12:06:59.496 INFO:teuthology.orchestra.run.trial131.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-20T12:06:59.497 INFO:teuthology.orchestra.run.trial131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T12:06:59.497 INFO:teuthology.orchestra.run.trial131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T12:06:59.497 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:06:59.498 INFO:teuthology.orchestra.run.trial131.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-20T12:06:59.511 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-20T12:06:59.511 INFO:teuthology.orchestra.run.trial131.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-20T12:06:59.567 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-20T12:06:59.567 INFO:teuthology.orchestra.run.trial131.stdout:Creating group 'qat' with GID 992. 2026-02-20T12:06:59.567 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:06:59.870 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-base-2:20.3.0-5312.g26bc9765.el9.x86_64 1/39 2026-02-20T12:06:59.870 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-common-2:20.3.0-5312.g26bc9765.el9.x86_64 2/39 2026-02-20T12:06:59.870 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-radosgw-2:20.3.0-5312.g26bc9765.el9.x86_64 3/39 2026-02-20T12:06:59.870 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-selinux-2:20.3.0-5312.g26bc9765.el9.x86_64 4/39 2026-02-20T12:06:59.870 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libcephfs-daemon-2:20.3.0-5312.g26bc9765.el9.x86_6 5/39 2026-02-20T12:06:59.870 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5312.g26bc9765.el9.x86_6 6/39 2026-02-20T12:06:59.870 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libcephfs2-2:20.3.0-5312.g26bc9765.el9.x86_64 7/39 2026-02-20T12:06:59.870 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libradosstriper1-2:20.3.0-5312.g26bc9765.el9.x86_6 8/39 2026-02-20T12:06:59.871 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : librgw2-2:20.3.0-5312.g26bc9765.el9.x86_64 9/39 2026-02-20T12:06:59.871 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5312.g26bc9765.el9. 10/39 2026-02-20T12:06:59.871 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-ceph-common-2:20.3.0-5312.g26bc9765.el9.x8 11/39 2026-02-20T12:06:59.871 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-cephfs-2:20.3.0-5312.g26bc9765.el9.x86_64 12/39 2026-02-20T12:06:59.871 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-rados-2:20.3.0-5312.g26bc9765.el9.x86_64 13/39 2026-02-20T12:06:59.871 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-rbd-2:20.3.0-5312.g26bc9765.el9.x86_64 14/39 2026-02-20T12:06:59.871 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-rgw-2:20.3.0-5312.g26bc9765.el9.x86_64 15/39 2026-02-20T12:06:59.871 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-20T12:06:59.871 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-20T12:06:59.871 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-20T12:06:59.871 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-20T12:06:59.871 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-20T12:06:59.871 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-20T12:06:59.872 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-20T12:06:59.872 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-20T12:06:59.872 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-20T12:06:59.872 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-20T12:06:59.872 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-20T12:06:59.872 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-20T12:06:59.872 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-20T12:06:59.872 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-20T12:06:59.872 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-20T12:06:59.872 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-20T12:06:59.872 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-20T12:06:59.872 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-20T12:06:59.872 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-20T12:06:59.873 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-20T12:06:59.873 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : librados2-2:20.3.0-5312.g26bc9765.el9.x86_64 36/39 2026-02-20T12:06:59.873 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-20T12:06:59.873 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : librbd1-2:20.3.0-5312.g26bc9765.el9.x86_64 38/39 2026-02-20T12:06:59.932 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-20T12:06:59.932 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:06:59.932 INFO:teuthology.orchestra.run.trial131.stdout:Upgraded: 2026-02-20T12:06:59.932 INFO:teuthology.orchestra.run.trial131.stdout: librados2-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.932 INFO:teuthology.orchestra.run.trial131.stdout: librbd1-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.932 INFO:teuthology.orchestra.run.trial131.stdout:Installed: 2026-02-20T12:06:59.932 INFO:teuthology.orchestra.run.trial131.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-20T12:06:59.933 INFO:teuthology.orchestra.run.trial131.stdout: ceph-base-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.933 INFO:teuthology.orchestra.run.trial131.stdout: ceph-common-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.933 INFO:teuthology.orchestra.run.trial131.stdout: ceph-radosgw-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.933 INFO:teuthology.orchestra.run.trial131.stdout: ceph-selinux-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.933 INFO:teuthology.orchestra.run.trial131.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-20T12:06:59.933 INFO:teuthology.orchestra.run.trial131.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-20T12:06:59.933 INFO:teuthology.orchestra.run.trial131.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-20T12:06:59.933 INFO:teuthology.orchestra.run.trial131.stdout: libcephfs-daemon-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.933 INFO:teuthology.orchestra.run.trial131.stdout: libcephfs-proxy2-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.933 INFO:teuthology.orchestra.run.trial131.stdout: libcephfs2-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.933 INFO:teuthology.orchestra.run.trial131.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-20T12:06:59.933 INFO:teuthology.orchestra.run.trial131.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-20T12:06:59.933 INFO:teuthology.orchestra.run.trial131.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-20T12:06:59.934 INFO:teuthology.orchestra.run.trial131.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-20T12:06:59.934 INFO:teuthology.orchestra.run.trial131.stdout: libradosstriper1-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.934 INFO:teuthology.orchestra.run.trial131.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-20T12:06:59.934 INFO:teuthology.orchestra.run.trial131.stdout: librgw2-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.934 INFO:teuthology.orchestra.run.trial131.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-20T12:06:59.934 INFO:teuthology.orchestra.run.trial131.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-20T12:06:59.934 INFO:teuthology.orchestra.run.trial131.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-20T12:06:59.934 INFO:teuthology.orchestra.run.trial131.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-20T12:06:59.934 INFO:teuthology.orchestra.run.trial131.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-20T12:06:59.934 INFO:teuthology.orchestra.run.trial131.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-20T12:06:59.934 INFO:teuthology.orchestra.run.trial131.stdout: python3-ceph-argparse-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.934 INFO:teuthology.orchestra.run.trial131.stdout: python3-ceph-common-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.935 INFO:teuthology.orchestra.run.trial131.stdout: python3-cephfs-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.935 INFO:teuthology.orchestra.run.trial131.stdout: python3-rados-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.935 INFO:teuthology.orchestra.run.trial131.stdout: python3-rbd-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.935 INFO:teuthology.orchestra.run.trial131.stdout: python3-rgw-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:06:59.935 INFO:teuthology.orchestra.run.trial131.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-20T12:06:59.935 INFO:teuthology.orchestra.run.trial131.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-20T12:06:59.935 INFO:teuthology.orchestra.run.trial131.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-20T12:06:59.935 INFO:teuthology.orchestra.run.trial131.stdout: re2-1:20211101-20.el9.x86_64 2026-02-20T12:06:59.935 INFO:teuthology.orchestra.run.trial131.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-20T12:06:59.935 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:06:59.935 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:06:59.974 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-test 2026-02-20T12:07:00.286 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:14 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:07:00.369 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:07:00.371 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:07:00.371 INFO:teuthology.orchestra.run.trial131.stdout: Package Arch Version Repository Size 2026-02-20T12:07:00.371 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:07:00.371 INFO:teuthology.orchestra.run.trial131.stdout:Installing: 2026-02-20T12:07:00.372 INFO:teuthology.orchestra.run.trial131.stdout: ceph-test x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 125 M 2026-02-20T12:07:00.372 INFO:teuthology.orchestra.run.trial131.stdout:Installing dependencies: 2026-02-20T12:07:00.372 INFO:teuthology.orchestra.run.trial131.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-20T12:07:00.372 INFO:teuthology.orchestra.run.trial131.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-20T12:07:00.372 INFO:teuthology.orchestra.run.trial131.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-20T12:07:00.372 INFO:teuthology.orchestra.run.trial131.stdout: libcephsqlite x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 153 k 2026-02-20T12:07:00.373 INFO:teuthology.orchestra.run.trial131.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-20T12:07:00.373 INFO:teuthology.orchestra.run.trial131.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-20T12:07:00.373 INFO:teuthology.orchestra.run.trial131.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-20T12:07:00.373 INFO:teuthology.orchestra.run.trial131.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-20T12:07:00.373 INFO:teuthology.orchestra.run.trial131.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-20T12:07:00.373 INFO:teuthology.orchestra.run.trial131.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-20T12:07:00.374 INFO:teuthology.orchestra.run.trial131.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-20T12:07:00.374 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:00.374 INFO:teuthology.orchestra.run.trial131.stdout:Transaction Summary 2026-02-20T12:07:00.374 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:07:00.374 INFO:teuthology.orchestra.run.trial131.stdout:Install 12 Packages 2026-02-20T12:07:00.374 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:00.375 INFO:teuthology.orchestra.run.trial131.stdout:Total download size: 129 M 2026-02-20T12:07:00.375 INFO:teuthology.orchestra.run.trial131.stdout:Installed size: 963 M 2026-02-20T12:07:00.375 INFO:teuthology.orchestra.run.trial131.stdout:Downloading Packages: 2026-02-20T12:07:01.224 INFO:teuthology.orchestra.run.trial131.stdout:(1/12): libcephsqlite-20.3.0-5312.g26bc9765.el9 551 kB/s | 153 kB 00:00 2026-02-20T12:07:01.702 INFO:teuthology.orchestra.run.trial131.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 48 kB/s | 23 kB 00:00 2026-02-20T12:07:02.017 INFO:teuthology.orchestra.run.trial131.stdout:(3/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 299 kB/s | 94 kB 00:00 2026-02-20T12:07:02.725 INFO:teuthology.orchestra.run.trial131.stdout:(4/12): libasan8-15.2.1-7.el9.x86_64.rpm 741 kB/s | 525 kB 00:00 2026-02-20T12:07:03.041 INFO:teuthology.orchestra.run.trial131.stdout:(5/12): libubsan-11.5.0-14.el9.x86_64.rpm 558 kB/s | 176 kB 00:00 2026-02-20T12:07:03.138 INFO:teuthology.orchestra.run.trial131.stdout:(6/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 980 kB/s | 2.1 MB 00:02 2026-02-20T12:07:03.201 INFO:teuthology.orchestra.run.trial131.stdout:(7/12): libxslt-1.1.34-12.el9.x86_64.rpm 1.4 MB/s | 233 kB 00:00 2026-02-20T12:07:03.374 INFO:teuthology.orchestra.run.trial131.stdout:(8/12): protobuf-3.14.0-17.el9.x86_64.rpm 4.2 MB/s | 1.0 MB 00:00 2026-02-20T12:07:03.437 INFO:teuthology.orchestra.run.trial131.stdout:(9/12): socat-1.7.4.1-8.el9.x86_64.rpm 1.3 MB/s | 303 kB 00:00 2026-02-20T12:07:03.520 INFO:teuthology.orchestra.run.trial131.stdout:(10/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 1.4 MB/s | 122 kB 00:00 2026-02-20T12:07:03.533 INFO:teuthology.orchestra.run.trial131.stdout:(11/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 401 kB/s | 64 kB 00:00 2026-02-20T12:07:06.152 INFO:teuthology.orchestra.run.trial131.stdout:(12/12): ceph-test-20.3.0-5312.g26bc9765.el9.x8 24 MB/s | 125 MB 00:05 2026-02-20T12:07:06.154 INFO:teuthology.orchestra.run.trial131.stdout:-------------------------------------------------------------------------------- 2026-02-20T12:07:06.155 INFO:teuthology.orchestra.run.trial131.stdout:Total 22 MB/s | 129 MB 00:05 2026-02-20T12:07:06.235 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction check 2026-02-20T12:07:06.245 INFO:teuthology.orchestra.run.trial131.stdout:Transaction check succeeded. 2026-02-20T12:07:06.245 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction test 2026-02-20T12:07:06.667 INFO:teuthology.orchestra.run.trial131.stdout:Transaction test succeeded. 2026-02-20T12:07:06.667 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction 2026-02-20T12:07:07.116 INFO:teuthology.orchestra.run.trial131.stdout: Preparing : 1/1 2026-02-20T12:07:07.125 INFO:teuthology.orchestra.run.trial131.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-20T12:07:07.133 INFO:teuthology.orchestra.run.trial131.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-20T12:07:07.138 INFO:teuthology.orchestra.run.trial131.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-20T12:07:07.142 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-20T12:07:07.145 INFO:teuthology.orchestra.run.trial131.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-20T12:07:07.149 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-20T12:07:07.150 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-20T12:07:07.169 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-20T12:07:07.171 INFO:teuthology.orchestra.run.trial131.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-20T12:07:07.181 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-20T12:07:07.189 INFO:teuthology.orchestra.run.trial131.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-20T12:07:07.190 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libcephsqlite-2:20.3.0-5312.g26bc9765.el9.x86_64 11/12 2026-02-20T12:07:13.983 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5312.g26bc9765.el9.x86_64 11/12 2026-02-20T12:07:14.001 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-test-2:20.3.0-5312.g26bc9765.el9.x86_64 12/12 2026-02-20T12:07:14.792 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-test-2:20.3.0-5312.g26bc9765.el9.x86_64 12/12 2026-02-20T12:07:14.792 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-test-2:20.3.0-5312.g26bc9765.el9.x86_64 1/12 2026-02-20T12:07:14.792 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libcephsqlite-2:20.3.0-5312.g26bc9765.el9.x86_64 2/12 2026-02-20T12:07:14.792 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-20T12:07:14.792 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-20T12:07:14.793 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-20T12:07:14.793 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-20T12:07:14.793 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-20T12:07:14.793 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-20T12:07:14.793 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-20T12:07:14.793 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-20T12:07:14.793 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-20T12:07:14.846 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-20T12:07:14.846 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:14.846 INFO:teuthology.orchestra.run.trial131.stdout:Installed: 2026-02-20T12:07:14.846 INFO:teuthology.orchestra.run.trial131.stdout: ceph-test-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:07:14.847 INFO:teuthology.orchestra.run.trial131.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-20T12:07:14.847 INFO:teuthology.orchestra.run.trial131.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-20T12:07:14.847 INFO:teuthology.orchestra.run.trial131.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-20T12:07:14.847 INFO:teuthology.orchestra.run.trial131.stdout: libcephsqlite-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:07:14.847 INFO:teuthology.orchestra.run.trial131.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-20T12:07:14.847 INFO:teuthology.orchestra.run.trial131.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-20T12:07:14.847 INFO:teuthology.orchestra.run.trial131.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-20T12:07:14.847 INFO:teuthology.orchestra.run.trial131.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-20T12:07:14.847 INFO:teuthology.orchestra.run.trial131.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-20T12:07:14.847 INFO:teuthology.orchestra.run.trial131.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-20T12:07:14.847 INFO:teuthology.orchestra.run.trial131.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-20T12:07:14.847 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:14.848 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:07:14.889 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph 2026-02-20T12:07:15.191 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:29 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:07:15.278 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:07:15.280 INFO:teuthology.orchestra.run.trial131.stdout:======================================================================================= 2026-02-20T12:07:15.280 INFO:teuthology.orchestra.run.trial131.stdout: Package Arch Version Repository Size 2026-02-20T12:07:15.280 INFO:teuthology.orchestra.run.trial131.stdout:======================================================================================= 2026-02-20T12:07:15.280 INFO:teuthology.orchestra.run.trial131.stdout:Installing: 2026-02-20T12:07:15.280 INFO:teuthology.orchestra.run.trial131.stdout: ceph x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 6.5 k 2026-02-20T12:07:15.280 INFO:teuthology.orchestra.run.trial131.stdout:Installing dependencies: 2026-02-20T12:07:15.280 INFO:teuthology.orchestra.run.trial131.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5312.g26bc9765.el9 ceph-noarch 57 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mds x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 2.0 M 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mgr x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 937 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5312.g26bc9765.el9 ceph-noarch 297 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mon x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 4.9 M 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: ceph-osd x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 163 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: ceph-osd-classic x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 15 M 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: ceph-osd-crimson x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 85 M 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5312.g26bc9765.el9 ceph-noarch 17 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: cephadm noarch 2:20.3.0-5312.g26bc9765.el9 ceph-noarch 1.0 M 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-20T12:07:15.281 INFO:teuthology.orchestra.run.trial131.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-20T12:07:15.282 INFO:teuthology.orchestra.run.trial131.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout:Installing weak dependencies: 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5312.g26bc9765.el9 ceph-noarch 184 k 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5312.g26bc9765.el9 ceph-noarch 51 M 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5312.g26bc9765.el9 ceph-noarch 7.4 M 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5312.g26bc9765.el9 ceph-noarch 22 k 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mgr-rook noarch 2:20.3.0-5312.g26bc9765.el9 ceph-noarch 50 k 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout: ceph-volume noarch 2:20.3.0-5312.g26bc9765.el9 ceph-noarch 299 k 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout:Transaction Summary 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout:======================================================================================= 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout:Install 96 Packages 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout:Total download size: 223 M 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout:Installed size: 1.5 G 2026-02-20T12:07:15.283 INFO:teuthology.orchestra.run.trial131.stdout:Downloading Packages: 2026-02-20T12:07:16.051 INFO:teuthology.orchestra.run.trial131.stdout:(1/96): ceph-20.3.0-5312.g26bc9765.el9.x86_64.r 28 kB/s | 6.5 kB 00:00 2026-02-20T12:07:16.497 INFO:teuthology.orchestra.run.trial131.stdout:(2/96): ceph-mgr-20.3.0-5312.g26bc9765.el9.x86_ 1.3 MB/s | 937 kB 00:00 2026-02-20T12:07:16.553 INFO:teuthology.orchestra.run.trial131.stdout:(3/96): ceph-mds-20.3.0-5312.g26bc9765.el9.x86_ 2.7 MB/s | 2.0 MB 00:00 2026-02-20T12:07:16.558 INFO:teuthology.orchestra.run.trial131.stdout:(4/96): ceph-osd-20.3.0-5312.g26bc9765.el9.x86_ 2.6 MB/s | 163 kB 00:00 2026-02-20T12:07:18.182 INFO:teuthology.orchestra.run.trial131.stdout:(5/96): ceph-mon-20.3.0-5312.g26bc9765.el9.x86_ 2.3 MB/s | 4.9 MB 00:02 2026-02-20T12:07:18.212 INFO:teuthology.orchestra.run.trial131.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5312.g26 1.9 MB/s | 57 kB 00:00 2026-02-20T12:07:18.280 INFO:teuthology.orchestra.run.trial131.stdout:(7/96): ceph-osd-classic-20.3.0-5312.g26bc9765. 8.5 MB/s | 15 MB 00:01 2026-02-20T12:07:18.281 INFO:teuthology.orchestra.run.trial131.stdout:(8/96): ceph-mgr-cephadm-20.3.0-5312.g26bc9765. 2.6 MB/s | 184 kB 00:00 2026-02-20T12:07:20.092 INFO:teuthology.orchestra.run.trial131.stdout:(9/96): ceph-mgr-diskprediction-local-20.3.0-53 4.1 MB/s | 7.4 MB 00:01 2026-02-20T12:07:20.138 INFO:teuthology.orchestra.run.trial131.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5312.g26bc97 489 kB/s | 22 kB 00:00 2026-02-20T12:07:20.239 INFO:teuthology.orchestra.run.trial131.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5312.g26b 2.9 MB/s | 297 kB 00:00 2026-02-20T12:07:20.288 INFO:teuthology.orchestra.run.trial131.stdout:(12/96): ceph-mgr-rook-20.3.0-5312.g26bc9765.el 1.0 MB/s | 50 kB 00:00 2026-02-20T12:07:20.338 INFO:teuthology.orchestra.run.trial131.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5312.g26 346 kB/s | 17 kB 00:00 2026-02-20T12:07:20.446 INFO:teuthology.orchestra.run.trial131.stdout:(14/96): ceph-volume-20.3.0-5312.g26bc9765.el9. 2.7 MB/s | 299 kB 00:00 2026-02-20T12:07:20.789 INFO:teuthology.orchestra.run.trial131.stdout:(15/96): cephadm-20.3.0-5312.g26bc9765.el9.noar 3.0 MB/s | 1.0 MB 00:00 2026-02-20T12:07:21.615 INFO:teuthology.orchestra.run.trial131.stdout:(16/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 425 kB/s | 351 kB 00:00 2026-02-20T12:07:21.714 INFO:teuthology.orchestra.run.trial131.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 411 kB/s | 40 kB 00:00 2026-02-20T12:07:21.890 INFO:teuthology.orchestra.run.trial131.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 408 kB/s | 72 kB 00:00 2026-02-20T12:07:22.846 INFO:teuthology.orchestra.run.trial131.stdout:(19/96): libgfortran-11.5.0-14.el9.x86_64.rpm 831 kB/s | 794 kB 00:00 2026-02-20T12:07:23.101 INFO:teuthology.orchestra.run.trial131.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 724 kB/s | 184 kB 00:00 2026-02-20T12:07:23.436 INFO:teuthology.orchestra.run.trial131.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 757 kB/s | 253 kB 00:00 2026-02-20T12:07:24.847 INFO:teuthology.orchestra.run.trial131.stdout:(22/96): ceph-mgr-dashboard-20.3.0-5312.g26bc97 7.7 MB/s | 51 MB 00:06 2026-02-20T12:07:24.940 INFO:teuthology.orchestra.run.trial131.stdout:(23/96): python3-cryptography-36.0.1-5.el9.x86_ 848 kB/s | 1.2 MB 00:01 2026-02-20T12:07:25.116 INFO:teuthology.orchestra.run.trial131.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 764 kB/s | 135 kB 00:00 2026-02-20T12:07:25.371 INFO:teuthology.orchestra.run.trial131.stdout:(25/96): python3-pyparsing-2.4.7-9.el9.noarch.r 591 kB/s | 150 kB 00:00 2026-02-20T12:07:25.423 INFO:teuthology.orchestra.run.trial131.stdout:(26/96): python3-ply-3.11-14.el9.noarch.rpm 185 kB/s | 106 kB 00:00 2026-02-20T12:07:25.548 INFO:teuthology.orchestra.run.trial131.stdout:(27/96): python3-requests-2.25.1-10.el9.noarch. 715 kB/s | 126 kB 00:00 2026-02-20T12:07:25.758 INFO:teuthology.orchestra.run.trial131.stdout:(28/96): python3-urllib3-1.26.5-7.el9.noarch.rp 650 kB/s | 218 kB 00:00 2026-02-20T12:07:25.802 INFO:teuthology.orchestra.run.trial131.stdout:(29/96): unzip-6.0-59.el9.x86_64.rpm 715 kB/s | 182 kB 00:00 2026-02-20T12:07:25.845 INFO:teuthology.orchestra.run.trial131.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 690 kB/s | 30 kB 00:00 2026-02-20T12:07:25.977 INFO:teuthology.orchestra.run.trial131.stdout:(31/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 23 MB/s | 3.0 MB 00:00 2026-02-20T12:07:25.998 INFO:teuthology.orchestra.run.trial131.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 682 kB/s | 15 kB 00:00 2026-02-20T12:07:26.023 INFO:teuthology.orchestra.run.trial131.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 10 MB/s | 246 kB 00:00 2026-02-20T12:07:26.047 INFO:teuthology.orchestra.run.trial131.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 7.8 MB/s | 188 kB 00:00 2026-02-20T12:07:26.069 INFO:teuthology.orchestra.run.trial131.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 1.9 MB/s | 42 kB 00:00 2026-02-20T12:07:26.093 INFO:teuthology.orchestra.run.trial131.stdout:(36/96): zip-3.0-35.el9.x86_64.rpm 794 kB/s | 266 kB 00:00 2026-02-20T12:07:26.120 INFO:teuthology.orchestra.run.trial131.stdout:(37/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 102 MB/s | 5.3 MB 00:00 2026-02-20T12:07:26.145 INFO:teuthology.orchestra.run.trial131.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 9.6 MB/s | 244 kB 00:00 2026-02-20T12:07:26.171 INFO:teuthology.orchestra.run.trial131.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 9.8 MB/s | 249 kB 00:00 2026-02-20T12:07:26.193 INFO:teuthology.orchestra.run.trial131.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 2.1 MB/s | 48 kB 00:00 2026-02-20T12:07:26.217 INFO:teuthology.orchestra.run.trial131.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 7.4 MB/s | 177 kB 00:00 2026-02-20T12:07:26.286 INFO:teuthology.orchestra.run.trial131.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 18 MB/s | 1.2 MB 00:00 2026-02-20T12:07:26.312 INFO:teuthology.orchestra.run.trial131.stdout:(43/96): python3-babel-2.9.1-2.el9.noarch.rpm 27 MB/s | 6.0 MB 00:00 2026-02-20T12:07:26.313 INFO:teuthology.orchestra.run.trial131.stdout:(44/96): python3-markupsafe-1.1.1-12.el9.x86_64 1.3 MB/s | 35 kB 00:00 2026-02-20T12:07:26.337 INFO:teuthology.orchestra.run.trial131.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 18 MB/s | 442 kB 00:00 2026-02-20T12:07:26.361 INFO:teuthology.orchestra.run.trial131.stdout:(46/96): python3-packaging-20.9-5.el9.noarch.rp 3.3 MB/s | 77 kB 00:00 2026-02-20T12:07:26.384 INFO:teuthology.orchestra.run.trial131.stdout:(47/96): python3-protobuf-3.14.0-17.el9.noarch. 11 MB/s | 267 kB 00:00 2026-02-20T12:07:26.409 INFO:teuthology.orchestra.run.trial131.stdout:(48/96): python3-pyasn1-0.4.8-6.el9.noarch.rpm 6.4 MB/s | 159 kB 00:00 2026-02-20T12:07:26.433 INFO:teuthology.orchestra.run.trial131.stdout:(49/96): python3-pyasn1-modules-0.4.8-6.el9.noa 11 MB/s | 279 kB 00:00 2026-02-20T12:07:26.449 INFO:teuthology.orchestra.run.trial131.stdout:(50/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 45 MB/s | 6.1 MB 00:00 2026-02-20T12:07:26.456 INFO:teuthology.orchestra.run.trial131.stdout:(51/96): python3-requests-oauthlib-1.3.0-12.el9 2.3 MB/s | 54 kB 00:00 2026-02-20T12:07:26.478 INFO:teuthology.orchestra.run.trial131.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 1.8 MB/s | 42 kB 00:00 2026-02-20T12:07:26.501 INFO:teuthology.orchestra.run.trial131.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 8.2 MB/s | 189 kB 00:00 2026-02-20T12:07:26.524 INFO:teuthology.orchestra.run.trial131.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 3.9 MB/s | 90 kB 00:00 2026-02-20T12:07:26.850 INFO:teuthology.orchestra.run.trial131.stdout:(55/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 48 MB/s | 19 MB 00:00 2026-02-20T12:07:27.086 INFO:teuthology.orchestra.run.trial131.stdout:(56/96): lua-devel-5.4.4-4.el9.x86_64.rpm 40 kB/s | 22 kB 00:00 2026-02-20T12:07:27.428 INFO:teuthology.orchestra.run.trial131.stdout:(57/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.6 MB/s | 551 kB 00:00 2026-02-20T12:07:27.458 INFO:teuthology.orchestra.run.trial131.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 636 kB/s | 19 kB 00:00 2026-02-20T12:07:27.494 INFO:teuthology.orchestra.run.trial131.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 4.2 MB/s | 151 kB 00:00 2026-02-20T12:07:27.544 INFO:teuthology.orchestra.run.trial131.stdout:(60/96): python3-asyncssh-2.13.2-5.el9.noarch.r 11 MB/s | 548 kB 00:00 2026-02-20T12:07:27.575 INFO:teuthology.orchestra.run.trial131.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 966 kB/s | 29 kB 00:00 2026-02-20T12:07:27.606 INFO:teuthology.orchestra.run.trial131.stdout:(62/96): python3-backports-tarfile-1.2.0-1.el9. 1.9 MB/s | 60 kB 00:00 2026-02-20T12:07:27.637 INFO:teuthology.orchestra.run.trial131.stdout:(63/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.4 MB/s | 43 kB 00:00 2026-02-20T12:07:27.667 INFO:teuthology.orchestra.run.trial131.stdout:(64/96): python3-cachetools-4.2.4-1.el9.noarch. 1.0 MB/s | 32 kB 00:00 2026-02-20T12:07:27.697 INFO:teuthology.orchestra.run.trial131.stdout:(65/96): python3-certifi-2023.05.07-4.el9.noarc 469 kB/s | 14 kB 00:00 2026-02-20T12:07:27.732 INFO:teuthology.orchestra.run.trial131.stdout:(66/96): python3-cheroot-10.0.1-4.el9.noarch.rp 4.9 MB/s | 173 kB 00:00 2026-02-20T12:07:27.758 INFO:teuthology.orchestra.run.trial131.stdout:(67/96): protobuf-compiler-3.14.0-17.el9.x86_64 950 kB/s | 862 kB 00:00 2026-02-20T12:07:27.773 INFO:teuthology.orchestra.run.trial131.stdout:(68/96): python3-cherrypy-18.6.1-2.el9.noarch.r 8.6 MB/s | 358 kB 00:00 2026-02-20T12:07:27.865 INFO:teuthology.orchestra.run.trial131.stdout:(69/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 22 MB/s | 2.0 MB 00:00 2026-02-20T12:07:27.897 INFO:teuthology.orchestra.run.trial131.stdout:(70/96): python3-grpcio-tools-1.46.7-10.el9.x86 4.5 MB/s | 144 kB 00:00 2026-02-20T12:07:27.928 INFO:teuthology.orchestra.run.trial131.stdout:(71/96): python3-influxdb-5.3.1-1.el9.noarch.rp 4.4 MB/s | 139 kB 00:00 2026-02-20T12:07:27.959 INFO:teuthology.orchestra.run.trial131.stdout:(72/96): python3-isodate-0.6.1-3.el9.noarch.rpm 1.8 MB/s | 56 kB 00:00 2026-02-20T12:07:27.979 INFO:teuthology.orchestra.run.trial131.stdout:(73/96): python3-google-auth-2.45.0-1.el9.noarc 1.1 MB/s | 254 kB 00:00 2026-02-20T12:07:27.989 INFO:teuthology.orchestra.run.trial131.stdout:(74/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 354 kB/s | 11 kB 00:00 2026-02-20T12:07:28.010 INFO:teuthology.orchestra.run.trial131.stdout:(75/96): python3-jaraco-classes-3.2.1-5.el9.noa 578 kB/s | 18 kB 00:00 2026-02-20T12:07:28.019 INFO:teuthology.orchestra.run.trial131.stdout:(76/96): python3-jaraco-collections-3.0.0-8.el9 764 kB/s | 23 kB 00:00 2026-02-20T12:07:28.040 INFO:teuthology.orchestra.run.trial131.stdout:(77/96): python3-jaraco-context-6.0.1-3.el9.noa 641 kB/s | 20 kB 00:00 2026-02-20T12:07:28.049 INFO:teuthology.orchestra.run.trial131.stdout:(78/96): python3-jaraco-functools-3.5.0-2.el9.n 645 kB/s | 19 kB 00:00 2026-02-20T12:07:28.072 INFO:teuthology.orchestra.run.trial131.stdout:(79/96): python3-jaraco-text-4.0.0-2.el9.noarch 839 kB/s | 26 kB 00:00 2026-02-20T12:07:28.091 INFO:teuthology.orchestra.run.trial131.stdout:(80/96): python3-kubernetes-26.1.0-3.el9.noarch 25 MB/s | 1.0 MB 00:00 2026-02-20T12:07:28.108 INFO:teuthology.orchestra.run.trial131.stdout:(81/96): python3-more-itertools-8.12.0-2.el9.no 2.1 MB/s | 79 kB 00:00 2026-02-20T12:07:28.122 INFO:teuthology.orchestra.run.trial131.stdout:(82/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 2.7 MB/s | 86 kB 00:00 2026-02-20T12:07:28.142 INFO:teuthology.orchestra.run.trial131.stdout:(83/96): python3-natsort-7.1.1-5.el9.noarch.rpm 1.6 MB/s | 58 kB 00:00 2026-02-20T12:07:28.152 INFO:teuthology.orchestra.run.trial131.stdout:(84/96): python3-portend-3.1.0-2.el9.noarch.rpm 543 kB/s | 16 kB 00:00 2026-02-20T12:07:28.179 INFO:teuthology.orchestra.run.trial131.stdout:(85/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 2.4 MB/s | 90 kB 00:00 2026-02-20T12:07:28.183 INFO:teuthology.orchestra.run.trial131.stdout:(86/96): python3-repoze-lru-0.7-16.el9.noarch.r 1.0 MB/s | 31 kB 00:00 2026-02-20T12:07:28.213 INFO:teuthology.orchestra.run.trial131.stdout:(87/96): python3-rsa-4.9-2.el9.noarch.rpm 1.9 MB/s | 59 kB 00:00 2026-02-20T12:07:28.225 INFO:teuthology.orchestra.run.trial131.stdout:(88/96): python3-routes-2.5.1-5.el9.noarch.rpm 4.1 MB/s | 188 kB 00:00 2026-02-20T12:07:28.244 INFO:teuthology.orchestra.run.trial131.stdout:(89/96): python3-saml-1.16.0-1.el9.noarch.rpm 3.9 MB/s | 125 kB 00:00 2026-02-20T12:07:28.257 INFO:teuthology.orchestra.run.trial131.stdout:(90/96): python3-tempora-5.0.0-2.el9.noarch.rpm 1.1 MB/s | 36 kB 00:00 2026-02-20T12:07:28.275 INFO:teuthology.orchestra.run.trial131.stdout:(91/96): python3-typing-extensions-4.15.0-1.el9 2.7 MB/s | 86 kB 00:00 2026-02-20T12:07:28.294 INFO:teuthology.orchestra.run.trial131.stdout:(92/96): python3-websocket-client-1.2.3-2.el9.n 2.4 MB/s | 90 kB 00:00 2026-02-20T12:07:28.306 INFO:teuthology.orchestra.run.trial131.stdout:(93/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 1.6 MB/s | 48 kB 00:00 2026-02-20T12:07:28.324 INFO:teuthology.orchestra.run.trial131.stdout:(94/96): python3-xmltodict-0.12.0-15.el9.noarch 723 kB/s | 22 kB 00:00 2026-02-20T12:07:28.348 INFO:teuthology.orchestra.run.trial131.stdout:(95/96): python3-zc-lockfile-2.0-10.el9.noarch. 471 kB/s | 20 kB 00:00 2026-02-20T12:07:35.954 INFO:teuthology.orchestra.run.trial131.stdout:(96/96): ceph-osd-crimson-20.3.0-5312.g26bc9765 4.4 MB/s | 85 MB 00:19 2026-02-20T12:07:35.957 INFO:teuthology.orchestra.run.trial131.stdout:-------------------------------------------------------------------------------- 2026-02-20T12:07:35.957 INFO:teuthology.orchestra.run.trial131.stdout:Total 11 MB/s | 223 MB 00:20 2026-02-20T12:07:36.366 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction check 2026-02-20T12:07:36.406 INFO:teuthology.orchestra.run.trial131.stdout:Transaction check succeeded. 2026-02-20T12:07:36.406 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction test 2026-02-20T12:07:37.199 INFO:teuthology.orchestra.run.trial131.stdout:Transaction test succeeded. 2026-02-20T12:07:37.199 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction 2026-02-20T12:07:38.032 INFO:teuthology.orchestra.run.trial131.stdout: Preparing : 1/1 2026-02-20T12:07:38.034 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-20T12:07:38.038 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-20T12:07:38.059 INFO:teuthology.orchestra.run.trial131.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-20T12:07:38.093 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 4/96 2026-02-20T12:07:38.099 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-20T12:07:38.101 INFO:teuthology.orchestra.run.trial131.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-20T12:07:38.106 INFO:teuthology.orchestra.run.trial131.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-20T12:07:38.129 INFO:teuthology.orchestra.run.trial131.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-20T12:07:38.142 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-20T12:07:38.145 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-20T12:07:38.151 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-20T12:07:38.152 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-20T12:07:38.163 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-20T12:07:38.169 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-20T12:07:38.193 INFO:teuthology.orchestra.run.trial131.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-20T12:07:38.197 INFO:teuthology.orchestra.run.trial131.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-20T12:07:38.247 INFO:teuthology.orchestra.run.trial131.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-20T12:07:38.259 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 18/96 2026-02-20T12:07:38.262 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-20T12:07:38.266 INFO:teuthology.orchestra.run.trial131.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-20T12:07:38.271 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-20T12:07:38.276 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-20T12:07:38.279 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-20T12:07:38.294 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-20T12:07:38.298 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-20T12:07:38.303 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-20T12:07:38.314 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-20T12:07:38.323 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-20T12:07:38.330 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-20T12:07:38.349 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-20T12:07:38.360 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-20T12:07:38.375 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-20T12:07:38.380 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-20T12:07:38.388 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-20T12:07:38.428 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-20T12:07:38.751 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-20T12:07:38.761 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-20T12:07:38.765 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-20T12:07:38.770 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-20T12:07:38.772 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-20T12:07:38.799 INFO:teuthology.orchestra.run.trial131.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-20T12:07:38.843 INFO:teuthology.orchestra.run.trial131.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-20T12:07:38.852 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-20T12:07:38.857 INFO:teuthology.orchestra.run.trial131.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-20T12:07:38.860 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-20T12:07:38.866 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-20T12:07:38.870 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-20T12:07:38.877 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-20T12:07:38.880 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-20T12:07:38.908 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-20T12:07:38.917 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-20T12:07:38.923 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-20T12:07:38.930 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-20T12:07:38.964 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-20T12:07:39.188 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-20T12:07:39.212 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-20T12:07:39.216 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-20T12:07:39.274 INFO:teuthology.orchestra.run.trial131.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-20T12:07:39.276 INFO:teuthology.orchestra.run.trial131.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-20T12:07:39.297 INFO:teuthology.orchestra.run.trial131.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-20T12:07:39.622 INFO:teuthology.orchestra.run.trial131.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-20T12:07:39.696 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-20T12:07:40.366 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-20T12:07:40.383 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-20T12:07:40.393 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-20T12:07:40.400 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-20T12:07:40.414 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-20T12:07:40.430 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-20T12:07:40.505 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-20T12:07:40.514 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-20T12:07:40.537 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-20T12:07:40.567 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-20T12:07:40.617 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-20T12:07:40.624 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-20T12:07:40.627 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-20T12:07:40.629 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-20T12:07:40.647 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-20T12:07:40.647 INFO:teuthology.orchestra.run.trial131.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-20T12:07:40.647 INFO:teuthology.orchestra.run.trial131.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-20T12:07:40.647 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:40.657 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-20T12:07:40.684 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-20T12:07:40.684 INFO:teuthology.orchestra.run.trial131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-20T12:07:40.684 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:40.719 INFO:teuthology.orchestra.run.trial131.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-20T12:07:41.309 INFO:teuthology.orchestra.run.trial131.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-20T12:07:46.591 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-osd-classic-2:20.3.0-5312.g26bc9765.el9.x86_6 80/96 2026-02-20T12:07:46.655 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-osd-crimson-2:20.3.0-5312.g26bc9765.el9.x86_6 81/96 2026-02-20T12:07:46.657 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-volume-2:20.3.0-5312.g26bc9765.el9.noarch 82/96 2026-02-20T12:07:46.672 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-volume-2:20.3.0-5312.g26bc9765.el9.noarch 82/96 2026-02-20T12:07:46.672 INFO:teuthology.orchestra.run.trial131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T12:07:46.672 INFO:teuthology.orchestra.run.trial131.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-20T12:07:46.673 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:46.684 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-osd-2:20.3.0-5312.g26bc9765.el9.x86_64 83/96 2026-02-20T12:07:46.709 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-osd-2:20.3.0-5312.g26bc9765.el9.x86_64 83/96 2026-02-20T12:07:46.709 INFO:teuthology.orchestra.run.trial131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T12:07:46.709 INFO:teuthology.orchestra.run.trial131.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-20T12:07:46.709 INFO:teuthology.orchestra.run.trial131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T12:07:46.709 INFO:teuthology.orchestra.run.trial131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T12:07:46.709 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:46.769 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: cephadm-2:20.3.0-5312.g26bc9765.el9.noarch 84/96 2026-02-20T12:07:46.771 INFO:teuthology.orchestra.run.trial131.stdout: Installing : cephadm-2:20.3.0-5312.g26bc9765.el9.noarch 84/96 2026-02-20T12:07:46.780 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5312.g26bc9765.el9 85/96 2026-02-20T12:07:46.804 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5312.g26bc9765.el 86/96 2026-02-20T12:07:46.805 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5312.g26bc9765.el9.noarc 87/96 2026-02-20T12:07:49.702 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5312.g26bc9765.el9.noarc 87/96 2026-02-20T12:07:49.709 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5312.g26bc9765.el9.noa 88/96 2026-02-20T12:07:50.168 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5312.g26bc9765.el9.noa 88/96 2026-02-20T12:07:50.169 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5312.g26bc9 89/96 2026-02-20T12:07:50.182 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5312.g26bc9 89/96 2026-02-20T12:07:50.183 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5312.g26bc9765.el9.noa 90/96 2026-02-20T12:07:50.244 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5312.g26bc9765.el9.noa 90/96 2026-02-20T12:07:50.284 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5312.g26bc9765.el9. 91/96 2026-02-20T12:07:50.285 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-mgr-2:20.3.0-5312.g26bc9765.el9.x86_64 92/96 2026-02-20T12:07:50.308 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5312.g26bc9765.el9.x86_64 92/96 2026-02-20T12:07:50.308 INFO:teuthology.orchestra.run.trial131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T12:07:50.309 INFO:teuthology.orchestra.run.trial131.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-20T12:07:50.309 INFO:teuthology.orchestra.run.trial131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T12:07:50.309 INFO:teuthology.orchestra.run.trial131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T12:07:50.309 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:50.321 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-mgr-rook-2:20.3.0-5312.g26bc9765.el9.noarch 93/96 2026-02-20T12:07:50.537 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5312.g26bc9765.el9.noarch 93/96 2026-02-20T12:07:50.539 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-mon-2:20.3.0-5312.g26bc9765.el9.x86_64 94/96 2026-02-20T12:07:50.560 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-mon-2:20.3.0-5312.g26bc9765.el9.x86_64 94/96 2026-02-20T12:07:50.561 INFO:teuthology.orchestra.run.trial131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T12:07:50.561 INFO:teuthology.orchestra.run.trial131.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-20T12:07:50.561 INFO:teuthology.orchestra.run.trial131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T12:07:50.561 INFO:teuthology.orchestra.run.trial131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T12:07:50.561 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:50.645 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-mds-2:20.3.0-5312.g26bc9765.el9.x86_64 95/96 2026-02-20T12:07:50.666 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-mds-2:20.3.0-5312.g26bc9765.el9.x86_64 95/96 2026-02-20T12:07:50.667 INFO:teuthology.orchestra.run.trial131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T12:07:50.667 INFO:teuthology.orchestra.run.trial131.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-20T12:07:50.667 INFO:teuthology.orchestra.run.trial131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T12:07:50.667 INFO:teuthology.orchestra.run.trial131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T12:07:50.667 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:50.668 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-2:20.3.0-5312.g26bc9765.el9.x86_64 96/96 2026-02-20T12:07:50.674 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5312.g26bc9765.el9.x86_6 96/96 2026-02-20T12:07:50.700 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5312.g26bc9765.el9.x86_6 96/96 2026-02-20T12:07:52.066 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-2:20.3.0-5312.g26bc9765.el9.x86_64 96/96 2026-02-20T12:07:52.067 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-2:20.3.0-5312.g26bc9765.el9.x86_64 1/96 2026-02-20T12:07:52.067 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-mds-2:20.3.0-5312.g26bc9765.el9.x86_64 2/96 2026-02-20T12:07:52.067 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-mgr-2:20.3.0-5312.g26bc9765.el9.x86_64 3/96 2026-02-20T12:07:52.067 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-mon-2:20.3.0-5312.g26bc9765.el9.x86_64 4/96 2026-02-20T12:07:52.067 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-osd-2:20.3.0-5312.g26bc9765.el9.x86_64 5/96 2026-02-20T12:07:52.067 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-osd-classic-2:20.3.0-5312.g26bc9765.el9.x86_6 6/96 2026-02-20T12:07:52.067 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5312.g26bc9765.el9.x86_6 7/96 2026-02-20T12:07:52.067 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5312.g26bc9765.el 8/96 2026-02-20T12:07:52.067 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5312.g26bc9765.el9.noarc 9/96 2026-02-20T12:07:52.068 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5312.g26bc9765.el9.noa 10/96 2026-02-20T12:07:52.068 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5312.g26bc9 11/96 2026-02-20T12:07:52.068 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5312.g26bc9765.el9.noa 12/96 2026-02-20T12:07:52.068 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5312.g26bc9765.el9. 13/96 2026-02-20T12:07:52.068 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5312.g26bc9765.el9.noarch 14/96 2026-02-20T12:07:52.068 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5312.g26bc9765.el9 15/96 2026-02-20T12:07:52.068 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-volume-2:20.3.0-5312.g26bc9765.el9.noarch 16/96 2026-02-20T12:07:52.069 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : cephadm-2:20.3.0-5312.g26bc9765.el9.noarch 17/96 2026-02-20T12:07:52.069 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-20T12:07:52.069 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-20T12:07:52.069 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-20T12:07:52.069 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-20T12:07:52.069 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-20T12:07:52.069 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-20T12:07:52.069 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-20T12:07:52.069 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-20T12:07:52.069 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-20T12:07:52.069 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-20T12:07:52.069 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-20T12:07:52.070 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-20T12:07:52.070 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-20T12:07:52.070 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-20T12:07:52.070 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-20T12:07:52.070 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-20T12:07:52.070 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-20T12:07:52.070 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-20T12:07:52.070 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-20T12:07:52.070 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-20T12:07:52.070 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-20T12:07:52.070 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-20T12:07:52.070 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-20T12:07:52.070 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-20T12:07:52.071 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-20T12:07:52.071 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-20T12:07:52.071 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-20T12:07:52.071 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-20T12:07:52.071 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-20T12:07:52.071 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-20T12:07:52.071 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-20T12:07:52.071 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-20T12:07:52.071 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 50/96 2026-02-20T12:07:52.071 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 51/96 2026-02-20T12:07:52.071 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-20T12:07:52.071 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-20T12:07:52.071 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-20T12:07:52.072 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-20T12:07:52.072 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-20T12:07:52.072 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-20T12:07:52.072 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-20T12:07:52.072 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-20T12:07:52.072 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-20T12:07:52.072 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-20T12:07:52.072 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-20T12:07:52.072 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-20T12:07:52.072 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-20T12:07:52.072 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-20T12:07:52.072 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-20T12:07:52.072 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-20T12:07:52.073 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-20T12:07:52.073 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-20T12:07:52.073 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-20T12:07:52.073 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-20T12:07:52.073 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-20T12:07:52.073 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-20T12:07:52.073 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-20T12:07:52.073 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-20T12:07:52.073 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-20T12:07:52.073 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-20T12:07:52.073 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-20T12:07:52.073 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-20T12:07:52.073 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-20T12:07:52.074 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-20T12:07:52.074 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-20T12:07:52.074 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-20T12:07:52.074 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-20T12:07:52.074 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-20T12:07:52.074 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-20T12:07:52.074 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-20T12:07:52.074 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-20T12:07:52.074 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-20T12:07:52.074 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-20T12:07:52.074 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-20T12:07:52.074 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-20T12:07:52.074 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-20T12:07:52.075 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-20T12:07:52.075 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-20T12:07:52.130 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-20T12:07:52.130 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:52.130 INFO:teuthology.orchestra.run.trial131.stdout:Installed: 2026-02-20T12:07:52.130 INFO:teuthology.orchestra.run.trial131.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-20T12:07:52.130 INFO:teuthology.orchestra.run.trial131.stdout: ceph-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:07:52.130 INFO:teuthology.orchestra.run.trial131.stdout: ceph-grafana-dashboards-2:20.3.0-5312.g26bc9765.el9.noarch 2026-02-20T12:07:52.131 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mds-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:07:52.131 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mgr-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:07:52.131 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mgr-cephadm-2:20.3.0-5312.g26bc9765.el9.noarch 2026-02-20T12:07:52.131 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mgr-dashboard-2:20.3.0-5312.g26bc9765.el9.noarch 2026-02-20T12:07:52.131 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5312.g26bc9765.el9.noarch 2026-02-20T12:07:52.131 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mgr-k8sevents-2:20.3.0-5312.g26bc9765.el9.noarch 2026-02-20T12:07:52.131 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mgr-modules-core-2:20.3.0-5312.g26bc9765.el9.noarch 2026-02-20T12:07:52.131 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mgr-rook-2:20.3.0-5312.g26bc9765.el9.noarch 2026-02-20T12:07:52.131 INFO:teuthology.orchestra.run.trial131.stdout: ceph-mon-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:07:52.131 INFO:teuthology.orchestra.run.trial131.stdout: ceph-osd-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:07:52.131 INFO:teuthology.orchestra.run.trial131.stdout: ceph-osd-classic-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:07:52.132 INFO:teuthology.orchestra.run.trial131.stdout: ceph-osd-crimson-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:07:52.132 INFO:teuthology.orchestra.run.trial131.stdout: ceph-prometheus-alerts-2:20.3.0-5312.g26bc9765.el9.noarch 2026-02-20T12:07:52.132 INFO:teuthology.orchestra.run.trial131.stdout: ceph-volume-2:20.3.0-5312.g26bc9765.el9.noarch 2026-02-20T12:07:52.132 INFO:teuthology.orchestra.run.trial131.stdout: cephadm-2:20.3.0-5312.g26bc9765.el9.noarch 2026-02-20T12:07:52.132 INFO:teuthology.orchestra.run.trial131.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-20T12:07:52.132 INFO:teuthology.orchestra.run.trial131.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-20T12:07:52.132 INFO:teuthology.orchestra.run.trial131.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-20T12:07:52.132 INFO:teuthology.orchestra.run.trial131.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-20T12:07:52.132 INFO:teuthology.orchestra.run.trial131.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-20T12:07:52.132 INFO:teuthology.orchestra.run.trial131.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-20T12:07:52.132 INFO:teuthology.orchestra.run.trial131.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-20T12:07:52.132 INFO:teuthology.orchestra.run.trial131.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-20T12:07:52.132 INFO:teuthology.orchestra.run.trial131.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-20T12:07:52.133 INFO:teuthology.orchestra.run.trial131.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T12:07:52.133 INFO:teuthology.orchestra.run.trial131.stdout: lua-5.4.4-4.el9.x86_64 2026-02-20T12:07:52.133 INFO:teuthology.orchestra.run.trial131.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-20T12:07:52.133 INFO:teuthology.orchestra.run.trial131.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-20T12:07:52.133 INFO:teuthology.orchestra.run.trial131.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-20T12:07:52.133 INFO:teuthology.orchestra.run.trial131.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-20T12:07:52.133 INFO:teuthology.orchestra.run.trial131.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-20T12:07:52.133 INFO:teuthology.orchestra.run.trial131.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-20T12:07:52.133 INFO:teuthology.orchestra.run.trial131.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-20T12:07:52.133 INFO:teuthology.orchestra.run.trial131.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-20T12:07:52.133 INFO:teuthology.orchestra.run.trial131.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-20T12:07:52.133 INFO:teuthology.orchestra.run.trial131.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-20T12:07:52.134 INFO:teuthology.orchestra.run.trial131.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-20T12:07:52.134 INFO:teuthology.orchestra.run.trial131.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-20T12:07:52.134 INFO:teuthology.orchestra.run.trial131.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-20T12:07:52.134 INFO:teuthology.orchestra.run.trial131.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-20T12:07:52.134 INFO:teuthology.orchestra.run.trial131.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-20T12:07:52.134 INFO:teuthology.orchestra.run.trial131.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-20T12:07:52.134 INFO:teuthology.orchestra.run.trial131.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-20T12:07:52.134 INFO:teuthology.orchestra.run.trial131.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-20T12:07:52.134 INFO:teuthology.orchestra.run.trial131.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-20T12:07:52.134 INFO:teuthology.orchestra.run.trial131.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-20T12:07:52.134 INFO:teuthology.orchestra.run.trial131.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-20T12:07:52.134 INFO:teuthology.orchestra.run.trial131.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-20T12:07:52.134 INFO:teuthology.orchestra.run.trial131.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-20T12:07:52.135 INFO:teuthology.orchestra.run.trial131.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-20T12:07:52.135 INFO:teuthology.orchestra.run.trial131.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-20T12:07:52.135 INFO:teuthology.orchestra.run.trial131.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-20T12:07:52.135 INFO:teuthology.orchestra.run.trial131.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-20T12:07:52.135 INFO:teuthology.orchestra.run.trial131.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-20T12:07:52.135 INFO:teuthology.orchestra.run.trial131.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-20T12:07:52.135 INFO:teuthology.orchestra.run.trial131.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-20T12:07:52.135 INFO:teuthology.orchestra.run.trial131.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-20T12:07:52.135 INFO:teuthology.orchestra.run.trial131.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T12:07:52.135 INFO:teuthology.orchestra.run.trial131.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-20T12:07:52.135 INFO:teuthology.orchestra.run.trial131.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-20T12:07:52.135 INFO:teuthology.orchestra.run.trial131.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-20T12:07:52.135 INFO:teuthology.orchestra.run.trial131.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-20T12:07:52.136 INFO:teuthology.orchestra.run.trial131.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-20T12:07:52.136 INFO:teuthology.orchestra.run.trial131.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-20T12:07:52.136 INFO:teuthology.orchestra.run.trial131.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-20T12:07:52.136 INFO:teuthology.orchestra.run.trial131.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-20T12:07:52.136 INFO:teuthology.orchestra.run.trial131.stdout: python3-ply-3.11-14.el9.noarch 2026-02-20T12:07:52.136 INFO:teuthology.orchestra.run.trial131.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-20T12:07:52.136 INFO:teuthology.orchestra.run.trial131.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-20T12:07:52.136 INFO:teuthology.orchestra.run.trial131.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-20T12:07:52.136 INFO:teuthology.orchestra.run.trial131.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-20T12:07:52.136 INFO:teuthology.orchestra.run.trial131.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-20T12:07:52.136 INFO:teuthology.orchestra.run.trial131.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-20T12:07:52.136 INFO:teuthology.orchestra.run.trial131.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-20T12:07:52.136 INFO:teuthology.orchestra.run.trial131.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-20T12:07:52.137 INFO:teuthology.orchestra.run.trial131.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-20T12:07:52.137 INFO:teuthology.orchestra.run.trial131.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-20T12:07:52.137 INFO:teuthology.orchestra.run.trial131.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-20T12:07:52.137 INFO:teuthology.orchestra.run.trial131.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-20T12:07:52.137 INFO:teuthology.orchestra.run.trial131.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-20T12:07:52.137 INFO:teuthology.orchestra.run.trial131.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-20T12:07:52.137 INFO:teuthology.orchestra.run.trial131.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-20T12:07:52.137 INFO:teuthology.orchestra.run.trial131.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-20T12:07:52.137 INFO:teuthology.orchestra.run.trial131.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-20T12:07:52.137 INFO:teuthology.orchestra.run.trial131.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-20T12:07:52.137 INFO:teuthology.orchestra.run.trial131.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-20T12:07:52.137 INFO:teuthology.orchestra.run.trial131.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-20T12:07:52.138 INFO:teuthology.orchestra.run.trial131.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-20T12:07:52.138 INFO:teuthology.orchestra.run.trial131.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-20T12:07:52.138 INFO:teuthology.orchestra.run.trial131.stdout: unzip-6.0-59.el9.x86_64 2026-02-20T12:07:52.138 INFO:teuthology.orchestra.run.trial131.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-20T12:07:52.138 INFO:teuthology.orchestra.run.trial131.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-20T12:07:52.138 INFO:teuthology.orchestra.run.trial131.stdout: zip-3.0-35.el9.x86_64 2026-02-20T12:07:52.138 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:52.138 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:07:52.199 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-base 2026-02-20T12:07:52.517 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:06 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:07:52.581 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-base-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:07:52.600 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:07:52.600 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:07:52.600 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:07:52.622 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install cephadm 2026-02-20T12:07:52.934 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:06 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:07:52.998 INFO:teuthology.orchestra.run.trial131.stdout:Package cephadm-2:20.3.0-5312.g26bc9765.el9.noarch is already installed. 2026-02-20T12:07:53.017 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:07:53.017 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:07:53.017 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:07:53.038 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-immutable-object-cache 2026-02-20T12:07:53.347 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:07 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:07:53.431 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:07:53.431 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:07:53.431 INFO:teuthology.orchestra.run.trial131.stdout: Package Arch Version Repo Size 2026-02-20T12:07:53.431 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:07:53.431 INFO:teuthology.orchestra.run.trial131.stdout:Installing: 2026-02-20T12:07:53.431 INFO:teuthology.orchestra.run.trial131.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 149 k 2026-02-20T12:07:53.431 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:53.432 INFO:teuthology.orchestra.run.trial131.stdout:Transaction Summary 2026-02-20T12:07:53.432 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:07:53.432 INFO:teuthology.orchestra.run.trial131.stdout:Install 1 Package 2026-02-20T12:07:53.432 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:53.432 INFO:teuthology.orchestra.run.trial131.stdout:Total download size: 149 k 2026-02-20T12:07:53.432 INFO:teuthology.orchestra.run.trial131.stdout:Installed size: 455 k 2026-02-20T12:07:53.432 INFO:teuthology.orchestra.run.trial131.stdout:Downloading Packages: 2026-02-20T12:07:53.762 INFO:teuthology.orchestra.run.trial131.stdout:ceph-immutable-object-cache-20.3.0-5312.g26bc97 452 kB/s | 149 kB 00:00 2026-02-20T12:07:53.763 INFO:teuthology.orchestra.run.trial131.stdout:-------------------------------------------------------------------------------- 2026-02-20T12:07:53.763 INFO:teuthology.orchestra.run.trial131.stdout:Total 450 kB/s | 149 kB 00:00 2026-02-20T12:07:53.763 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction check 2026-02-20T12:07:53.767 INFO:teuthology.orchestra.run.trial131.stdout:Transaction check succeeded. 2026-02-20T12:07:53.767 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction test 2026-02-20T12:07:53.798 INFO:teuthology.orchestra.run.trial131.stdout:Transaction test succeeded. 2026-02-20T12:07:53.799 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction 2026-02-20T12:07:53.857 INFO:teuthology.orchestra.run.trial131.stdout: Preparing : 1/1 2026-02-20T12:07:53.858 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5312.g26bc9765. 1/1 2026-02-20T12:07:53.881 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5312.g26bc9765. 1/1 2026-02-20T12:07:53.881 INFO:teuthology.orchestra.run.trial131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T12:07:53.881 INFO:teuthology.orchestra.run.trial131.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-20T12:07:53.881 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:54.203 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5312.g26bc9765. 1/1 2026-02-20T12:07:54.203 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:54.204 INFO:teuthology.orchestra.run.trial131.stdout:Installed: 2026-02-20T12:07:54.204 INFO:teuthology.orchestra.run.trial131.stdout: ceph-immutable-object-cache-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:07:54.204 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:54.204 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:07:54.231 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-mgr 2026-02-20T12:07:54.547 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:08 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:07:54.611 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-mgr-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:07:54.629 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:07:54.630 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:07:54.630 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:07:54.651 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-mgr-dashboard 2026-02-20T12:07:54.958 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:08 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:07:55.023 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-mgr-dashboard-2:20.3.0-5312.g26bc9765.el9.noarch is already installed. 2026-02-20T12:07:55.041 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:07:55.042 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:07:55.042 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:07:55.064 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-20T12:07:55.372 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:09 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:07:55.437 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5312.g26bc9765.el9.noarch is already installed. 2026-02-20T12:07:55.455 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:07:55.456 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:07:55.456 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:07:55.477 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-mgr-rook 2026-02-20T12:07:55.785 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:09 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:07:55.849 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-mgr-rook-2:20.3.0-5312.g26bc9765.el9.noarch is already installed. 2026-02-20T12:07:55.868 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:07:55.868 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:07:55.868 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:07:55.889 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-mgr-cephadm 2026-02-20T12:07:56.197 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:10 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:07:56.261 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-mgr-cephadm-2:20.3.0-5312.g26bc9765.el9.noarch is already installed. 2026-02-20T12:07:56.279 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:07:56.280 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:07:56.280 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:07:56.302 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-osd 2026-02-20T12:07:56.616 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:10 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:07:56.682 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-osd-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:07:56.701 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:07:56.701 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:07:56.702 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:07:56.723 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-osd-classic 2026-02-20T12:07:57.039 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:11 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:07:57.102 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-osd-classic-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:07:57.121 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:07:57.124 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:07:57.124 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:07:57.144 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-fuse 2026-02-20T12:07:57.454 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:11 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:07:57.537 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:07:57.538 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:07:57.538 INFO:teuthology.orchestra.run.trial131.stdout: Package Arch Version Repo Size 2026-02-20T12:07:57.538 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:07:57.539 INFO:teuthology.orchestra.run.trial131.stdout:Installing: 2026-02-20T12:07:57.539 INFO:teuthology.orchestra.run.trial131.stdout: ceph-fuse x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 914 k 2026-02-20T12:07:57.539 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:57.539 INFO:teuthology.orchestra.run.trial131.stdout:Transaction Summary 2026-02-20T12:07:57.539 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:07:57.539 INFO:teuthology.orchestra.run.trial131.stdout:Install 1 Package 2026-02-20T12:07:57.539 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:57.539 INFO:teuthology.orchestra.run.trial131.stdout:Total download size: 914 k 2026-02-20T12:07:57.539 INFO:teuthology.orchestra.run.trial131.stdout:Installed size: 2.9 M 2026-02-20T12:07:57.539 INFO:teuthology.orchestra.run.trial131.stdout:Downloading Packages: 2026-02-20T12:07:57.744 INFO:teuthology.orchestra.run.trial131.stdout:ceph-fuse-20.3.0-5312.g26bc9765.el9.x86_64.rpm 4.4 MB/s | 914 kB 00:00 2026-02-20T12:07:57.744 INFO:teuthology.orchestra.run.trial131.stdout:-------------------------------------------------------------------------------- 2026-02-20T12:07:57.745 INFO:teuthology.orchestra.run.trial131.stdout:Total 4.3 MB/s | 914 kB 00:00 2026-02-20T12:07:57.745 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction check 2026-02-20T12:07:57.749 INFO:teuthology.orchestra.run.trial131.stdout:Transaction check succeeded. 2026-02-20T12:07:57.749 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction test 2026-02-20T12:07:57.782 INFO:teuthology.orchestra.run.trial131.stdout:Transaction test succeeded. 2026-02-20T12:07:57.783 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction 2026-02-20T12:07:57.873 INFO:teuthology.orchestra.run.trial131.stdout: Preparing : 1/1 2026-02-20T12:07:57.890 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-fuse-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:07:58.123 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:07:58.182 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-fuse-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:07:58.183 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:58.183 INFO:teuthology.orchestra.run.trial131.stdout:Installed: 2026-02-20T12:07:58.183 INFO:teuthology.orchestra.run.trial131.stdout: ceph-fuse-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:07:58.183 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:58.183 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:07:58.210 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-volume 2026-02-20T12:07:58.515 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:12 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:07:58.580 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-volume-2:20.3.0-5312.g26bc9765.el9.noarch is already installed. 2026-02-20T12:07:58.598 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:07:58.599 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:07:58.599 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:07:58.619 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install librados-devel 2026-02-20T12:07:58.922 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:12 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:07:59.005 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:07:59.005 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:07:59.005 INFO:teuthology.orchestra.run.trial131.stdout: Package Arch Version Repo Size 2026-02-20T12:07:59.005 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:07:59.006 INFO:teuthology.orchestra.run.trial131.stdout:Installing: 2026-02-20T12:07:59.006 INFO:teuthology.orchestra.run.trial131.stdout: librados-devel x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 124 k 2026-02-20T12:07:59.006 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:59.006 INFO:teuthology.orchestra.run.trial131.stdout:Transaction Summary 2026-02-20T12:07:59.006 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:07:59.006 INFO:teuthology.orchestra.run.trial131.stdout:Install 1 Package 2026-02-20T12:07:59.006 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:59.006 INFO:teuthology.orchestra.run.trial131.stdout:Total download size: 124 k 2026-02-20T12:07:59.006 INFO:teuthology.orchestra.run.trial131.stdout:Installed size: 434 k 2026-02-20T12:07:59.006 INFO:teuthology.orchestra.run.trial131.stdout:Downloading Packages: 2026-02-20T12:07:59.092 INFO:teuthology.orchestra.run.trial131.stdout:librados-devel-20.3.0-5312.g26bc9765.el9.x86_64 1.4 MB/s | 124 kB 00:00 2026-02-20T12:07:59.092 INFO:teuthology.orchestra.run.trial131.stdout:-------------------------------------------------------------------------------- 2026-02-20T12:07:59.092 INFO:teuthology.orchestra.run.trial131.stdout:Total 1.4 MB/s | 124 kB 00:00 2026-02-20T12:07:59.092 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction check 2026-02-20T12:07:59.095 INFO:teuthology.orchestra.run.trial131.stdout:Transaction check succeeded. 2026-02-20T12:07:59.095 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction test 2026-02-20T12:07:59.128 INFO:teuthology.orchestra.run.trial131.stdout:Transaction test succeeded. 2026-02-20T12:07:59.128 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction 2026-02-20T12:07:59.182 INFO:teuthology.orchestra.run.trial131.stdout: Preparing : 1/1 2026-02-20T12:07:59.199 INFO:teuthology.orchestra.run.trial131.stdout: Installing : librados-devel-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:07:59.280 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: librados-devel-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:07:59.338 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : librados-devel-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:07:59.338 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:59.338 INFO:teuthology.orchestra.run.trial131.stdout:Installed: 2026-02-20T12:07:59.338 INFO:teuthology.orchestra.run.trial131.stdout: librados-devel-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:07:59.338 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:07:59.338 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:07:59.366 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install libcephfs2 2026-02-20T12:07:59.672 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:13 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:07:59.737 INFO:teuthology.orchestra.run.trial131.stdout:Package libcephfs2-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:07:59.755 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:07:59.756 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:07:59.756 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:07:59.777 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install libcephfs-devel 2026-02-20T12:08:00.095 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:14 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:08:00.177 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:08:00.178 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:08:00.178 INFO:teuthology.orchestra.run.trial131.stdout: Package Arch Version Repo Size 2026-02-20T12:08:00.178 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:08:00.179 INFO:teuthology.orchestra.run.trial131.stdout:Installing: 2026-02-20T12:08:00.179 INFO:teuthology.orchestra.run.trial131.stdout: libcephfs-devel x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 37 k 2026-02-20T12:08:00.179 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:00.179 INFO:teuthology.orchestra.run.trial131.stdout:Transaction Summary 2026-02-20T12:08:00.179 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:08:00.179 INFO:teuthology.orchestra.run.trial131.stdout:Install 1 Package 2026-02-20T12:08:00.179 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:00.179 INFO:teuthology.orchestra.run.trial131.stdout:Total download size: 37 k 2026-02-20T12:08:00.179 INFO:teuthology.orchestra.run.trial131.stdout:Installed size: 171 k 2026-02-20T12:08:00.179 INFO:teuthology.orchestra.run.trial131.stdout:Downloading Packages: 2026-02-20T12:08:00.249 INFO:teuthology.orchestra.run.trial131.stdout:libcephfs-devel-20.3.0-5312.g26bc9765.el9.x86_6 534 kB/s | 37 kB 00:00 2026-02-20T12:08:00.249 INFO:teuthology.orchestra.run.trial131.stdout:-------------------------------------------------------------------------------- 2026-02-20T12:08:00.250 INFO:teuthology.orchestra.run.trial131.stdout:Total 525 kB/s | 37 kB 00:00 2026-02-20T12:08:00.250 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction check 2026-02-20T12:08:00.252 INFO:teuthology.orchestra.run.trial131.stdout:Transaction check succeeded. 2026-02-20T12:08:00.252 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction test 2026-02-20T12:08:00.254 INFO:teuthology.orchestra.run.trial131.stdout:Transaction test succeeded. 2026-02-20T12:08:00.254 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction 2026-02-20T12:08:00.275 INFO:teuthology.orchestra.run.trial131.stdout: Preparing : 1/1 2026-02-20T12:08:00.291 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libcephfs-devel-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:08:00.348 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:08:00.407 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libcephfs-devel-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:08:00.407 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:00.407 INFO:teuthology.orchestra.run.trial131.stdout:Installed: 2026-02-20T12:08:00.407 INFO:teuthology.orchestra.run.trial131.stdout: libcephfs-devel-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:08:00.407 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:00.407 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:08:00.434 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install librados2 2026-02-20T12:08:00.747 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:14 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:08:00.811 INFO:teuthology.orchestra.run.trial131.stdout:Package librados2-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:08:00.830 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:08:00.831 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:08:00.831 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:08:00.853 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install librbd1 2026-02-20T12:08:01.163 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:15 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:08:01.228 INFO:teuthology.orchestra.run.trial131.stdout:Package librbd1-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:08:01.247 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:08:01.247 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:08:01.248 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:08:01.271 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install python3-rados 2026-02-20T12:08:01.578 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:15 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:08:01.643 INFO:teuthology.orchestra.run.trial131.stdout:Package python3-rados-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:08:01.662 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:08:01.662 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:08:01.662 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:08:01.683 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install python3-rgw 2026-02-20T12:08:01.994 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:15 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:08:02.059 INFO:teuthology.orchestra.run.trial131.stdout:Package python3-rgw-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:08:02.078 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:08:02.079 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:08:02.079 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:08:02.099 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install python3-cephfs 2026-02-20T12:08:02.412 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:16 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:08:02.476 INFO:teuthology.orchestra.run.trial131.stdout:Package python3-cephfs-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:08:02.494 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:08:02.495 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:08:02.495 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:08:02.517 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install python3-rbd 2026-02-20T12:08:02.826 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:16 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:08:02.892 INFO:teuthology.orchestra.run.trial131.stdout:Package python3-rbd-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:08:02.911 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:08:02.911 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:08:02.911 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:08:02.933 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install rbd-fuse 2026-02-20T12:08:03.243 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:17 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:08:03.326 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:08:03.326 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:08:03.326 INFO:teuthology.orchestra.run.trial131.stdout: Package Architecture Version Repository Size 2026-02-20T12:08:03.326 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:08:03.326 INFO:teuthology.orchestra.run.trial131.stdout:Installing: 2026-02-20T12:08:03.326 INFO:teuthology.orchestra.run.trial131.stdout: rbd-fuse x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 87 k 2026-02-20T12:08:03.326 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:03.326 INFO:teuthology.orchestra.run.trial131.stdout:Transaction Summary 2026-02-20T12:08:03.326 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:08:03.326 INFO:teuthology.orchestra.run.trial131.stdout:Install 1 Package 2026-02-20T12:08:03.326 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:03.327 INFO:teuthology.orchestra.run.trial131.stdout:Total download size: 87 k 2026-02-20T12:08:03.327 INFO:teuthology.orchestra.run.trial131.stdout:Installed size: 236 k 2026-02-20T12:08:03.327 INFO:teuthology.orchestra.run.trial131.stdout:Downloading Packages: 2026-02-20T12:08:03.400 INFO:teuthology.orchestra.run.trial131.stdout:rbd-fuse-20.3.0-5312.g26bc9765.el9.x86_64.rpm 1.2 MB/s | 87 kB 00:00 2026-02-20T12:08:03.401 INFO:teuthology.orchestra.run.trial131.stdout:-------------------------------------------------------------------------------- 2026-02-20T12:08:03.401 INFO:teuthology.orchestra.run.trial131.stdout:Total 1.1 MB/s | 87 kB 00:00 2026-02-20T12:08:03.401 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction check 2026-02-20T12:08:03.404 INFO:teuthology.orchestra.run.trial131.stdout:Transaction check succeeded. 2026-02-20T12:08:03.405 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction test 2026-02-20T12:08:03.436 INFO:teuthology.orchestra.run.trial131.stdout:Transaction test succeeded. 2026-02-20T12:08:03.436 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction 2026-02-20T12:08:03.487 INFO:teuthology.orchestra.run.trial131.stdout: Preparing : 1/1 2026-02-20T12:08:03.504 INFO:teuthology.orchestra.run.trial131.stdout: Installing : rbd-fuse-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:08:03.585 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:08:03.644 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : rbd-fuse-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:08:03.644 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:03.644 INFO:teuthology.orchestra.run.trial131.stdout:Installed: 2026-02-20T12:08:03.644 INFO:teuthology.orchestra.run.trial131.stdout: rbd-fuse-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:08:03.644 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:03.644 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:08:03.673 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install rbd-mirror 2026-02-20T12:08:03.982 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:17 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:08:04.067 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:08:04.067 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:08:04.067 INFO:teuthology.orchestra.run.trial131.stdout: Package Arch Version Repo Size 2026-02-20T12:08:04.067 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:08:04.067 INFO:teuthology.orchestra.run.trial131.stdout:Installing: 2026-02-20T12:08:04.067 INFO:teuthology.orchestra.run.trial131.stdout: rbd-mirror x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 2.6 M 2026-02-20T12:08:04.067 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:04.068 INFO:teuthology.orchestra.run.trial131.stdout:Transaction Summary 2026-02-20T12:08:04.068 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:08:04.068 INFO:teuthology.orchestra.run.trial131.stdout:Install 1 Package 2026-02-20T12:08:04.068 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:04.068 INFO:teuthology.orchestra.run.trial131.stdout:Total download size: 2.6 M 2026-02-20T12:08:04.068 INFO:teuthology.orchestra.run.trial131.stdout:Installed size: 11 M 2026-02-20T12:08:04.068 INFO:teuthology.orchestra.run.trial131.stdout:Downloading Packages: 2026-02-20T12:08:04.214 INFO:teuthology.orchestra.run.trial131.stdout:rbd-mirror-20.3.0-5312.g26bc9765.el9.x86_64.rpm 18 MB/s | 2.6 MB 00:00 2026-02-20T12:08:04.214 INFO:teuthology.orchestra.run.trial131.stdout:-------------------------------------------------------------------------------- 2026-02-20T12:08:04.214 INFO:teuthology.orchestra.run.trial131.stdout:Total 18 MB/s | 2.6 MB 00:00 2026-02-20T12:08:04.214 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction check 2026-02-20T12:08:04.218 INFO:teuthology.orchestra.run.trial131.stdout:Transaction check succeeded. 2026-02-20T12:08:04.219 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction test 2026-02-20T12:08:04.258 INFO:teuthology.orchestra.run.trial131.stdout:Transaction test succeeded. 2026-02-20T12:08:04.258 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction 2026-02-20T12:08:04.424 INFO:teuthology.orchestra.run.trial131.stdout: Preparing : 1/1 2026-02-20T12:08:04.425 INFO:teuthology.orchestra.run.trial131.stdout: Installing : rbd-mirror-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:08:04.447 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:08:04.447 INFO:teuthology.orchestra.run.trial131.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T12:08:04.447 INFO:teuthology.orchestra.run.trial131.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-20T12:08:04.448 INFO:teuthology.orchestra.run.trial131.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T12:08:04.448 INFO:teuthology.orchestra.run.trial131.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T12:08:04.448 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:04.764 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : rbd-mirror-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:08:04.764 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:04.764 INFO:teuthology.orchestra.run.trial131.stdout:Installed: 2026-02-20T12:08:04.764 INFO:teuthology.orchestra.run.trial131.stdout: rbd-mirror-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:08:04.764 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:04.764 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:08:04.792 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install rbd-nbd 2026-02-20T12:08:05.104 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:19 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:08:05.186 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:08:05.186 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:08:05.186 INFO:teuthology.orchestra.run.trial131.stdout: Package Architecture Version Repository Size 2026-02-20T12:08:05.186 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:08:05.186 INFO:teuthology.orchestra.run.trial131.stdout:Installing: 2026-02-20T12:08:05.187 INFO:teuthology.orchestra.run.trial131.stdout: rbd-nbd x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 166 k 2026-02-20T12:08:05.187 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:05.187 INFO:teuthology.orchestra.run.trial131.stdout:Transaction Summary 2026-02-20T12:08:05.187 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:08:05.187 INFO:teuthology.orchestra.run.trial131.stdout:Install 1 Package 2026-02-20T12:08:05.187 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:05.187 INFO:teuthology.orchestra.run.trial131.stdout:Total download size: 166 k 2026-02-20T12:08:05.187 INFO:teuthology.orchestra.run.trial131.stdout:Installed size: 496 k 2026-02-20T12:08:05.187 INFO:teuthology.orchestra.run.trial131.stdout:Downloading Packages: 2026-02-20T12:08:05.274 INFO:teuthology.orchestra.run.trial131.stdout:rbd-nbd-20.3.0-5312.g26bc9765.el9.x86_64.rpm 1.9 MB/s | 166 kB 00:00 2026-02-20T12:08:05.274 INFO:teuthology.orchestra.run.trial131.stdout:-------------------------------------------------------------------------------- 2026-02-20T12:08:05.274 INFO:teuthology.orchestra.run.trial131.stdout:Total 1.9 MB/s | 166 kB 00:00 2026-02-20T12:08:05.275 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction check 2026-02-20T12:08:05.278 INFO:teuthology.orchestra.run.trial131.stdout:Transaction check succeeded. 2026-02-20T12:08:05.279 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction test 2026-02-20T12:08:05.310 INFO:teuthology.orchestra.run.trial131.stdout:Transaction test succeeded. 2026-02-20T12:08:05.310 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction 2026-02-20T12:08:05.366 INFO:teuthology.orchestra.run.trial131.stdout: Preparing : 1/1 2026-02-20T12:08:05.382 INFO:teuthology.orchestra.run.trial131.stdout: Installing : rbd-nbd-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:08:05.465 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:08:05.524 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : rbd-nbd-2:20.3.0-5312.g26bc9765.el9.x86_64 1/1 2026-02-20T12:08:05.524 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:05.525 INFO:teuthology.orchestra.run.trial131.stdout:Installed: 2026-02-20T12:08:05.525 INFO:teuthology.orchestra.run.trial131.stdout: rbd-nbd-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:08:05.525 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:05.525 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:08:05.552 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-osd-crimson 2026-02-20T12:08:05.855 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:19 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:08:05.919 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-osd-crimson-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:08:05.938 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:08:05.939 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:08:05.939 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:08:05.960 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-20T12:08:06.270 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:20 ago on Fri 20 Feb 2026 12:06:46 PM UTC. 2026-02-20T12:08:06.354 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:08:06.355 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:08:06.355 INFO:teuthology.orchestra.run.trial131.stdout: Package Arch Version Repo Size 2026-02-20T12:08:06.355 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:08:06.355 INFO:teuthology.orchestra.run.trial131.stdout:Installing: 2026-02-20T12:08:06.355 INFO:teuthology.orchestra.run.trial131.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 425 M 2026-02-20T12:08:06.355 INFO:teuthology.orchestra.run.trial131.stdout:Installing weak dependencies: 2026-02-20T12:08:06.355 INFO:teuthology.orchestra.run.trial131.stdout: ceph-debugsource x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 12 M 2026-02-20T12:08:06.355 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:06.355 INFO:teuthology.orchestra.run.trial131.stdout:Transaction Summary 2026-02-20T12:08:06.355 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:08:06.355 INFO:teuthology.orchestra.run.trial131.stdout:Install 2 Packages 2026-02-20T12:08:06.355 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:06.355 INFO:teuthology.orchestra.run.trial131.stdout:Total download size: 437 M 2026-02-20T12:08:06.355 INFO:teuthology.orchestra.run.trial131.stdout:Installed size: 2.6 G 2026-02-20T12:08:06.355 INFO:teuthology.orchestra.run.trial131.stdout:Downloading Packages: 2026-02-20T12:08:08.145 INFO:teuthology.orchestra.run.trial131.stdout:(1/2): ceph-debugsource-20.3.0-5312.g26bc9765.e 6.7 MB/s | 12 MB 00:01 2026-02-20T12:08:26.233 INFO:teuthology.orchestra.run.trial131.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5312.g 21 MB/s | 425 MB 00:19 2026-02-20T12:08:26.234 INFO:teuthology.orchestra.run.trial131.stdout:-------------------------------------------------------------------------------- 2026-02-20T12:08:26.235 INFO:teuthology.orchestra.run.trial131.stdout:Total 22 MB/s | 437 MB 00:19 2026-02-20T12:08:26.235 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction check 2026-02-20T12:08:26.240 INFO:teuthology.orchestra.run.trial131.stdout:Transaction check succeeded. 2026-02-20T12:08:26.240 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction test 2026-02-20T12:08:27.585 INFO:teuthology.orchestra.run.trial131.stdout:Transaction test succeeded. 2026-02-20T12:08:27.585 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction 2026-02-20T12:08:31.148 INFO:teuthology.orchestra.run.trial131.stdout: Preparing : 1/1 2026-02-20T12:08:53.271 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-debugsource-2:20.3.0-5312.g26bc9765.el9.x86_64 1/2 2026-02-20T12:08:53.288 INFO:teuthology.orchestra.run.trial131.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5312.g26bc9765.e 2/2 2026-02-20T12:08:55.213 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5312.g26bc9765.e 2/2 2026-02-20T12:08:55.213 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-debugsource-2:20.3.0-5312.g26bc9765.el9.x86_64 1/2 2026-02-20T12:08:55.272 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5312.g26bc9765.e 2/2 2026-02-20T12:08:55.272 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:55.272 INFO:teuthology.orchestra.run.trial131.stdout:Installed: 2026-02-20T12:08:55.272 INFO:teuthology.orchestra.run.trial131.stdout: ceph-debugsource-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:08:55.273 INFO:teuthology.orchestra.run.trial131.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:08:55.273 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:08:55.273 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:08:55.345 DEBUG:teuthology.parallel:result is None 2026-02-20T12:08:55.347 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=26bc97655ae0056b78c14ab91d1307332bd6dfca 2026-02-20T12:08:55.513 DEBUG:teuthology.orchestra.run.trial131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-20T12:08:55.535 INFO:teuthology.orchestra.run.trial131.stdout:20.3.0-5312.g26bc9765.el9 2026-02-20T12:08:55.536 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5312.g26bc9765.el9 2026-02-20T12:08:55.536 INFO:teuthology.task.install:The correct ceph version 20.3.0-5312.g26bc9765 is installed. 2026-02-20T12:08:55.538 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-20T12:08:55.538 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:08:55.539 DEBUG:teuthology.orchestra.run.trial131:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-20T12:08:55.599 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-20T12:08:55.600 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:08:55.600 DEBUG:teuthology.orchestra.run.trial131:> sudo dd of=/usr/bin/daemon-helper 2026-02-20T12:08:55.663 DEBUG:teuthology.orchestra.run.trial131:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-20T12:08:55.727 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-20T12:08:55.727 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:08:55.727 DEBUG:teuthology.orchestra.run.trial131:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-20T12:08:55.790 DEBUG:teuthology.orchestra.run.trial131:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-20T12:08:55.854 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-20T12:08:55.855 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:08:55.855 DEBUG:teuthology.orchestra.run.trial131:> sudo dd of=/usr/bin/stdin-killer 2026-02-20T12:08:55.920 DEBUG:teuthology.orchestra.run.trial131:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-20T12:08:55.984 INFO:teuthology.run_tasks:Running task pexec... 2026-02-20T12:08:56.005 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-20T12:08:56.006 DEBUG:teuthology.orchestra.run.trial131:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-20T12:08:56.027 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial131.front.sepia.ceph.com 2026-02-20T12:08:56.027 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-20T12:08:56.078 DEBUG:teuthology.parallel:result is None 2026-02-20T12:08:56.078 INFO:teuthology.run_tasks:Running task install... 2026-02-20T12:08:56.093 DEBUG:teuthology.task.install:project ceph 2026-02-20T12:08:56.093 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '26bc97655ae0056b78c14ab91d1307332bd6dfca'}} 2026-02-20T12:08:56.093 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': '26bc97655ae0056b78c14ab91d1307332bd6dfca'} 2026-02-20T12:08:56.093 INFO:teuthology.task.install:Using flavor: debug 2026-02-20T12:08:56.102 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-20T12:08:56.103 INFO:teuthology.task.install:extra packages: [] 2026-02-20T12:08:56.103 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '26bc97655ae0056b78c14ab91d1307332bd6dfca', 'tag': None, 'wait_for_package': False} 2026-02-20T12:08:56.103 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=26bc97655ae0056b78c14ab91d1307332bd6dfca 2026-02-20T12:08:56.264 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-12-02-2026-PR66245/26bc97655ae0056b78c14ab91d1307332bd6dfca/centos/9/flavors/debug/ 2026-02-20T12:08:56.264 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5312.g26bc9765 2026-02-20T12:08:56.631 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-12-02-2026-PR66245/26bc97655ae0056b78c14ab91d1307332bd6dfca/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-12-02-2026-PR66245/26bc97655ae0056b78c14ab91d1307332bd6dfca/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-12-02-2026-PR66245/26bc97655ae0056b78c14ab91d1307332bd6dfca/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-20T12:08:56.631 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:08:56.632 DEBUG:teuthology.orchestra.run.trial131:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-20T12:08:56.657 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-20T12:08:56.657 DEBUG:teuthology.orchestra.run.trial131:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/26bc97655ae0056b78c14ab91d1307332bd6dfca/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-20T12:08:56.721 DEBUG:teuthology.orchestra.run.trial131:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-20T12:08:56.785 DEBUG:teuthology.orchestra.run.trial131:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-20T12:08:56.842 INFO:teuthology.orchestra.run.trial131.stdout:check_obsoletes = 1 2026-02-20T12:08:56.852 DEBUG:teuthology.orchestra.run.trial131:> sudo yum clean all 2026-02-20T12:08:57.011 INFO:teuthology.orchestra.run.trial131.stdout:56 files removed 2026-02-20T12:08:57.026 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-radosgw 2026-02-20T12:08:57.808 INFO:teuthology.orchestra.run.trial131.stdout:ceph packages for x86_64 149 kB/s | 94 kB 00:00 2026-02-20T12:08:58.417 INFO:teuthology.orchestra.run.trial131.stdout:ceph noarch packages 42 kB/s | 24 kB 00:00 2026-02-20T12:08:58.986 INFO:teuthology.orchestra.run.trial131.stdout:ceph source packages 3.8 kB/s | 2.1 kB 00:00 2026-02-20T12:09:07.361 INFO:teuthology.orchestra.run.trial131.stdout:CentOS Stream 9 - BaseOS 1.1 MB/s | 8.9 MB 00:08 2026-02-20T12:09:35.457 INFO:teuthology.orchestra.run.trial131.stdout:CentOS Stream 9 - AppStream 996 kB/s | 27 MB 00:27 2026-02-20T12:09:38.842 INFO:teuthology.orchestra.run.trial131.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 7.8 MB 00:01 2026-02-20T12:09:39.799 INFO:teuthology.orchestra.run.trial131.stdout:CentOS Stream 9 - Extras packages 84 kB/s | 20 kB 00:00 2026-02-20T12:09:40.791 INFO:teuthology.orchestra.run.trial131.stdout:Extra Packages for Enterprise Linux 22 MB/s | 20 MB 00:00 2026-02-20T12:09:43.932 INFO:teuthology.orchestra.run.trial131.stdout:lab-extras 2.5 MB/s | 50 kB 00:00 2026-02-20T12:09:45.104 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-radosgw-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:45.122 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:45.123 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:45.123 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:45.147 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-test 2026-02-20T12:09:45.463 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:45.528 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-test-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:45.547 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:45.547 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:45.547 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:45.569 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph 2026-02-20T12:09:45.874 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:45.939 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:45.958 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:45.958 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:45.958 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:45.979 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-base 2026-02-20T12:09:46.289 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:03 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:46.354 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-base-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:46.373 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:46.373 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:46.374 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:46.395 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install cephadm 2026-02-20T12:09:46.704 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:03 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:46.769 INFO:teuthology.orchestra.run.trial131.stdout:Package cephadm-2:20.3.0-5312.g26bc9765.el9.noarch is already installed. 2026-02-20T12:09:46.788 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:46.788 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:46.788 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:46.809 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-immutable-object-cache 2026-02-20T12:09:47.119 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:04 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:47.183 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-immutable-object-cache-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:47.202 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:47.202 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:47.203 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:47.224 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-mgr 2026-02-20T12:09:47.534 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:04 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:47.599 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-mgr-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:47.618 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:47.619 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:47.619 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:47.641 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-mgr-dashboard 2026-02-20T12:09:47.959 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:04 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:48.023 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-mgr-dashboard-2:20.3.0-5312.g26bc9765.el9.noarch is already installed. 2026-02-20T12:09:48.042 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:48.042 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:48.042 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:48.064 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-20T12:09:48.373 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:05 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:48.437 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5312.g26bc9765.el9.noarch is already installed. 2026-02-20T12:09:48.456 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:48.456 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:48.457 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:48.477 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-mgr-rook 2026-02-20T12:09:48.784 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:05 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:48.849 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-mgr-rook-2:20.3.0-5312.g26bc9765.el9.noarch is already installed. 2026-02-20T12:09:48.868 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:48.869 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:48.869 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:48.890 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-mgr-cephadm 2026-02-20T12:09:49.206 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:06 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:49.271 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-mgr-cephadm-2:20.3.0-5312.g26bc9765.el9.noarch is already installed. 2026-02-20T12:09:49.289 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:49.290 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:49.290 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:49.312 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-osd 2026-02-20T12:09:49.626 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:06 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:49.692 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-osd-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:49.710 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:49.711 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:49.711 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:49.732 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-osd-classic 2026-02-20T12:09:50.046 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:07 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:50.110 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-osd-classic-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:50.129 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:50.129 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:50.130 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:50.151 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-fuse 2026-02-20T12:09:50.460 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:07 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:50.525 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-fuse-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:50.543 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:50.544 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:50.544 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:50.565 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install ceph-volume 2026-02-20T12:09:50.878 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:07 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:50.943 INFO:teuthology.orchestra.run.trial131.stdout:Package ceph-volume-2:20.3.0-5312.g26bc9765.el9.noarch is already installed. 2026-02-20T12:09:50.962 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:50.962 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:50.963 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:50.984 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install librados-devel 2026-02-20T12:09:51.296 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:08 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:51.363 INFO:teuthology.orchestra.run.trial131.stdout:Package librados-devel-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:51.381 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:51.382 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:51.382 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:51.405 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install libcephfs2 2026-02-20T12:09:51.708 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:08 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:51.773 INFO:teuthology.orchestra.run.trial131.stdout:Package libcephfs2-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:51.792 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:51.793 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:51.793 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:51.813 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install libcephfs-devel 2026-02-20T12:09:52.125 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:09 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:52.189 INFO:teuthology.orchestra.run.trial131.stdout:Package libcephfs-devel-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:52.208 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:52.209 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:52.209 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:52.230 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install librados2 2026-02-20T12:09:52.540 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:09 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:52.605 INFO:teuthology.orchestra.run.trial131.stdout:Package librados2-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:52.624 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:52.625 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:52.625 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:52.647 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install librbd1 2026-02-20T12:09:52.961 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:09 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:53.026 INFO:teuthology.orchestra.run.trial131.stdout:Package librbd1-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:53.045 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:53.046 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:53.046 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:53.067 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install python3-rados 2026-02-20T12:09:53.382 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:10 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:53.449 INFO:teuthology.orchestra.run.trial131.stdout:Package python3-rados-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:53.467 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:53.468 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:53.468 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:53.489 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install python3-rgw 2026-02-20T12:09:53.804 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:10 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:53.869 INFO:teuthology.orchestra.run.trial131.stdout:Package python3-rgw-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:53.887 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:53.888 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:53.888 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:53.909 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install python3-cephfs 2026-02-20T12:09:54.219 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:11 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:54.284 INFO:teuthology.orchestra.run.trial131.stdout:Package python3-cephfs-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:54.303 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:54.304 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:54.304 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:54.325 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install python3-rbd 2026-02-20T12:09:54.641 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:11 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:54.706 INFO:teuthology.orchestra.run.trial131.stdout:Package python3-rbd-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:54.724 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:54.725 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:54.725 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:54.746 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install rbd-fuse 2026-02-20T12:09:55.058 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:12 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:55.124 INFO:teuthology.orchestra.run.trial131.stdout:Package rbd-fuse-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:55.142 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:55.143 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:55.143 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:55.164 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install rbd-mirror 2026-02-20T12:09:55.475 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:12 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:55.541 INFO:teuthology.orchestra.run.trial131.stdout:Package rbd-mirror-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:55.559 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:55.560 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:55.560 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:55.581 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install rbd-nbd 2026-02-20T12:09:55.892 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:00:12 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:09:55.957 INFO:teuthology.orchestra.run.trial131.stdout:Package rbd-nbd-2:20.3.0-5312.g26bc9765.el9.x86_64 is already installed. 2026-02-20T12:09:55.976 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:09:55.976 INFO:teuthology.orchestra.run.trial131.stdout:Nothing to do. 2026-02-20T12:09:55.977 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:09:56.001 DEBUG:teuthology.parallel:result is None 2026-02-20T12:09:56.002 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=26bc97655ae0056b78c14ab91d1307332bd6dfca 2026-02-20T12:09:56.165 DEBUG:teuthology.orchestra.run.trial131:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-20T12:09:56.185 INFO:teuthology.orchestra.run.trial131.stdout:20.3.0-5312.g26bc9765.el9 2026-02-20T12:09:56.185 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5312.g26bc9765.el9 2026-02-20T12:09:56.186 INFO:teuthology.task.install:The correct ceph version 20.3.0-5312.g26bc9765 is installed. 2026-02-20T12:09:56.188 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-20T12:09:56.188 INFO:teuthology.run_tasks:Running task ceph... 2026-02-20T12:09:56.292 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-20T12:09:56.292 DEBUG:teuthology.orchestra.run.trial131:> sudo chmod 777 /var/log/ceph 2026-02-20T12:09:56.316 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-20T12:09:56.317 DEBUG:teuthology.orchestra.run.trial131:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-20T12:09:56.381 INFO:tasks.ceph:Creating extra log directories... 2026-02-20T12:09:56.381 DEBUG:teuthology.orchestra.run.trial131:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-20T12:09:56.455 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-20T12:09:56.455 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-20T12:09:56.456 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:29:19-crimson-rados-wip-perezjos-crimson-only-12-02-2026-PR66245-distro-debug-trial/60625', 'branch': 'wip-perezjos-crimson-only-12-02-2026-PR66245', 'description': 'crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} perf_ignorelist settings/optimized workloads/fio_4K_rand_rw}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '60625', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'jjperez-2026-02-20_10:29:19-crimson-rados-wip-perezjos-crimson-only-12-02-2026-PR66245-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-perezjos-crimson-only-12-02-2026-PR66245'}, 'ceph': {'conf': {'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': '26bc97655ae0056b78c14ab91d1307332bd6dfca'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': '26bc97655ae0056b78c14ab91d1307332bd6dfca'}}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '445d5c9ba18a4040e70e25303230c36c77773acb'}}, 'owner': 'scheduled_jjperez@soko04.front.sepia.ceph.com', 'priority': 101, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 5547, 'sha1': '26bc97655ae0056b78c14ab91d1307332bd6dfca', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': '445d5c9ba18a4040e70e25303230c36c77773acb', 'targets': {'trial131.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBGlv7HAgmqSscJYcergW8kdG4gz3ZdM0bq6HeVpKKAxyt9ZwUXgHoVxkFPk+TGY9aydadUJcGuF9bCKsCgZDRyA='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '26bc97655ae0056b78c14ab91d1307332bd6dfca'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'wait-for-scrub': False, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': '26bc97655ae0056b78c14ab91d1307332bd6dfca', 'cluster': 'ceph'}}, {'ssh_keys': None}, {'cbt': {'benchmarks': {'librbdfio': {'iodepth': [32], 'log_avg_msec': 100, 'mode': ['randrw'], 'norandommap': True, 'op_size': [4096], 'osd_ra': [4096], 'pool_profile': 'rbd', 'procs_per_volume': [1], 'time': 60, 'vol_size': 4096, 'volumes_per_client': [2]}}, 'cluster': {'iterations': 1, 'osds_per_node': 3, 'pool_profiles': {'rbd': {'pg_size': 128, 'pgp_size': 128, 'replication': 3}}, 'user': 'ubuntu'}, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'reimage-ssh-down', 'teuthology_sha1': 'e6d9099ce03dc21193fda653159ec82465c3a94a', 'timestamp': '2026-02-20_10:29:19', 'tube': 'trial', 'user': 'jjperez', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699'} 2026-02-20T12:09:56.456 DEBUG:teuthology.orchestra.run.trial131:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-20T12:09:56.500 DEBUG:teuthology.orchestra.run.trial131:> sudo install -d -m0777 -- /var/run/ceph 2026-02-20T12:09:56.564 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:09:56.564 DEBUG:teuthology.orchestra.run.trial131:> dd if=/scratch_devs of=/dev/stdout 2026-02-20T12:09:56.622 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-20T12:09:56.623 DEBUG:teuthology.orchestra.run.trial131:> stat /dev/vg_nvme/lv_1 2026-02-20T12:09:56.679 INFO:teuthology.orchestra.run.trial131.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-20T12:09:56.679 INFO:teuthology.orchestra.run.trial131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T12:09:56.679 INFO:teuthology.orchestra.run.trial131.stdout:Device: 6h/6d Inode: 1307 Links: 1 2026-02-20T12:09:56.679 INFO:teuthology.orchestra.run.trial131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T12:09:56.679 INFO:teuthology.orchestra.run.trial131.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T12:09:56.679 INFO:teuthology.orchestra.run.trial131.stdout:Access: 2026-02-20 12:08:04.587094744 +0000 2026-02-20T12:09:56.680 INFO:teuthology.orchestra.run.trial131.stdout:Modify: 2026-02-20 12:06:53.186054353 +0000 2026-02-20T12:09:56.680 INFO:teuthology.orchestra.run.trial131.stdout:Change: 2026-02-20 12:06:53.186054353 +0000 2026-02-20T12:09:56.680 INFO:teuthology.orchestra.run.trial131.stdout: Birth: 2026-02-20 12:06:53.186054353 +0000 2026-02-20T12:09:56.680 DEBUG:teuthology.orchestra.run.trial131:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-20T12:09:56.742 INFO:teuthology.orchestra.run.trial131.stderr:1+0 records in 2026-02-20T12:09:56.742 INFO:teuthology.orchestra.run.trial131.stderr:1+0 records out 2026-02-20T12:09:56.742 INFO:teuthology.orchestra.run.trial131.stderr:512 bytes copied, 0.000167323 s, 3.1 MB/s 2026-02-20T12:09:56.743 DEBUG:teuthology.orchestra.run.trial131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-20T12:09:56.802 DEBUG:teuthology.orchestra.run.trial131:> stat /dev/vg_nvme/lv_2 2026-02-20T12:09:56.858 INFO:teuthology.orchestra.run.trial131.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-20T12:09:56.859 INFO:teuthology.orchestra.run.trial131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T12:09:56.859 INFO:teuthology.orchestra.run.trial131.stdout:Device: 6h/6d Inode: 1315 Links: 1 2026-02-20T12:09:56.859 INFO:teuthology.orchestra.run.trial131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T12:09:56.859 INFO:teuthology.orchestra.run.trial131.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T12:09:56.859 INFO:teuthology.orchestra.run.trial131.stdout:Access: 2026-02-20 12:08:04.588094792 +0000 2026-02-20T12:09:56.859 INFO:teuthology.orchestra.run.trial131.stdout:Modify: 2026-02-20 12:06:53.187054314 +0000 2026-02-20T12:09:56.859 INFO:teuthology.orchestra.run.trial131.stdout:Change: 2026-02-20 12:06:53.187054314 +0000 2026-02-20T12:09:56.859 INFO:teuthology.orchestra.run.trial131.stdout: Birth: 2026-02-20 12:06:53.187054314 +0000 2026-02-20T12:09:56.860 DEBUG:teuthology.orchestra.run.trial131:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-20T12:09:56.921 INFO:teuthology.orchestra.run.trial131.stderr:1+0 records in 2026-02-20T12:09:56.921 INFO:teuthology.orchestra.run.trial131.stderr:1+0 records out 2026-02-20T12:09:56.921 INFO:teuthology.orchestra.run.trial131.stderr:512 bytes copied, 0.000141705 s, 3.6 MB/s 2026-02-20T12:09:56.922 DEBUG:teuthology.orchestra.run.trial131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-20T12:09:56.979 DEBUG:teuthology.orchestra.run.trial131:> stat /dev/vg_nvme/lv_3 2026-02-20T12:09:57.037 INFO:teuthology.orchestra.run.trial131.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-20T12:09:57.037 INFO:teuthology.orchestra.run.trial131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T12:09:57.037 INFO:teuthology.orchestra.run.trial131.stdout:Device: 6h/6d Inode: 1302 Links: 1 2026-02-20T12:09:57.037 INFO:teuthology.orchestra.run.trial131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T12:09:57.037 INFO:teuthology.orchestra.run.trial131.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T12:09:57.037 INFO:teuthology.orchestra.run.trial131.stdout:Access: 2026-02-20 12:08:04.588094792 +0000 2026-02-20T12:09:57.038 INFO:teuthology.orchestra.run.trial131.stdout:Modify: 2026-02-20 12:06:53.186054353 +0000 2026-02-20T12:09:57.038 INFO:teuthology.orchestra.run.trial131.stdout:Change: 2026-02-20 12:06:53.186054353 +0000 2026-02-20T12:09:57.038 INFO:teuthology.orchestra.run.trial131.stdout: Birth: 2026-02-20 12:06:53.186054353 +0000 2026-02-20T12:09:57.038 DEBUG:teuthology.orchestra.run.trial131:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-20T12:09:57.100 INFO:teuthology.orchestra.run.trial131.stderr:1+0 records in 2026-02-20T12:09:57.100 INFO:teuthology.orchestra.run.trial131.stderr:1+0 records out 2026-02-20T12:09:57.100 INFO:teuthology.orchestra.run.trial131.stderr:512 bytes copied, 0.000153768 s, 3.3 MB/s 2026-02-20T12:09:57.102 DEBUG:teuthology.orchestra.run.trial131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-20T12:09:57.158 DEBUG:teuthology.orchestra.run.trial131:> stat /dev/vg_nvme/lv_4 2026-02-20T12:09:57.215 INFO:teuthology.orchestra.run.trial131.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-20T12:09:57.215 INFO:teuthology.orchestra.run.trial131.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T12:09:57.215 INFO:teuthology.orchestra.run.trial131.stdout:Device: 6h/6d Inode: 1287 Links: 1 2026-02-20T12:09:57.216 INFO:teuthology.orchestra.run.trial131.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T12:09:57.216 INFO:teuthology.orchestra.run.trial131.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T12:09:57.216 INFO:teuthology.orchestra.run.trial131.stdout:Access: 2026-02-20 12:08:04.588094792 +0000 2026-02-20T12:09:57.216 INFO:teuthology.orchestra.run.trial131.stdout:Modify: 2026-02-20 12:06:53.185054391 +0000 2026-02-20T12:09:57.216 INFO:teuthology.orchestra.run.trial131.stdout:Change: 2026-02-20 12:06:53.185054391 +0000 2026-02-20T12:09:57.216 INFO:teuthology.orchestra.run.trial131.stdout: Birth: 2026-02-20 12:06:53.185054391 +0000 2026-02-20T12:09:57.216 DEBUG:teuthology.orchestra.run.trial131:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-20T12:09:57.278 INFO:teuthology.orchestra.run.trial131.stderr:1+0 records in 2026-02-20T12:09:57.278 INFO:teuthology.orchestra.run.trial131.stderr:1+0 records out 2026-02-20T12:09:57.278 INFO:teuthology.orchestra.run.trial131.stderr:512 bytes copied, 0.000145322 s, 3.5 MB/s 2026-02-20T12:09:57.279 DEBUG:teuthology.orchestra.run.trial131:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-20T12:09:57.336 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T12:09:57.337 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial131.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-20T12:09:57.337 INFO:tasks.ceph:Generating config... 2026-02-20T12:09:57.339 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-20T12:09:57.340 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-20T12:09:57.340 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-20T12:09:57.340 INFO:tasks.ceph:[osd] debug filestore = 0/0 2026-02-20T12:09:57.340 INFO:tasks.ceph:[osd] debug journal = 0/0 2026-02-20T12:09:57.340 INFO:tasks.ceph:[osd] debug ms = 0/0 2026-02-20T12:09:57.340 INFO:tasks.ceph:[osd] debug osd = 0/0 2026-02-20T12:09:57.340 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-20T12:09:57.340 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-20T12:09:57.340 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-20T12:09:57.340 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-20T12:09:57.341 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-20T12:09:57.341 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-20T12:09:57.341 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-20T12:09:57.341 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-20T12:09:57.341 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-20T12:09:57.341 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-20T12:09:57.341 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-20T12:09:57.341 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-20T12:09:57.341 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-20T12:09:57.341 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-20T12:09:57.341 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-20T12:09:57.341 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-20T12:09:57.341 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-20T12:09:57.342 INFO:tasks.ceph:[osd] seastore main device type = RANDOM_BLOCK_SSD 2026-02-20T12:09:57.342 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-20T12:09:57.342 INFO:tasks.ceph:[global] auth client required = none 2026-02-20T12:09:57.342 INFO:tasks.ceph:[global] auth cluster required = none 2026-02-20T12:09:57.342 INFO:tasks.ceph:[global] auth service required = none 2026-02-20T12:09:57.342 INFO:tasks.ceph:[global] auth supported = none 2026-02-20T12:09:57.342 INFO:tasks.ceph:[global] debug asok = 0/0 2026-02-20T12:09:57.342 INFO:tasks.ceph:[global] debug auth = 0/0 2026-02-20T12:09:57.342 INFO:tasks.ceph:[global] debug bdev = 0/0 2026-02-20T12:09:57.342 INFO:tasks.ceph:[global] debug bluefs = 0/0 2026-02-20T12:09:57.342 INFO:tasks.ceph:[global] debug bluestore = 0/0 2026-02-20T12:09:57.342 INFO:tasks.ceph:[global] debug buffer = 0/0 2026-02-20T12:09:57.342 INFO:tasks.ceph:[global] debug civetweb = 0/0 2026-02-20T12:09:57.343 INFO:tasks.ceph:[global] debug client = 0/0 2026-02-20T12:09:57.343 INFO:tasks.ceph:[global] debug compressor = 0/0 2026-02-20T12:09:57.343 INFO:tasks.ceph:[global] debug context = 0/0 2026-02-20T12:09:57.343 INFO:tasks.ceph:[global] debug crush = 0/0 2026-02-20T12:09:57.343 INFO:tasks.ceph:[global] debug crypto = 0/0 2026-02-20T12:09:57.343 INFO:tasks.ceph:[global] debug dpdk = 0/0 2026-02-20T12:09:57.343 INFO:tasks.ceph:[global] debug eventtrace = 0/0 2026-02-20T12:09:57.343 INFO:tasks.ceph:[global] debug filer = 0/0 2026-02-20T12:09:57.343 INFO:tasks.ceph:[global] debug filestore = 0/0 2026-02-20T12:09:57.343 INFO:tasks.ceph:[global] debug finisher = 0/0 2026-02-20T12:09:57.343 INFO:tasks.ceph:[global] debug fuse = 0/0 2026-02-20T12:09:57.343 INFO:tasks.ceph:[global] debug heartbeatmap = 0/0 2026-02-20T12:09:57.343 INFO:tasks.ceph:[global] debug javaclient = 0/0 2026-02-20T12:09:57.344 INFO:tasks.ceph:[global] debug journal = 0/0 2026-02-20T12:09:57.344 INFO:tasks.ceph:[global] debug journaler = 0/0 2026-02-20T12:09:57.344 INFO:tasks.ceph:[global] debug kstore = 0/0 2026-02-20T12:09:57.344 INFO:tasks.ceph:[global] debug leveldb = 0/0 2026-02-20T12:09:57.344 INFO:tasks.ceph:[global] debug lockdep = 0/0 2026-02-20T12:09:57.344 INFO:tasks.ceph:[global] debug mds = 0/0 2026-02-20T12:09:57.344 INFO:tasks.ceph:[global] debug mds balancer = 0/0 2026-02-20T12:09:57.344 INFO:tasks.ceph:[global] debug mds locker = 0/0 2026-02-20T12:09:57.344 INFO:tasks.ceph:[global] debug mds log = 0/0 2026-02-20T12:09:57.344 INFO:tasks.ceph:[global] debug mds log expire = 0/0 2026-02-20T12:09:57.344 INFO:tasks.ceph:[global] debug mds migrator = 0/0 2026-02-20T12:09:57.344 INFO:tasks.ceph:[global] debug memdb = 0/0 2026-02-20T12:09:57.344 INFO:tasks.ceph:[global] debug mgr = 0/0 2026-02-20T12:09:57.345 INFO:tasks.ceph:[global] debug mgrc = 0/0 2026-02-20T12:09:57.345 INFO:tasks.ceph:[global] debug mon = 0/0 2026-02-20T12:09:57.345 INFO:tasks.ceph:[global] debug monc = 0/0 2026-02-20T12:09:57.345 INFO:tasks.ceph:[global] debug ms = 0/0 2026-02-20T12:09:57.345 INFO:tasks.ceph:[global] debug objclass = 0/0 2026-02-20T12:09:57.345 INFO:tasks.ceph:[global] debug objectcacher = 0/0 2026-02-20T12:09:57.345 INFO:tasks.ceph:[global] debug objecter = 0/0 2026-02-20T12:09:57.345 INFO:tasks.ceph:[global] debug optracker = 0/0 2026-02-20T12:09:57.345 INFO:tasks.ceph:[global] debug osd = 0/0 2026-02-20T12:09:57.345 INFO:tasks.ceph:[global] debug paxos = 0/0 2026-02-20T12:09:57.345 INFO:tasks.ceph:[global] debug perfcounter = 0/0 2026-02-20T12:09:57.345 INFO:tasks.ceph:[global] debug rados = 0/0 2026-02-20T12:09:57.345 INFO:tasks.ceph:[global] debug rbd = 0/0 2026-02-20T12:09:57.346 INFO:tasks.ceph:[global] debug rbd mirror = 0/0 2026-02-20T12:09:57.346 INFO:tasks.ceph:[global] debug rbd replay = 0/0 2026-02-20T12:09:57.346 INFO:tasks.ceph:[global] debug refs = 0/0 2026-02-20T12:09:57.346 INFO:tasks.ceph:[global] debug rgw = 0/0 2026-02-20T12:09:57.346 INFO:tasks.ceph:[global] debug rgw lifecycle = 0/0 2026-02-20T12:09:57.346 INFO:tasks.ceph:[global] debug rgw notification = 0/0 2026-02-20T12:09:57.346 INFO:tasks.ceph:[global] debug rgw sync = 0/0 2026-02-20T12:09:57.346 INFO:tasks.ceph:[global] debug rocksdb = 0/0 2026-02-20T12:09:57.346 INFO:tasks.ceph:[global] debug striper = 0/0 2026-02-20T12:09:57.346 INFO:tasks.ceph:[global] debug throttle = 0/0 2026-02-20T12:09:57.346 INFO:tasks.ceph:[global] debug timer = 0/0 2026-02-20T12:09:57.346 INFO:tasks.ceph:[global] debug tp = 0/0 2026-02-20T12:09:57.346 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-20T12:09:57.346 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-20T12:09:57.347 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-20T12:09:57.347 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-20T12:09:57.347 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-20T12:09:57.347 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-20T12:09:57.347 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-20T12:09:57.347 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-20T12:09:57.347 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-20T12:09:57.347 INFO:tasks.ceph:[mon] debug mon = 0/0 2026-02-20T12:09:57.347 INFO:tasks.ceph:[mon] debug ms = 0/0 2026-02-20T12:09:57.347 INFO:tasks.ceph:[mon] debug paxos = 0/0 2026-02-20T12:09:57.347 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-20T12:09:57.348 INFO:tasks.ceph:Setting up mon.a... 2026-02-20T12:09:57.348 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-20T12:09:57.417 INFO:teuthology.orchestra.run.trial131.stdout:creating /etc/ceph/ceph.keyring 2026-02-20T12:09:57.419 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-20T12:09:57.502 DEBUG:teuthology.orchestra.run.trial131:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-20T12:09:57.566 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.131')] 2026-02-20T12:09:57.567 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'none', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.131', 'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': '0/0', 'debug mon': '0/0', 'debug paxos': '0/0', 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-20T12:09:57.569 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:09:57.569 DEBUG:teuthology.orchestra.run.trial131:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-20T12:09:57.624 DEBUG:teuthology.orchestra.run.trial131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.131 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-20T12:09:57.703 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:09:57.702+0000 7fe335292f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:09:57.703 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:09:57.702+0000 7fe335292f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:09:57.703 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:09:57.702+0000 7fe335292f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:09:57.704 INFO:teuthology.orchestra.run.trial131.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-20T12:09:57.705 INFO:teuthology.orchestra.run.trial131.stdout:monmaptool: generated fsid 7a6ea43f-ce6a-400f-9bcb-948d8c241da2 2026-02-20T12:09:57.705 INFO:teuthology.orchestra.run.trial131.stdout:setting min_mon_release = tentacle 2026-02-20T12:09:57.705 INFO:teuthology.orchestra.run.trial131.stdout:epoch 0 2026-02-20T12:09:57.705 INFO:teuthology.orchestra.run.trial131.stdout:fsid 7a6ea43f-ce6a-400f-9bcb-948d8c241da2 2026-02-20T12:09:57.705 INFO:teuthology.orchestra.run.trial131.stdout:last_changed 2026-02-20T12:09:57.703265+0000 2026-02-20T12:09:57.705 INFO:teuthology.orchestra.run.trial131.stdout:created 2026-02-20T12:09:57.703265+0000 2026-02-20T12:09:57.705 INFO:teuthology.orchestra.run.trial131.stdout:min_mon_release 20 (tentacle) 2026-02-20T12:09:57.705 INFO:teuthology.orchestra.run.trial131.stdout:election_strategy: 1 2026-02-20T12:09:57.705 INFO:teuthology.orchestra.run.trial131.stdout:0: [v2:10.20.193.131:3300/0,v1:10.20.193.131:6789/0] mon.a 2026-02-20T12:09:57.706 INFO:teuthology.orchestra.run.trial131.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-20T12:09:57.706 DEBUG:teuthology.orchestra.run.trial131:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-20T12:09:57.761 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 7a6ea43f-ce6a-400f-9bcb-948d8c241da2... 2026-02-20T12:09:57.763 DEBUG:teuthology.orchestra.run.trial131:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-20T12:09:57.843 INFO:teuthology.orchestra.run.trial131.stdout:[global] 2026-02-20T12:09:57.843 INFO:teuthology.orchestra.run.trial131.stdout: chdir = "" 2026-02-20T12:09:57.843 INFO:teuthology.orchestra.run.trial131.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-20T12:09:57.843 INFO:teuthology.orchestra.run.trial131.stdout: auth supported = none 2026-02-20T12:09:57.844 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.844 INFO:teuthology.orchestra.run.trial131.stdout: filestore xattr use omap = true 2026-02-20T12:09:57.844 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.844 INFO:teuthology.orchestra.run.trial131.stdout: mon clock drift allowed = 1.000 2026-02-20T12:09:57.844 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.845 INFO:teuthology.orchestra.run.trial131.stdout: osd crush chooseleaf type = 0 2026-02-20T12:09:57.845 INFO:teuthology.orchestra.run.trial131.stdout: auth debug = true 2026-02-20T12:09:57.845 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.845 INFO:teuthology.orchestra.run.trial131.stdout: ms die on old message = true 2026-02-20T12:09:57.845 INFO:teuthology.orchestra.run.trial131.stdout: ms die on bug = true 2026-02-20T12:09:57.845 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.846 INFO:teuthology.orchestra.run.trial131.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-20T12:09:57.846 INFO:teuthology.orchestra.run.trial131.stdout: mon pg warn max object skew = 0 2026-02-20T12:09:57.846 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.846 INFO:teuthology.orchestra.run.trial131.stdout: # disable pg_autoscaler by default for new pools 2026-02-20T12:09:57.846 INFO:teuthology.orchestra.run.trial131.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-20T12:09:57.846 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.846 INFO:teuthology.orchestra.run.trial131.stdout: osd pool default size = 2 2026-02-20T12:09:57.847 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.847 INFO:teuthology.orchestra.run.trial131.stdout: mon osd allow primary affinity = true 2026-02-20T12:09:57.847 INFO:teuthology.orchestra.run.trial131.stdout: mon osd allow pg remap = true 2026-02-20T12:09:57.847 INFO:teuthology.orchestra.run.trial131.stdout: mon warn on legacy crush tunables = false 2026-02-20T12:09:57.847 INFO:teuthology.orchestra.run.trial131.stdout: mon warn on crush straw calc version zero = false 2026-02-20T12:09:57.847 INFO:teuthology.orchestra.run.trial131.stdout: mon warn on no sortbitwise = false 2026-02-20T12:09:57.848 INFO:teuthology.orchestra.run.trial131.stdout: mon warn on osd down out interval zero = false 2026-02-20T12:09:57.848 INFO:teuthology.orchestra.run.trial131.stdout: mon warn on too few osds = false 2026-02-20T12:09:57.848 INFO:teuthology.orchestra.run.trial131.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-20T12:09:57.848 INFO:teuthology.orchestra.run.trial131.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-20T12:09:57.848 INFO:teuthology.orchestra.run.trial131.stdout: mon_allow_pool_size_one = true 2026-02-20T12:09:57.848 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.849 INFO:teuthology.orchestra.run.trial131.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-20T12:09:57.849 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.849 INFO:teuthology.orchestra.run.trial131.stdout: osd default data pool replay window = 5 2026-02-20T12:09:57.849 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.849 INFO:teuthology.orchestra.run.trial131.stdout: mon allow pool delete = true 2026-02-20T12:09:57.849 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.850 INFO:teuthology.orchestra.run.trial131.stdout: mon cluster log file level = debug 2026-02-20T12:09:57.850 INFO:teuthology.orchestra.run.trial131.stdout: debug asserts on shutdown = true 2026-02-20T12:09:57.850 INFO:teuthology.orchestra.run.trial131.stdout: mon health detail to clog = false 2026-02-20T12:09:57.850 INFO:teuthology.orchestra.run.trial131.stdout: mon host = 10.20.193.131 2026-02-20T12:09:57.850 INFO:teuthology.orchestra.run.trial131.stdout: auth client required = none 2026-02-20T12:09:57.850 INFO:teuthology.orchestra.run.trial131.stdout: auth cluster required = none 2026-02-20T12:09:57.851 INFO:teuthology.orchestra.run.trial131.stdout: auth service required = none 2026-02-20T12:09:57.851 INFO:teuthology.orchestra.run.trial131.stdout: debug asok = 0/0 2026-02-20T12:09:57.851 INFO:teuthology.orchestra.run.trial131.stdout: debug auth = 0/0 2026-02-20T12:09:57.851 INFO:teuthology.orchestra.run.trial131.stdout: debug bdev = 0/0 2026-02-20T12:09:57.851 INFO:teuthology.orchestra.run.trial131.stdout: debug bluefs = 0/0 2026-02-20T12:09:57.851 INFO:teuthology.orchestra.run.trial131.stdout: debug bluestore = 0/0 2026-02-20T12:09:57.852 INFO:teuthology.orchestra.run.trial131.stdout: debug buffer = 0/0 2026-02-20T12:09:57.852 INFO:teuthology.orchestra.run.trial131.stdout: debug civetweb = 0/0 2026-02-20T12:09:57.852 INFO:teuthology.orchestra.run.trial131.stdout: debug client = 0/0 2026-02-20T12:09:57.852 INFO:teuthology.orchestra.run.trial131.stdout: debug compressor = 0/0 2026-02-20T12:09:57.852 INFO:teuthology.orchestra.run.trial131.stdout: debug context = 0/0 2026-02-20T12:09:57.852 INFO:teuthology.orchestra.run.trial131.stdout: debug crush = 0/0 2026-02-20T12:09:57.853 INFO:teuthology.orchestra.run.trial131.stdout: debug crypto = 0/0 2026-02-20T12:09:57.853 INFO:teuthology.orchestra.run.trial131.stdout: debug dpdk = 0/0 2026-02-20T12:09:57.853 INFO:teuthology.orchestra.run.trial131.stdout: debug eventtrace = 0/0 2026-02-20T12:09:57.853 INFO:teuthology.orchestra.run.trial131.stdout: debug filer = 0/0 2026-02-20T12:09:57.853 INFO:teuthology.orchestra.run.trial131.stdout: debug filestore = 0/0 2026-02-20T12:09:57.853 INFO:teuthology.orchestra.run.trial131.stdout: debug finisher = 0/0 2026-02-20T12:09:57.854 INFO:teuthology.orchestra.run.trial131.stdout: debug fuse = 0/0 2026-02-20T12:09:57.854 INFO:teuthology.orchestra.run.trial131.stdout: debug heartbeatmap = 0/0 2026-02-20T12:09:57.854 INFO:teuthology.orchestra.run.trial131.stdout: debug javaclient = 0/0 2026-02-20T12:09:57.854 INFO:teuthology.orchestra.run.trial131.stdout: debug journal = 0/0 2026-02-20T12:09:57.854 INFO:teuthology.orchestra.run.trial131.stdout: debug journaler = 0/0 2026-02-20T12:09:57.854 INFO:teuthology.orchestra.run.trial131.stdout: debug kstore = 0/0 2026-02-20T12:09:57.854 INFO:teuthology.orchestra.run.trial131.stdout: debug leveldb = 0/0 2026-02-20T12:09:57.855 INFO:teuthology.orchestra.run.trial131.stdout: debug lockdep = 0/0 2026-02-20T12:09:57.855 INFO:teuthology.orchestra.run.trial131.stdout: debug mds = 0/0 2026-02-20T12:09:57.855 INFO:teuthology.orchestra.run.trial131.stdout: debug mds balancer = 0/0 2026-02-20T12:09:57.855 INFO:teuthology.orchestra.run.trial131.stdout: debug mds locker = 0/0 2026-02-20T12:09:57.855 INFO:teuthology.orchestra.run.trial131.stdout: debug mds log = 0/0 2026-02-20T12:09:57.855 INFO:teuthology.orchestra.run.trial131.stdout: debug mds log expire = 0/0 2026-02-20T12:09:57.855 INFO:teuthology.orchestra.run.trial131.stdout: debug mds migrator = 0/0 2026-02-20T12:09:57.856 INFO:teuthology.orchestra.run.trial131.stdout: debug memdb = 0/0 2026-02-20T12:09:57.856 INFO:teuthology.orchestra.run.trial131.stdout: debug mgr = 0/0 2026-02-20T12:09:57.856 INFO:teuthology.orchestra.run.trial131.stdout: debug mgrc = 0/0 2026-02-20T12:09:57.856 INFO:teuthology.orchestra.run.trial131.stdout: debug mon = 0/0 2026-02-20T12:09:57.856 INFO:teuthology.orchestra.run.trial131.stdout: debug monc = 0/0 2026-02-20T12:09:57.856 INFO:teuthology.orchestra.run.trial131.stdout: debug ms = 0/0 2026-02-20T12:09:57.856 INFO:teuthology.orchestra.run.trial131.stdout: debug objclass = 0/0 2026-02-20T12:09:57.856 INFO:teuthology.orchestra.run.trial131.stdout: debug objectcacher = 0/0 2026-02-20T12:09:57.857 INFO:teuthology.orchestra.run.trial131.stdout: debug objecter = 0/0 2026-02-20T12:09:57.857 INFO:teuthology.orchestra.run.trial131.stdout: debug optracker = 0/0 2026-02-20T12:09:57.857 INFO:teuthology.orchestra.run.trial131.stdout: debug osd = 0/0 2026-02-20T12:09:57.857 INFO:teuthology.orchestra.run.trial131.stdout: debug paxos = 0/0 2026-02-20T12:09:57.857 INFO:teuthology.orchestra.run.trial131.stdout: debug perfcounter = 0/0 2026-02-20T12:09:57.857 INFO:teuthology.orchestra.run.trial131.stdout: debug rados = 0/0 2026-02-20T12:09:57.857 INFO:teuthology.orchestra.run.trial131.stdout: debug rbd = 0/0 2026-02-20T12:09:57.857 INFO:teuthology.orchestra.run.trial131.stdout: debug rbd mirror = 0/0 2026-02-20T12:09:57.858 INFO:teuthology.orchestra.run.trial131.stdout: debug rbd replay = 0/0 2026-02-20T12:09:57.858 INFO:teuthology.orchestra.run.trial131.stdout: debug refs = 0/0 2026-02-20T12:09:57.858 INFO:teuthology.orchestra.run.trial131.stdout: debug rgw = 0/0 2026-02-20T12:09:57.858 INFO:teuthology.orchestra.run.trial131.stdout: debug rgw lifecycle = 0/0 2026-02-20T12:09:57.858 INFO:teuthology.orchestra.run.trial131.stdout: debug rgw notification = 0/0 2026-02-20T12:09:57.858 INFO:teuthology.orchestra.run.trial131.stdout: debug rgw sync = 0/0 2026-02-20T12:09:57.859 INFO:teuthology.orchestra.run.trial131.stdout: debug rocksdb = 0/0 2026-02-20T12:09:57.859 INFO:teuthology.orchestra.run.trial131.stdout: debug striper = 0/0 2026-02-20T12:09:57.859 INFO:teuthology.orchestra.run.trial131.stdout: debug throttle = 0/0 2026-02-20T12:09:57.859 INFO:teuthology.orchestra.run.trial131.stdout: debug timer = 0/0 2026-02-20T12:09:57.859 INFO:teuthology.orchestra.run.trial131.stdout: debug tp = 0/0 2026-02-20T12:09:57.859 INFO:teuthology.orchestra.run.trial131.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-20T12:09:57.859 INFO:teuthology.orchestra.run.trial131.stdout: ms client mode = crc 2026-02-20T12:09:57.859 INFO:teuthology.orchestra.run.trial131.stdout: ms cluster mode = crc 2026-02-20T12:09:57.860 INFO:teuthology.orchestra.run.trial131.stdout: ms mon client mode = crc 2026-02-20T12:09:57.860 INFO:teuthology.orchestra.run.trial131.stdout: ms mon cluster mode = crc 2026-02-20T12:09:57.860 INFO:teuthology.orchestra.run.trial131.stdout: ms mon service mode = crc 2026-02-20T12:09:57.860 INFO:teuthology.orchestra.run.trial131.stdout: ms service mode = crc 2026-02-20T12:09:57.860 INFO:teuthology.orchestra.run.trial131.stdout: fsid = 7a6ea43f-ce6a-400f-9bcb-948d8c241da2 2026-02-20T12:09:57.860 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.860 INFO:teuthology.orchestra.run.trial131.stdout:[osd] 2026-02-20T12:09:57.860 INFO:teuthology.orchestra.run.trial131.stdout: osd journal size = 100 2026-02-20T12:09:57.860 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.860 INFO:teuthology.orchestra.run.trial131.stdout: osd scrub load threshold = 5.0 2026-02-20T12:09:57.860 INFO:teuthology.orchestra.run.trial131.stdout: osd scrub max interval = 600 2026-02-20T12:09:57.861 INFO:teuthology.orchestra.run.trial131.stdout: osd mclock profile = high_recovery_ops 2026-02-20T12:09:57.861 INFO:teuthology.orchestra.run.trial131.stdout: osd mclock skip benchmark = true 2026-02-20T12:09:57.861 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.861 INFO:teuthology.orchestra.run.trial131.stdout: osd recover clone overlap = true 2026-02-20T12:09:57.861 INFO:teuthology.orchestra.run.trial131.stdout: osd recovery max chunk = 1048576 2026-02-20T12:09:57.861 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.861 INFO:teuthology.orchestra.run.trial131.stdout: osd debug shutdown = true 2026-02-20T12:09:57.861 INFO:teuthology.orchestra.run.trial131.stdout: osd debug op order = true 2026-02-20T12:09:57.861 INFO:teuthology.orchestra.run.trial131.stdout: osd debug verify stray on activate = true 2026-02-20T12:09:57.861 INFO:teuthology.orchestra.run.trial131.stdout: osd debug trim objects = true 2026-02-20T12:09:57.861 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.861 INFO:teuthology.orchestra.run.trial131.stdout: osd open classes on start = true 2026-02-20T12:09:57.862 INFO:teuthology.orchestra.run.trial131.stdout: osd debug pg log writeout = true 2026-02-20T12:09:57.862 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.862 INFO:teuthology.orchestra.run.trial131.stdout: osd deep scrub update digest min age = 30 2026-02-20T12:09:57.862 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.862 INFO:teuthology.orchestra.run.trial131.stdout: osd map max advance = 10 2026-02-20T12:09:57.862 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.862 INFO:teuthology.orchestra.run.trial131.stdout: journal zero on create = true 2026-02-20T12:09:57.862 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.862 INFO:teuthology.orchestra.run.trial131.stdout: filestore ondisk finisher threads = 3 2026-02-20T12:09:57.862 INFO:teuthology.orchestra.run.trial131.stdout: filestore apply finisher threads = 3 2026-02-20T12:09:57.862 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.862 INFO:teuthology.orchestra.run.trial131.stdout: bdev debug aio = true 2026-02-20T12:09:57.862 INFO:teuthology.orchestra.run.trial131.stdout: osd debug misdirected ops = true 2026-02-20T12:09:57.863 INFO:teuthology.orchestra.run.trial131.stdout: debug monc = 20 2026-02-20T12:09:57.863 INFO:teuthology.orchestra.run.trial131.stdout: crimson cpu num = 4 2026-02-20T12:09:57.863 INFO:teuthology.orchestra.run.trial131.stdout: crimson osd obc lru size = 10 2026-02-20T12:09:57.863 INFO:teuthology.orchestra.run.trial131.stdout: debug filestore = 0/0 2026-02-20T12:09:57.863 INFO:teuthology.orchestra.run.trial131.stdout: debug journal = 0/0 2026-02-20T12:09:57.863 INFO:teuthology.orchestra.run.trial131.stdout: debug ms = 0/0 2026-02-20T12:09:57.863 INFO:teuthology.orchestra.run.trial131.stdout: debug osd = 0/0 2026-02-20T12:09:57.863 INFO:teuthology.orchestra.run.trial131.stdout: debug seastore = 20 2026-02-20T12:09:57.863 INFO:teuthology.orchestra.run.trial131.stdout: debug seastore backref = 20 2026-02-20T12:09:57.863 INFO:teuthology.orchestra.run.trial131.stdout: debug seastore cache = 20 2026-02-20T12:09:57.863 INFO:teuthology.orchestra.run.trial131.stdout: debug seastore cleaner = 20 2026-02-20T12:09:57.863 INFO:teuthology.orchestra.run.trial131.stdout: debug seastore device = 20 2026-02-20T12:09:57.863 INFO:teuthology.orchestra.run.trial131.stdout: debug seastore epm = 20 2026-02-20T12:09:57.863 INFO:teuthology.orchestra.run.trial131.stdout: debug seastore fixedkv tree = 20 2026-02-20T12:09:57.864 INFO:teuthology.orchestra.run.trial131.stdout: debug seastore journal = 20 2026-02-20T12:09:57.864 INFO:teuthology.orchestra.run.trial131.stdout: debug seastore lba = 20 2026-02-20T12:09:57.864 INFO:teuthology.orchestra.run.trial131.stdout: debug seastore odata = 20 2026-02-20T12:09:57.864 INFO:teuthology.orchestra.run.trial131.stdout: debug seastore omap = 20 2026-02-20T12:09:57.864 INFO:teuthology.orchestra.run.trial131.stdout: debug seastore onode = 20 2026-02-20T12:09:57.864 INFO:teuthology.orchestra.run.trial131.stdout: debug seastore t = 20 2026-02-20T12:09:57.864 INFO:teuthology.orchestra.run.trial131.stdout: debug seastore tm = 20 2026-02-20T12:09:57.864 INFO:teuthology.orchestra.run.trial131.stdout: osd objectstore = seastore 2026-02-20T12:09:57.864 INFO:teuthology.orchestra.run.trial131.stdout: osd shutdown pgref assert = True 2026-02-20T12:09:57.864 INFO:teuthology.orchestra.run.trial131.stdout: seastore cachepin size pershard = 64M 2026-02-20T12:09:57.864 INFO:teuthology.orchestra.run.trial131.stdout: seastore main device type = RANDOM_BLOCK_SSD 2026-02-20T12:09:57.864 INFO:teuthology.orchestra.run.trial131.stdout: seastore max concurrent transactions = 8 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout:[mgr] 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: debug ms = 1 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: debug mgr = 20 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: debug mon = 20 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: debug auth = 20 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: mon reweight min pgs per osd = 4 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: mon reweight min bytes per osd = 10 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: mgr/telemetry/nag = false 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout:[mon] 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: debug ms = 0/0 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: debug mon = 0/0 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: debug paxos = 0/0 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: debug auth = 20 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: mon data avail warn = 5 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: mon mgr mkfs grace = 240 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: mon reweight min pgs per osd = 4 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: mon osd reporter subtree level = osd 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: mon osd prime pg temp = true 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: mon reweight min bytes per osd = 10 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: auth mon ticket ttl = 660 # 11m 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: auth service ticket ttl = 240 # 4m 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: # don't complain about insecure global_id in the test suite 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: # 1m isn't quite enough 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: mon_down_mkfs_grace = 2m 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: mon_warn_on_filestore_osds = false 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: osd pool default crimson = True 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:09:57.865 INFO:teuthology.orchestra.run.trial131.stdout:[client] 2026-02-20T12:09:57.866 INFO:teuthology.orchestra.run.trial131.stdout: rgw cache enabled = true 2026-02-20T12:09:57.866 INFO:teuthology.orchestra.run.trial131.stdout: rgw enable ops log = true 2026-02-20T12:09:57.866 INFO:teuthology.orchestra.run.trial131.stdout: rgw enable usage log = true 2026-02-20T12:09:57.866 INFO:teuthology.orchestra.run.trial131.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-20T12:09:57.866 INFO:teuthology.orchestra.run.trial131.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-20T12:09:57.866 INFO:teuthology.orchestra.run.trial131.stdout:[mon.a] 2026-02-20T12:09:57.866 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-20T12:09:57.866 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-20T12:09:57.957 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-20T12:09:57.957 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:09:57.957 DEBUG:teuthology.orchestra.run.trial131:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-20T12:09:57.973 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:09:57.973 DEBUG:teuthology.orchestra.run.trial131:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-20T12:09:58.029 INFO:tasks.ceph:Sending monmap to node ubuntu@trial131.front.sepia.ceph.com 2026-02-20T12:09:58.030 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:09:58.030 DEBUG:teuthology.orchestra.run.trial131:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-20T12:09:58.030 DEBUG:teuthology.orchestra.run.trial131:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-20T12:09:58.104 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:09:58.104 DEBUG:teuthology.orchestra.run.trial131:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-20T12:09:58.160 INFO:tasks.ceph:Setting up mon nodes... 2026-02-20T12:09:58.160 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-20T12:09:58.161 DEBUG:teuthology.orchestra.run.trial131:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-20T12:09:58.252 INFO:teuthology.orchestra.run.trial131.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-20T12:09:58.255 INFO:tasks.ceph:Setting up mds nodes... 2026-02-20T12:09:58.256 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-20T12:09:58.256 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-20T12:09:58.298 INFO:teuthology.orchestra.run.trial131.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-20T12:09:58.311 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-20T12:09:58.311 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial131.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-20T12:09:58.311 DEBUG:teuthology.orchestra.run.trial131:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-20T12:09:58.375 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T12:09:58.375 INFO:tasks.ceph:role: osd.0 2026-02-20T12:09:58.375 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial131.front.sepia.ceph.com 2026-02-20T12:09:58.375 DEBUG:teuthology.orchestra.run.trial131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-20T12:09:58.441 INFO:teuthology.orchestra.run.trial131.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T12:09:58.441 INFO:teuthology.orchestra.run.trial131.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T12:09:58.441 INFO:teuthology.orchestra.run.trial131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T12:09:58.441 INFO:teuthology.orchestra.run.trial131.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T12:09:58.441 INFO:teuthology.orchestra.run.trial131.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T12:09:58.441 INFO:teuthology.orchestra.run.trial131.stdout: = sunit=0 swidth=0 blks 2026-02-20T12:09:58.441 INFO:teuthology.orchestra.run.trial131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T12:09:58.441 INFO:teuthology.orchestra.run.trial131.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T12:09:58.441 INFO:teuthology.orchestra.run.trial131.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T12:09:58.441 INFO:teuthology.orchestra.run.trial131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T12:09:59.012 INFO:teuthology.orchestra.run.trial131.stdout:Discarding blocks...Done. 2026-02-20T12:09:59.014 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial131.front.sepia.ceph.com -o noatime 2026-02-20T12:09:59.014 DEBUG:teuthology.orchestra.run.trial131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-20T12:09:59.083 DEBUG:teuthology.orchestra.run.trial131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-20T12:09:59.151 DEBUG:teuthology.orchestra.run.trial131:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-20T12:09:59.216 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T12:09:59.216 INFO:tasks.ceph:role: osd.1 2026-02-20T12:09:59.216 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial131.front.sepia.ceph.com 2026-02-20T12:09:59.216 DEBUG:teuthology.orchestra.run.trial131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-20T12:09:59.280 INFO:teuthology.orchestra.run.trial131.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T12:09:59.281 INFO:teuthology.orchestra.run.trial131.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T12:09:59.281 INFO:teuthology.orchestra.run.trial131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T12:09:59.281 INFO:teuthology.orchestra.run.trial131.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T12:09:59.281 INFO:teuthology.orchestra.run.trial131.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T12:09:59.281 INFO:teuthology.orchestra.run.trial131.stdout: = sunit=0 swidth=0 blks 2026-02-20T12:09:59.281 INFO:teuthology.orchestra.run.trial131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T12:09:59.281 INFO:teuthology.orchestra.run.trial131.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T12:09:59.281 INFO:teuthology.orchestra.run.trial131.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T12:09:59.281 INFO:teuthology.orchestra.run.trial131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T12:09:59.839 INFO:teuthology.orchestra.run.trial131.stdout:Discarding blocks...Done. 2026-02-20T12:09:59.841 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial131.front.sepia.ceph.com -o noatime 2026-02-20T12:09:59.841 DEBUG:teuthology.orchestra.run.trial131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-20T12:09:59.914 DEBUG:teuthology.orchestra.run.trial131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-20T12:09:59.985 DEBUG:teuthology.orchestra.run.trial131:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-20T12:10:00.049 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T12:10:00.049 INFO:tasks.ceph:role: osd.2 2026-02-20T12:10:00.050 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial131.front.sepia.ceph.com 2026-02-20T12:10:00.050 DEBUG:teuthology.orchestra.run.trial131:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-20T12:10:00.115 INFO:teuthology.orchestra.run.trial131.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T12:10:00.115 INFO:teuthology.orchestra.run.trial131.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T12:10:00.115 INFO:teuthology.orchestra.run.trial131.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T12:10:00.115 INFO:teuthology.orchestra.run.trial131.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T12:10:00.116 INFO:teuthology.orchestra.run.trial131.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T12:10:00.116 INFO:teuthology.orchestra.run.trial131.stdout: = sunit=0 swidth=0 blks 2026-02-20T12:10:00.116 INFO:teuthology.orchestra.run.trial131.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T12:10:00.116 INFO:teuthology.orchestra.run.trial131.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T12:10:00.116 INFO:teuthology.orchestra.run.trial131.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T12:10:00.116 INFO:teuthology.orchestra.run.trial131.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T12:10:00.672 INFO:teuthology.orchestra.run.trial131.stdout:Discarding blocks...Done. 2026-02-20T12:10:00.674 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial131.front.sepia.ceph.com -o noatime 2026-02-20T12:10:00.675 DEBUG:teuthology.orchestra.run.trial131:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-20T12:10:00.742 DEBUG:teuthology.orchestra.run.trial131:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-20T12:10:00.808 DEBUG:teuthology.orchestra.run.trial131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T12:10:00.958 INFO:teuthology.orchestra.run.trial131.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T12:10:01.020 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:01,020 seastar - Reactor backend: linux-aio 2026-02-20T12:10:01.038 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:01,038 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T12:10:01.038 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:01,038 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T12:10:01.040 INFO:teuthology.orchestra.run.trial131.stderr:==39251==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T12:10:01.099 INFO:teuthology.orchestra.run.trial131.stderr:DEBUG 2026-02-20 12:10:01,099 [shard 0:main] seastar - reactor::drain 2026-02-20T12:10:03.160 INFO:teuthology.orchestra.run.trial131.stderr:----------------------------------------------------- 2026-02-20T12:10:03.160 INFO:teuthology.orchestra.run.trial131.stderr:Suppressions used: 2026-02-20T12:10:03.161 INFO:teuthology.orchestra.run.trial131.stderr: count bytes template 2026-02-20T12:10:03.161 INFO:teuthology.orchestra.run.trial131.stderr: 1 8 MallocExtension::Register 2026-02-20T12:10:03.161 INFO:teuthology.orchestra.run.trial131.stderr:----------------------------------------------------- 2026-02-20T12:10:03.161 INFO:teuthology.orchestra.run.trial131.stderr: 2026-02-20T12:10:03.179 INFO:teuthology.orchestra.run.trial131.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T12:10:03.179 INFO:teuthology.orchestra.run.trial131.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T12:10:03.225 INFO:teuthology.orchestra.run.trial131.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T12:10:03.225 INFO:teuthology.orchestra.run.trial131.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T12:10:03.232 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:03,232 seastar - Reactor backend: linux-aio 2026-02-20T12:10:03.240 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:03,240 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T12:10:03.240 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:03,240 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T12:10:03.244 INFO:teuthology.orchestra.run.trial131.stderr:==39249==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T12:10:03.244 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:03,244 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T12:10:03.382 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x10dd764 0x10dd7ff 0x993a077 0x993a541 0x993a7b4 0x993c4db 0x993ca28 0x993cc27 0x9a4ee89 0x9a4f5e4 0x9a59876 0x9a5dcef 0x99c20b5 0x99c10b5 0x7b60901 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f6f022 0x6f72e23 0x6f789cd 0x73e30d9 0x7412614 0x7256527 0x725661f 0x732cdbe 0x732ce77 0x732d02e 0x7412695 0x725669c 0x7256721 0x728124b 0x73ebced 0x73ebd72 0x73ebe50 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:03.383 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9808b5b 0x9a3729a 0x9a373c8 0x9a54fa3 0x9a55746 0x9a5fc13 0x786f2eb 0x7b60233 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f6f022 0x6f72e23 0x6f789cd 0x73e30d9 0x7412614 0x7256527 0x725661f 0x732cdbe 0x732ce77 0x732d02e 0x7412695 0x725669c 0x7256721 0x728124b 0x73ebced 0x73ebd72 0x73ebe50 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:03.383 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc9e9 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9808b5b 0x98beab2 0x98bebc8 0x98beca4 0x98bef20 0x98befd9 0x98bf4c7 0x98bf8cb 0x98bfb88 0x98c0646 0x9a3fbc0 0x9a3fcb9 0x7b60270 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f6f022 0x6f72e23 0x6f789cd 0x73e30d9 0x7412614 0x7256527 0x725661f 0x732cdbe 0x732ce77 0x732d02e 0x7412695 0x725669c 0x7256721 0x728124b 0x73ebced 0x73ebd72 0x73ebe50 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:03.420 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f 0x999878a 0x9a4a752 0x9a5a88c 0x9a5dcef 0x99c20b5 0x99c10b5 0x7b60901 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f6fe0e 0x7510654 0x75107a8 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:03.531 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xd91811 0x9a5f433 0x786f2eb 0x7b61657 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f0b27c 0x6f0d5e4 0x73e2d78 0x740ccd4 0x7254c6e 0x7254d66 0x732c65e 0x732c717 0x732c8ce 0x740cd55 0x7254de3 0x7254e68 0x727c2ab 0x73ea73c 0x73ea7c1 0x73ea89f 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:03.531 INFO:teuthology.orchestra.run.trial131.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffa6527ef7 0xffffffffa61f7f16 0xffffffffa61fb8d0 0xffffffffa61fcc31 0xffffffffa6224893 0xffffffffa6224a55 0xffffffffa61c97c8 0xffffffffa61d3aeb 0xffffffffa61d3f4b 0xffffffffa5e910d2 0xffffffffa6b4d451 0xffffffffa6c00ba2 2026-02-20T12:10:03.531 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x10dd764 0x10dd7ff 0x993a077 0x993a541 0x993a7b4 0x993c4db 0x9a4cdf2 0x9a4ea44 0x9a4f09e 0x9a4f5e4 0x9a59876 0x9a5dcef 0x99c20b5 0x99c10b5 0x7b61702 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f0b27c 0x6f0d5e4 0x73e2d78 0x740ccd4 0x7254c6e 0x7254d66 0x732c65e 0x732c717 0x732c8ce 0x740cd55 0x7254de3 0x7254e68 0x727c2ab 0x73ea73c 0x73ea7c1 0x73ea89f 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:03.531 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb052c /lib64/libasan.so.8+0xb1437 0xb3a067 0x1022af4 0x10dd80a 0x993a077 0x993a541 0x993a7b4 0x993aa6a 0x993b02c 0x9940151 0x9940831 0x9a59644 0x9a5dcef 0x99c20b5 0x99c10b5 0x7b60901 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f0b27c 0x6f0d5e4 0x73e2d78 0x740ccd4 0x7254c6e 0x7254d66 0x732c65e 0x732c717 0x732c8ce 0x740cd55 0x7254de3 0x7254e68 0x727c2ab 0x73ea73c 0x73ea7c1 0x73ea89f 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:05.655 INFO:teuthology.orchestra.run.trial131.stderr:----------------------------------------------------- 2026-02-20T12:10:05.655 INFO:teuthology.orchestra.run.trial131.stderr:Suppressions used: 2026-02-20T12:10:05.655 INFO:teuthology.orchestra.run.trial131.stderr: count bytes template 2026-02-20T12:10:05.655 INFO:teuthology.orchestra.run.trial131.stderr: 1 8 MallocExtension::Register 2026-02-20T12:10:05.655 INFO:teuthology.orchestra.run.trial131.stderr:----------------------------------------------------- 2026-02-20T12:10:05.656 INFO:teuthology.orchestra.run.trial131.stderr: 2026-02-20T12:10:05.687 DEBUG:teuthology.orchestra.run.trial131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-20T12:10:05.713 DEBUG:teuthology.orchestra.run.trial131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T12:10:05.865 INFO:teuthology.orchestra.run.trial131.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T12:10:05.922 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:05,922 seastar - Reactor backend: linux-aio 2026-02-20T12:10:05.932 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:05,932 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T12:10:05.932 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:05,932 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T12:10:05.934 INFO:teuthology.orchestra.run.trial131.stderr:==39316==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T12:10:05.993 INFO:teuthology.orchestra.run.trial131.stderr:DEBUG 2026-02-20 12:10:05,993 [shard 0:main] seastar - reactor::drain 2026-02-20T12:10:08.031 INFO:teuthology.orchestra.run.trial131.stderr:----------------------------------------------------- 2026-02-20T12:10:08.031 INFO:teuthology.orchestra.run.trial131.stderr:Suppressions used: 2026-02-20T12:10:08.031 INFO:teuthology.orchestra.run.trial131.stderr: count bytes template 2026-02-20T12:10:08.031 INFO:teuthology.orchestra.run.trial131.stderr: 1 8 MallocExtension::Register 2026-02-20T12:10:08.031 INFO:teuthology.orchestra.run.trial131.stderr:----------------------------------------------------- 2026-02-20T12:10:08.031 INFO:teuthology.orchestra.run.trial131.stderr: 2026-02-20T12:10:08.051 INFO:teuthology.orchestra.run.trial131.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T12:10:08.052 INFO:teuthology.orchestra.run.trial131.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T12:10:08.094 INFO:teuthology.orchestra.run.trial131.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T12:10:08.094 INFO:teuthology.orchestra.run.trial131.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T12:10:08.101 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:08,101 seastar - Reactor backend: linux-aio 2026-02-20T12:10:08.114 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:08,114 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T12:10:08.114 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:08,114 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T12:10:08.118 INFO:teuthology.orchestra.run.trial131.stderr:==39314==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T12:10:08.118 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:08,118 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T12:10:08.256 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f 0xc80be1 0x1022a48 0x10dd80a 0x993a077 0x993a541 0x993a7b4 0x993aa6a 0x993b519 0x993ba17 0x993c047 0x993c1b1 0x9a49461 0x9a5c268 0x99c20b5 0x99c10b5 0x7b60901 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f6f022 0x6f72e23 0x6f789cd 0x73e30d9 0x7412614 0x7256527 0x725661f 0x732cdbe 0x732ce77 0x732d02e 0x7412695 0x725669c 0x7256721 0x728124b 0x73ebced 0x73ebd72 0x73ebe50 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:08.256 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15ef5 0x993a402 0x993a7b4 0x993aa6a 0x993b519 0x993ba17 0x993c047 0x993c1b1 0x9a49461 0x9a5c268 0x99c20b5 0x99c10b5 0x7b60901 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f6f022 0x6f72e23 0x6f789cd 0x73e30d9 0x7412614 0x7256527 0x725661f 0x732cdbe 0x732ce77 0x732d02e 0x7412695 0x725669c 0x7256721 0x728124b 0x73ebced 0x73ebd72 0x73ebe50 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:08.256 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x10069f /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x10dd764 0x10dd7ff 0x993a244 0x993a541 0x993a7b4 0x993c4db 0x993ca28 0x993cc27 0x9a4ee89 0x9a4f5e4 0x9a59876 0x9a5dcef 0x99c20b5 0x99c10b5 0x7b60901 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f6f022 0x6f72e23 0x6f789cd 0x73e30d9 0x7412614 0x7256527 0x725661f 0x732cdbe 0x732ce77 0x732d02e 0x7412695 0x725669c 0x7256721 0x728124b 0x73ebced 0x73ebd72 0x73ebe50 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:08.402 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119517 /lib64/libasan.so.8+0xec7bb 0xc80a85 0x786f34d 0x7b60233 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f0b27c 0x6f0d5e4 0x73e2d78 0x740ccd4 0x7254c6e 0x7254d66 0x732c65e 0x732c717 0x732c8ce 0x740cd55 0x7254de3 0x7254e68 0x727c2ab 0x73ea73c 0x73ea7c1 0x73ea89f 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:08.402 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f 0x1022c0d 0x10dd80a 0x993a077 0x993a541 0x993a7b4 0x993aa6a 0x993b519 0x993ba17 0x993c047 0x993c1b1 0x9a49461 0x9a5c268 0x99c20b5 0x99c10b5 0x7b60901 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f0b27c 0x6f0d5e4 0x73e2d78 0x740ccd4 0x7254c6e 0x7254d66 0x732c65e 0x732c717 0x732c8ce 0x740cd55 0x7254de3 0x7254e68 0x727c2ab 0x73ea73c 0x73ea7c1 0x73ea89f 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:08.403 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xeaeb7 0x35d1849 0x9a410be 0x9a41453 0x9a41874 0x9a59007 0x9a5dcef 0x99c20b5 0x99c10b5 0x7b60901 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f0b27c 0x6f0d5e4 0x73e2d78 0x740ccd4 0x7254c6e 0x7254d66 0x732c65e 0x732c717 0x732c8ce 0x740cd55 0x7254de3 0x7254e68 0x727c2ab 0x73ea73c 0x73ea7c1 0x73ea89f 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:10.533 INFO:teuthology.orchestra.run.trial131.stderr:----------------------------------------------------- 2026-02-20T12:10:10.534 INFO:teuthology.orchestra.run.trial131.stderr:Suppressions used: 2026-02-20T12:10:10.534 INFO:teuthology.orchestra.run.trial131.stderr: count bytes template 2026-02-20T12:10:10.534 INFO:teuthology.orchestra.run.trial131.stderr: 1 8 MallocExtension::Register 2026-02-20T12:10:10.534 INFO:teuthology.orchestra.run.trial131.stderr:----------------------------------------------------- 2026-02-20T12:10:10.534 INFO:teuthology.orchestra.run.trial131.stderr: 2026-02-20T12:10:10.565 DEBUG:teuthology.orchestra.run.trial131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-20T12:10:10.591 DEBUG:teuthology.orchestra.run.trial131:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T12:10:10.742 INFO:teuthology.orchestra.run.trial131.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T12:10:10.795 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:10,795 seastar - Reactor backend: linux-aio 2026-02-20T12:10:10.813 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:10,813 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T12:10:10.813 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:10,813 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T12:10:10.815 INFO:teuthology.orchestra.run.trial131.stderr:==39380==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T12:10:10.875 INFO:teuthology.orchestra.run.trial131.stderr:DEBUG 2026-02-20 12:10:10,875 [shard 0:main] seastar - reactor::drain 2026-02-20T12:10:12.929 INFO:teuthology.orchestra.run.trial131.stderr:----------------------------------------------------- 2026-02-20T12:10:12.929 INFO:teuthology.orchestra.run.trial131.stderr:Suppressions used: 2026-02-20T12:10:12.929 INFO:teuthology.orchestra.run.trial131.stderr: count bytes template 2026-02-20T12:10:12.929 INFO:teuthology.orchestra.run.trial131.stderr: 1 8 MallocExtension::Register 2026-02-20T12:10:12.929 INFO:teuthology.orchestra.run.trial131.stderr:----------------------------------------------------- 2026-02-20T12:10:12.929 INFO:teuthology.orchestra.run.trial131.stderr: 2026-02-20T12:10:12.949 INFO:teuthology.orchestra.run.trial131.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T12:10:12.949 INFO:teuthology.orchestra.run.trial131.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T12:10:12.993 INFO:teuthology.orchestra.run.trial131.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T12:10:12.993 INFO:teuthology.orchestra.run.trial131.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T12:10:13.000 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:13,000 seastar - Reactor backend: linux-aio 2026-02-20T12:10:13.014 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:13,014 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T12:10:13.015 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:13,014 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T12:10:13.018 INFO:teuthology.orchestra.run.trial131.stderr:==39378==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T12:10:13.018 INFO:teuthology.orchestra.run.trial131.stderr:INFO 2026-02-20 12:10:13,018 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T12:10:13.156 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x10dd764 0x10dd7ff 0x9a4fe76 0x9a55108 0x9a55746 0x9a6003c 0x786f2eb 0x7b60838 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f6f022 0x6f72e23 0x6f789cd 0x73e30d9 0x7412614 0x7256527 0x725661f 0x732cdbe 0x732ce77 0x732d02e 0x7412695 0x725669c 0x7256721 0x728124b 0x73ebced 0x73ebd72 0x73ebe50 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:13.156 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x157c7 0x1022b6b 0x10dd80a 0x993a244 0x993a541 0x993a7b4 0x993aa6a 0x993b519 0x993ba17 0x993c047 0x993c1b1 0x9a49461 0x9a5c268 0x99c20b5 0x99c10b5 0x7b60901 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f6f022 0x6f72e23 0x6f789cd 0x73e30d9 0x7412614 0x7256527 0x725661f 0x732cdbe 0x732ce77 0x732d02e 0x7412695 0x725669c 0x7256721 0x728124b 0x73ebced 0x73ebd72 0x73ebe50 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:13.156 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x157c7 0x1022b6b 0x10dd80a 0x9a49dc1 0x9a4a3ae 0x9a4a731 0x9a5a88c 0x9a5dcef 0x99c20b5 0x99c10b5 0x7b60901 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f6f022 0x6f72e23 0x6f789cd 0x73e30d9 0x7412614 0x7256527 0x725661f 0x732cdbe 0x732ce77 0x732d02e 0x7412695 0x725669c 0x7256721 0x728124b 0x73ebced 0x73ebd72 0x73ebe50 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:13.300 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x11908d /lib64/libasan.so.8+0xebbf /lib64/libasan.so.8+0xec776 0xc80a85 0x3135771 0x3135ad7 0x3135bcc 0x9a5c285 0x99c20b5 0x99c10b5 0x7b60901 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f0b27c 0x6f0d5e4 0x73e2d78 0x740ccd4 0x7254c6e 0x7254d66 0x732c65e 0x732c717 0x732c8ce 0x740cd55 0x7254de3 0x7254e68 0x727c2ab 0x73ea73c 0x73ea7c1 0x73ea89f 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:13.301 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1294d2 /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x993a0e0 0x993a541 0x993a7b4 0x993c4db 0x993ca28 0x993cc27 0x9a4ee89 0x9a4f5e4 0x9a59876 0x9a5dcef 0x99c20b5 0x99c10b5 0x7b60901 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f0b27c 0x6f0d5e4 0x73e2d78 0x740ccd4 0x7254c6e 0x7254d66 0x732c65e 0x732c717 0x732c8ce 0x740cd55 0x7254de3 0x7254e68 0x727c2ab 0x73ea73c 0x73ea7c1 0x73ea89f 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:13.301 INFO:teuthology.orchestra.run.trial131.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8bb26 0x9abe76d 0x9af96d3 0x9afcc8c 0x9afcefa 0x9afd042 0x9afd501 /lib64/libc.so.6+0x3fc2f 0x10dd6d0 0x10dd7ff 0x993a244 0x993a541 0x993a7b4 0x993aa6a 0x993b519 0x993ba17 0x993c047 0x993c1b1 0x9a49461 0x9a5c268 0x99c20b5 0x99c10b5 0x7b60901 0x7b6a470 0x7a0ec75 0x7a27c05 0x70e904b 0x6f0b27c 0x6f0d5e4 0x73e2d78 0x740ccd4 0x7254c6e 0x7254d66 0x732c65e 0x732c717 0x732c8ce 0x740cd55 0x7254de3 0x7254e68 0x727c2ab 0x73ea73c 0x73ea7c1 0x73ea89f 0x740ced4 0x7387fb8 0x7552c41 0x73f3ad4 0x73f3b4f 0x74a8152 0x759b55c 0x9bc0b9d 0x9ac6d0c 0x9b7f759 0x9bcfcf7 0x9d147e2 0x9d14e5e 0x265f87f 0xa34f610 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T12:10:15.431 INFO:teuthology.orchestra.run.trial131.stderr:----------------------------------------------------- 2026-02-20T12:10:15.431 INFO:teuthology.orchestra.run.trial131.stderr:Suppressions used: 2026-02-20T12:10:15.431 INFO:teuthology.orchestra.run.trial131.stderr: count bytes template 2026-02-20T12:10:15.431 INFO:teuthology.orchestra.run.trial131.stderr: 1 8 MallocExtension::Register 2026-02-20T12:10:15.431 INFO:teuthology.orchestra.run.trial131.stderr:----------------------------------------------------- 2026-02-20T12:10:15.431 INFO:teuthology.orchestra.run.trial131.stderr: 2026-02-20T12:10:15.463 DEBUG:teuthology.orchestra.run.trial131:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-20T12:10:15.492 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-20T12:10:15.492 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:10:15.492 DEBUG:teuthology.orchestra.run.trial131:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-20T12:10:15.557 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:10:15.557 DEBUG:teuthology.orchestra.run.trial131:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-20T12:10:15.624 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:10:15.624 DEBUG:teuthology.orchestra.run.trial131:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-20T12:10:15.689 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:10:15.689 DEBUG:teuthology.orchestra.run.trial131:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-20T12:10:15.752 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:10:15.752 DEBUG:teuthology.orchestra.run.trial131:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-20T12:10:15.809 INFO:tasks.ceph:Adding keys to all mons... 2026-02-20T12:10:15.809 DEBUG:teuthology.orchestra.run.trial131:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-20T12:10:15.871 INFO:teuthology.orchestra.run.trial131.stdout:[mgr.x] 2026-02-20T12:10:15.871 INFO:teuthology.orchestra.run.trial131.stdout: key = AQAWT5hpSoERDxAAx1cnWX/j5kx6XRUvJ3FCLg== 2026-02-20T12:10:15.871 INFO:teuthology.orchestra.run.trial131.stdout:[osd.0] 2026-02-20T12:10:15.871 INFO:teuthology.orchestra.run.trial131.stdout: key = AQAbT5hpFkNbEhAAH94HQ2LMspLI1qfKwJF+xw== 2026-02-20T12:10:15.871 INFO:teuthology.orchestra.run.trial131.stdout:[osd.1] 2026-02-20T12:10:15.872 INFO:teuthology.orchestra.run.trial131.stdout: key = AQAgT5hpENHfChAAtTCXsIc7Q+Kw0SyswVkD/g== 2026-02-20T12:10:15.872 INFO:teuthology.orchestra.run.trial131.stdout:[osd.2] 2026-02-20T12:10:15.872 INFO:teuthology.orchestra.run.trial131.stdout: key = AQAlT5hpQcXqBBAANiY/ii3lIyz5fLEBzUU8cw== 2026-02-20T12:10:15.872 INFO:teuthology.orchestra.run.trial131.stdout:[client.0] 2026-02-20T12:10:15.872 INFO:teuthology.orchestra.run.trial131.stdout: key = AQAWT5hpzQbJERAA8gyZi99L05oY3poHGlJuAw== 2026-02-20T12:10:15.873 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-20T12:10:15.962 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T12:10:16.010 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T12:10:16.058 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T12:10:16.105 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-20T12:10:16.156 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-20T12:10:16.156 DEBUG:teuthology.orchestra.run.trial131:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-20T12:10:16.181 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-20T12:10:16.266 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:16.266+0000 7f23a68a9c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:16.266 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:16.266+0000 7f23a68a9c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:16.273 DEBUG:teuthology.orchestra.run.trial131:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-20T12:10:16.298 DEBUG:teuthology.orchestra.run.trial131:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-20T12:10:16.353 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-20T12:10:16.354 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-20T12:10:16.354 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-20T12:10:16.395 INFO:tasks.ceph.mon.a:Started 2026-02-20T12:10:16.396 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-20T12:10:16.396 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-20T12:10:16.396 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-20T12:10:16.398 INFO:tasks.ceph.mgr.x:Started 2026-02-20T12:10:16.399 DEBUG:tasks.ceph:set 0 configs 2026-02-20T12:10:16.399 DEBUG:teuthology.orchestra.run.trial131:> sudo ceph --cluster ceph config dump 2026-02-20T12:10:16.458 INFO:tasks.ceph.mon.a.trial131.stderr:2026-02-20T12:10:16.457+0000 7fe157295c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:16.459 INFO:tasks.ceph.mon.a.trial131.stderr:2026-02-20T12:10:16.458+0000 7fe157295c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:16.459 INFO:tasks.ceph.mon.a.trial131.stderr:2026-02-20T12:10:16.458+0000 7fe157295c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:16.462 INFO:tasks.ceph.mgr.x.trial131.stderr:2026-02-20T12:10:16.461+0000 7faa0c210fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:16.474 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:16.471+0000 7f17b808b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:16.477 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:16.477+0000 7f17b808b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:16.508 INFO:teuthology.orchestra.run.trial131.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-20T12:10:16.516 INFO:tasks.ceph:Setting crush tunables to default 2026-02-20T12:10:16.516 DEBUG:teuthology.orchestra.run.trial131:> sudo ceph --cluster ceph osd crush tunables default 2026-02-20T12:10:16.590 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:16.587+0000 7f2acea5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:16.591 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:16.591+0000 7f2acea5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:16.623 INFO:teuthology.orchestra.run.trial131.stderr:adjusted tunables profile to default 2026-02-20T12:10:16.631 INFO:tasks.ceph:check_enable_crimson: True 2026-02-20T12:10:16.631 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-20T12:10:16.631 DEBUG:teuthology.orchestra.run.trial131:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-20T12:10:16.664 INFO:tasks.ceph.mgr.x.trial131.stderr:2026-02-20T12:10:16.664+0000 7faa0c210fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:16.665 INFO:tasks.ceph.mgr.x.trial131.stderr:2026-02-20T12:10:16.664+0000 7faa0c210fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:16.704 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:16.701+0000 7f00823cc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:16.705 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:16.705+0000 7f00823cc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:16.744 INFO:tasks.ceph.mon.a.trial131.stderr:2026-02-20T12:10:16.743+0000 7fe14e9f1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-20T12:10:16.744 INFO:tasks.ceph.mon.a.trial131.stderr:Please be aware that this feature is experimental, untested, 2026-02-20T12:10:16.744 INFO:tasks.ceph.mon.a.trial131.stderr:unsupported, and may result in data corruption, data loss, 2026-02-20T12:10:16.744 INFO:tasks.ceph.mon.a.trial131.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-20T12:10:16.744 INFO:tasks.ceph.mon.a.trial131.stderr:feature with important data. 2026-02-20T12:10:16.744 INFO:tasks.ceph.mon.a.trial131.stderr: 2026-02-20T12:10:17.498 INFO:tasks.ceph.mgr.x.trial131.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-20T12:10:17.498 INFO:tasks.ceph.mgr.x.trial131.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-20T12:10:17.498 INFO:tasks.ceph.mgr.x.trial131.stderr: from numpy import show_config as show_numpy_config 2026-02-20T12:10:17.633 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-20T12:10:17.634 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:10:17.634 DEBUG:teuthology.orchestra.run.trial131:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-20T12:10:17.698 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:10:17.698 DEBUG:teuthology.orchestra.run.trial131:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-20T12:10:17.763 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:10:17.763 DEBUG:teuthology.orchestra.run.trial131:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-20T12:10:17.827 DEBUG:teuthology.orchestra.run.trial131:> sudo ceph --cluster ceph osd new 0ba07e41-d2b3-49b0-a9c9-ccbc0bd0b2b6 0 2026-02-20T12:10:17.940 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:17.936+0000 7f33563dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:17.941 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:17.941+0000 7f33563dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:17.979 INFO:teuthology.orchestra.run.trial131.stdout:0 2026-02-20T12:10:17.987 DEBUG:teuthology.orchestra.run.trial131:> sudo ceph --cluster ceph osd new 32ca0826-88b0-4f2e-985a-e34757b25783 1 2026-02-20T12:10:18.059 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:18.055+0000 7fbf46e8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:18.060 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:18.060+0000 7fbf46e8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:18.097 INFO:teuthology.orchestra.run.trial131.stdout:1 2026-02-20T12:10:18.106 DEBUG:teuthology.orchestra.run.trial131:> sudo ceph --cluster ceph osd new 578aff15-792e-4506-b8b4-5786652c8a91 2 2026-02-20T12:10:18.179 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:18.176+0000 7faecb5dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:18.181 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:18.180+0000 7faecb5dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:18.217 INFO:teuthology.orchestra.run.trial131.stdout:2 2026-02-20T12:10:18.225 DEBUG:teuthology.orchestra.run.trial131:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-20T12:10:18.254 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-20T12:10:18.254 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-20T12:10:18.295 INFO:tasks.ceph.osd.0:Started 2026-02-20T12:10:18.296 DEBUG:teuthology.orchestra.run.trial131:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-20T12:10:18.326 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-20T12:10:18.326 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-20T12:10:18.368 INFO:tasks.ceph.osd.1:Started 2026-02-20T12:10:18.368 DEBUG:teuthology.orchestra.run.trial131:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-20T12:10:18.395 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-20T12:10:18.395 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-20T12:10:18.438 INFO:tasks.ceph.osd.2:Started 2026-02-20T12:10:18.438 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T12:10:18.515 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:18.512+0000 7fd6f0b6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:18.516 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:18.516+0000 7fd6f0b6f640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:18.548 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:10:18.549 INFO:teuthology.orchestra.run.trial131.stdout:{"epoch":6,"fsid":"7a6ea43f-ce6a-400f-9bcb-948d8c241da2","created":"2026-02-20T12:10:16.473664+0000","modified":"2026-02-20T12:10:18.216891+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T12:10:18.216891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"0ba07e41-d2b3-49b0-a9c9-ccbc0bd0b2b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"32ca0826-88b0-4f2e-985a-e34757b25783","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"578aff15-792e-4506-b8b4-5786652c8a91","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T12:10:18.559 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-20T12:10:18.559 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-20T12:10:18.861 DEBUG:teuthology.orchestra.run.trial131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T12:10:18.928 INFO:teuthology.misc.health.trial131.stderr:2026-02-20T12:10:18.926+0000 7fc7a925a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:18.930 INFO:teuthology.misc.health.trial131.stderr:2026-02-20T12:10:18.929+0000 7fc7a925a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:18.961 INFO:teuthology.misc.health.trial131.stdout: 2026-02-20T12:10:18.962 INFO:teuthology.misc.health.trial131.stdout:{"epoch":6,"fsid":"7a6ea43f-ce6a-400f-9bcb-948d8c241da2","created":"2026-02-20T12:10:16.473664+0000","modified":"2026-02-20T12:10:18.216891+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T12:10:18.216891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"0ba07e41-d2b3-49b0-a9c9-ccbc0bd0b2b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"32ca0826-88b0-4f2e-985a-e34757b25783","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"578aff15-792e-4506-b8b4-5786652c8a91","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T12:10:18.968 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-20T12:10:19.882 INFO:tasks.ceph.mgr.x.trial131.stderr:2026-02-20T12:10:19.882+0000 7fa9276d9640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-20T12:10:25.270 DEBUG:teuthology.orchestra.run.trial131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T12:10:25.340 INFO:teuthology.misc.health.trial131.stderr:2026-02-20T12:10:25.338+0000 7fc601c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:25.341 INFO:teuthology.misc.health.trial131.stderr:2026-02-20T12:10:25.341+0000 7fc601c5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:25.451 INFO:teuthology.misc.health.trial131.stdout: 2026-02-20T12:10:25.451 INFO:teuthology.misc.health.trial131.stdout:{"epoch":8,"fsid":"7a6ea43f-ce6a-400f-9bcb-948d8c241da2","created":"2026-02-20T12:10:16.473664+0000","modified":"2026-02-20T12:10:24.951047+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T12:10:18.216891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"0ba07e41-d2b3-49b0-a9c9-ccbc0bd0b2b6","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"32ca0826-88b0-4f2e-985a-e34757b25783","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"578aff15-792e-4506-b8b4-5786652c8a91","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T12:10:25.459 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-20T12:10:31.762 DEBUG:teuthology.orchestra.run.trial131:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T12:10:31.831 INFO:teuthology.misc.health.trial131.stderr:2026-02-20T12:10:31.828+0000 7f08f31f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:31.832 INFO:teuthology.misc.health.trial131.stderr:2026-02-20T12:10:31.831+0000 7f08f31f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:31.941 INFO:teuthology.misc.health.trial131.stdout: 2026-02-20T12:10:31.941 INFO:teuthology.misc.health.trial131.stdout:{"epoch":13,"fsid":"7a6ea43f-ce6a-400f-9bcb-948d8c241da2","created":"2026-02-20T12:10:16.473664+0000","modified":"2026-02-20T12:10:30.971930+0000","last_up_change":"2026-02-20T12:10:26.958533+0000","last_in_change":"2026-02-20T12:10:18.216891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T12:10:28.892296+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ba07e41-d2b3-49b0-a9c9-ccbc0bd0b2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6804","nonce":798634831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6807","nonce":798634831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6808","nonce":798634831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6809","nonce":798634831}]},"public_addr":"10.20.193.131:6804/798634831","cluster_addr":"10.20.193.131:6807/798634831","heartbeat_back_addr":"10.20.193.131:6808/798634831","heartbeat_front_addr":"10.20.193.131:6809/798634831","state":["exists","up"]},{"osd":1,"uuid":"32ca0826-88b0-4f2e-985a-e34757b25783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6802","nonce":13093902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6803","nonce":13093902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6810","nonce":13093902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6811","nonce":13093902}]},"public_addr":"10.20.193.131:6802/13093902","cluster_addr":"10.20.193.131:6803/13093902","heartbeat_back_addr":"10.20.193.131:6810/13093902","heartbeat_front_addr":"10.20.193.131:6811/13093902","state":["exists","up"]},{"osd":2,"uuid":"578aff15-792e-4506-b8b4-5786652c8a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6805","nonce":1908451256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6806","nonce":1908451256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6813","nonce":1908451256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6812","nonce":1908451256}]},"public_addr":"10.20.193.131:6805/1908451256","cluster_addr":"10.20.193.131:6806/1908451256","heartbeat_back_addr":"10.20.193.131:6813/1908451256","heartbeat_front_addr":"10.20.193.131:6812/1908451256","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T12:10:31.949 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-20T12:10:31.949 INFO:tasks.ceph:Creating RBD pool 2026-02-20T12:10:31.949 DEBUG:teuthology.orchestra.run.trial131:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-20T12:10:32.024 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:32.020+0000 7f49bb01a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:32.025 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:32.024+0000 7f49bb01a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:32.978 INFO:teuthology.orchestra.run.trial131.stderr:pool 'rbd' created 2026-02-20T12:10:32.991 DEBUG:teuthology.orchestra.run.trial131:> rbd --cluster ceph pool init rbd 2026-02-20T12:10:33.032 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:33.031+0000 7f95a71cd240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:33.032 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:33.031+0000 7f95a71cd240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:33.033 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:33.033+0000 7f95a71cd240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:36.011 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-20T12:10:36.012 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-20T12:10:36.013 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-20T12:10:36.129 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:36.126+0000 7fe6603e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:36.130 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:36.129+0000 7fe6603e4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:36.237 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:10:36.246 INFO:teuthology.orchestra.run.trial131.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-20T12:10:36.247 INFO:tasks.ceph_manager:config epoch is 1 2026-02-20T12:10:36.247 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-20T12:10:36.247 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-20T12:10:36.247 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-20T12:10:36.323 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:36.320+0000 7f997f5dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:36.324 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:36.324+0000 7f997f5dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:36.468 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:10:36.478 INFO:teuthology.orchestra.run.trial131.stdout:{"epoch":5,"flags":0,"active_gid":4104,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6800","nonce":1867208602},{"type":"v1","addr":"10.20.193.131:6801","nonce":1867208602}]},"active_addr":"10.20.193.131:6801/1867208602","active_change":"2026-02-20T12:10:18.876129+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.131:0","nonce":4123179717}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.131:0","nonce":1696898735}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.131:0","nonce":2644276504}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.131:0","nonce":920351857}]}]} 2026-02-20T12:10:36.482 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-20T12:10:36.482 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-20T12:10:36.482 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T12:10:36.559 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:36.556+0000 7f8c2bd0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:36.560 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:36.559+0000 7f8c2bd0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:36.667 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:10:36.668 INFO:teuthology.orchestra.run.trial131.stdout:{"epoch":17,"fsid":"7a6ea43f-ce6a-400f-9bcb-948d8c241da2","created":"2026-02-20T12:10:16.473664+0000","modified":"2026-02-20T12:10:35.984014+0000","last_up_change":"2026-02-20T12:10:26.958533+0000","last_in_change":"2026-02-20T12:10:18.216891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T12:10:28.892296+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-20T12:10:32.137096+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ba07e41-d2b3-49b0-a9c9-ccbc0bd0b2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6804","nonce":798634831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6807","nonce":798634831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6808","nonce":798634831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6809","nonce":798634831}]},"public_addr":"10.20.193.131:6804/798634831","cluster_addr":"10.20.193.131:6807/798634831","heartbeat_back_addr":"10.20.193.131:6808/798634831","heartbeat_front_addr":"10.20.193.131:6809/798634831","state":["exists","up"]},{"osd":1,"uuid":"32ca0826-88b0-4f2e-985a-e34757b25783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6802","nonce":13093902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6803","nonce":13093902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6810","nonce":13093902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6811","nonce":13093902}]},"public_addr":"10.20.193.131:6802/13093902","cluster_addr":"10.20.193.131:6803/13093902","heartbeat_back_addr":"10.20.193.131:6810/13093902","heartbeat_front_addr":"10.20.193.131:6811/13093902","state":["exists","up"]},{"osd":2,"uuid":"578aff15-792e-4506-b8b4-5786652c8a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6805","nonce":1908451256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6806","nonce":1908451256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6813","nonce":1908451256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6812","nonce":1908451256}]},"public_addr":"10.20.193.131:6805/1908451256","cluster_addr":"10.20.193.131:6806/1908451256","heartbeat_back_addr":"10.20.193.131:6813/1908451256","heartbeat_front_addr":"10.20.193.131:6812/1908451256","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T12:10:36.677 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-20T12:10:36.677 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T12:10:36.753 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:36.751+0000 7f38cd61a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:36.754 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:36.754+0000 7f38cd61a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:36.866 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:10:36.866 INFO:teuthology.orchestra.run.trial131.stdout:{"epoch":17,"fsid":"7a6ea43f-ce6a-400f-9bcb-948d8c241da2","created":"2026-02-20T12:10:16.473664+0000","modified":"2026-02-20T12:10:35.984014+0000","last_up_change":"2026-02-20T12:10:26.958533+0000","last_in_change":"2026-02-20T12:10:18.216891+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T12:10:28.892296+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-20T12:10:32.137096+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"0ba07e41-d2b3-49b0-a9c9-ccbc0bd0b2b6","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6804","nonce":798634831}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6807","nonce":798634831}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6808","nonce":798634831}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6809","nonce":798634831}]},"public_addr":"10.20.193.131:6804/798634831","cluster_addr":"10.20.193.131:6807/798634831","heartbeat_back_addr":"10.20.193.131:6808/798634831","heartbeat_front_addr":"10.20.193.131:6809/798634831","state":["exists","up"]},{"osd":1,"uuid":"32ca0826-88b0-4f2e-985a-e34757b25783","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6802","nonce":13093902}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6803","nonce":13093902}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6810","nonce":13093902}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6811","nonce":13093902}]},"public_addr":"10.20.193.131:6802/13093902","cluster_addr":"10.20.193.131:6803/13093902","heartbeat_back_addr":"10.20.193.131:6810/13093902","heartbeat_front_addr":"10.20.193.131:6811/13093902","state":["exists","up"]},{"osd":2,"uuid":"578aff15-792e-4506-b8b4-5786652c8a91","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6805","nonce":1908451256}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6806","nonce":1908451256}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6813","nonce":1908451256}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.131:6812","nonce":1908451256}]},"public_addr":"10.20.193.131:6805/1908451256","cluster_addr":"10.20.193.131:6806/1908451256","heartbeat_back_addr":"10.20.193.131:6813/1908451256","heartbeat_front_addr":"10.20.193.131:6812/1908451256","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T12:10:36.877 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-20T12:10:36.877 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-20T12:10:36.878 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-20T12:10:36.958 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:36.955+0000 7ff65030b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:36.959 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:36.955+0000 7f5c4d451640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:36.959 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:36.955+0000 7f4e4017b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:36.959 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:36.959+0000 7f4e4017b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:36.960 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:36.959+0000 7ff65030b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:36.960 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:36.959+0000 7f5c4d451640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:36.974 INFO:teuthology.orchestra.run.trial131.stdout:38654705674 2026-02-20T12:10:36.975 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-20T12:10:36.977 INFO:teuthology.orchestra.run.trial131.stdout:42949672969 2026-02-20T12:10:36.978 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-20T12:10:36.979 INFO:teuthology.orchestra.run.trial131.stdout:42949672969 2026-02-20T12:10:36.980 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-20T12:10:37.097 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:37.094+0000 7f456a8cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:37.098 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:37.094+0000 7f86a8b74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:37.098 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:37.098+0000 7f456a8cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:37.099 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:37.096+0000 7fa4701cc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:37.099 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:37.099+0000 7f86a8b74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:37.100 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:37.100+0000 7fa4701cc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:37.212 INFO:teuthology.orchestra.run.trial131.stdout:38654705674 2026-02-20T12:10:37.214 INFO:teuthology.orchestra.run.trial131.stdout:42949672969 2026-02-20T12:10:37.214 INFO:teuthology.orchestra.run.trial131.stdout:42949672969 2026-02-20T12:10:37.223 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705674 for osd.1 2026-02-20T12:10:37.223 DEBUG:teuthology.parallel:result is None 2026-02-20T12:10:37.225 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672969 for osd.0 2026-02-20T12:10:37.226 DEBUG:teuthology.parallel:result is None 2026-02-20T12:10:37.226 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672969 for osd.2 2026-02-20T12:10:37.226 DEBUG:teuthology.parallel:result is None 2026-02-20T12:10:37.226 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-20T12:10:37.227 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T12:10:37.340 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:37.338+0000 7feb1fc06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:37.341 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:37.341+0000 7feb1fc06640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:37.451 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:10:37.451 INFO:teuthology.orchestra.run.trial131.stderr:dumped all 2026-02-20T12:10:37.459 INFO:teuthology.orchestra.run.trial131.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2026-02-20T12:10:36.968442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62917004,"kb_used_data":62917004,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94369348,"statfs":{"total":161061224448,"available":96634212352,"internally_reserved":0,"allocated":64427012096,"data_stored":64427012096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.995778"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:35.998808+0000","last_change":"2026-02-20T12:10:36.002294+0000","last_active":"2026-02-20T12:10:35.998808+0000","last_peered":"2026-02-20T12:10:35.998808+0000","last_clean":"2026-02-20T12:10:35.998808+0000","last_became_active":"2026-02-20T12:10:34.013044+0000","last_became_peered":"2026-02-20T12:10:34.013044+0000","last_unstale":"2026-02-20T12:10:35.998808+0000","last_undegraded":"2026-02-20T12:10:35.998808+0000","last_fullsized":"2026-02-20T12:10:35.998808+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_clean_scrub_stamp":"2026-02-20T12:10:32.975486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:35.999763+0000","last_change":"2026-02-20T12:10:36.001510+0000","last_active":"2026-02-20T12:10:35.999763+0000","last_peered":"2026-02-20T12:10:35.999763+0000","last_clean":"2026-02-20T12:10:35.999763+0000","last_became_active":"2026-02-20T12:10:34.010781+0000","last_became_peered":"2026-02-20T12:10:34.010781+0000","last_unstale":"2026-02-20T12:10:35.999763+0000","last_undegraded":"2026-02-20T12:10:35.999763+0000","last_fullsized":"2026-02-20T12:10:35.999763+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_clean_scrub_stamp":"2026-02-20T12:10:32.975486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:35.999724+0000","last_change":"2026-02-20T12:10:36.002769+0000","last_active":"2026-02-20T12:10:35.999724+0000","last_peered":"2026-02-20T12:10:35.999724+0000","last_clean":"2026-02-20T12:10:35.999724+0000","last_became_active":"2026-02-20T12:10:34.016759+0000","last_became_peered":"2026-02-20T12:10:34.016759+0000","last_unstale":"2026-02-20T12:10:35.999724+0000","last_undegraded":"2026-02-20T12:10:35.999724+0000","last_fullsized":"2026-02-20T12:10:35.999724+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_clean_scrub_stamp":"2026-02-20T12:10:32.975486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:36.000280+0000","last_change":"2026-02-20T12:10:36.001808+0000","last_active":"2026-02-20T12:10:36.000280+0000","last_peered":"2026-02-20T12:10:36.000280+0000","last_clean":"2026-02-20T12:10:36.000280+0000","last_became_active":"2026-02-20T12:10:34.012723+0000","last_became_peered":"2026-02-20T12:10:34.012723+0000","last_unstale":"2026-02-20T12:10:36.000280+0000","last_undegraded":"2026-02-20T12:10:36.000280+0000","last_fullsized":"2026-02-20T12:10:36.000280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_clean_scrub_stamp":"2026-02-20T12:10:32.975486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:35.999581+0000","last_change":"2026-02-20T12:10:36.000892+0000","last_active":"2026-02-20T12:10:35.999581+0000","last_peered":"2026-02-20T12:10:35.999581+0000","last_clean":"2026-02-20T12:10:35.999581+0000","last_became_active":"2026-02-20T12:10:34.021605+0000","last_became_peered":"2026-02-20T12:10:34.021605+0000","last_unstale":"2026-02-20T12:10:35.999581+0000","last_undegraded":"2026-02-20T12:10:35.999581+0000","last_fullsized":"2026-02-20T12:10:35.999581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_clean_scrub_stamp":"2026-02-20T12:10:32.975486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:35.998610+0000","last_change":"2026-02-20T12:10:36.000803+0000","last_active":"2026-02-20T12:10:35.998610+0000","last_peered":"2026-02-20T12:10:35.998610+0000","last_clean":"2026-02-20T12:10:35.998610+0000","last_became_active":"2026-02-20T12:10:34.016549+0000","last_became_peered":"2026-02-20T12:10:34.016549+0000","last_unstale":"2026-02-20T12:10:35.998610+0000","last_undegraded":"2026-02-20T12:10:35.998610+0000","last_fullsized":"2026-02-20T12:10:35.998610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_clean_scrub_stamp":"2026-02-20T12:10:32.975486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:35.996937+0000","last_change":"2026-02-20T12:10:35.998842+0000","last_active":"2026-02-20T12:10:35.996937+0000","last_peered":"2026-02-20T12:10:35.996937+0000","last_clean":"2026-02-20T12:10:35.996937+0000","last_became_active":"2026-02-20T12:10:34.012044+0000","last_became_peered":"2026-02-20T12:10:34.012044+0000","last_unstale":"2026-02-20T12:10:35.996937+0000","last_undegraded":"2026-02-20T12:10:35.996937+0000","last_fullsized":"2026-02-20T12:10:35.996937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_clean_scrub_stamp":"2026-02-20T12:10:32.975486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:35.999817+0000","last_change":"2026-02-20T12:10:36.002756+0000","last_active":"2026-02-20T12:10:35.999817+0000","last_peered":"2026-02-20T12:10:35.999817+0000","last_clean":"2026-02-20T12:10:35.999817+0000","last_became_active":"2026-02-20T12:10:34.012541+0000","last_became_peered":"2026-02-20T12:10:34.012541+0000","last_unstale":"2026-02-20T12:10:35.999817+0000","last_undegraded":"2026-02-20T12:10:35.999817+0000","last_fullsized":"2026-02-20T12:10:35.999817+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_clean_scrub_stamp":"2026-02-20T12:10:32.975486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:35.998592+0000","last_change":"2026-02-20T12:10:30.000224+0000","last_active":"2026-02-20T12:10:35.998592+0000","last_peered":"2026-02-20T12:10:35.998592+0000","last_clean":"2026-02-20T12:10:35.998592+0000","last_became_active":"2026-02-20T12:10:29.993228+0000","last_became_peered":"2026-02-20T12:10:29.993228+0000","last_unstale":"2026-02-20T12:10:35.998592+0000","last_undegraded":"2026-02-20T12:10:35.998592+0000","last_fullsized":"2026-02-20T12:10:35.998592+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:28.964644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:28.964644+0000","last_clean_scrub_stamp":"2026-02-20T12:10:28.964644+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971812,"kb_used_data":20971812,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456972,"statfs":{"total":53687074816,"available":32211939328,"internally_reserved":0,"allocated":21475135488,"data_stored":21475135488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972572,"kb_used_data":20972572,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456212,"statfs":{"total":53687074816,"available":32211161088,"internally_reserved":0,"allocated":21475913728,"data_stored":21475913728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972620,"kb_used_data":20972620,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456164,"statfs":{"total":53687074816,"available":32211111936,"internally_reserved":0,"allocated":21475962880,"data_stored":21475962880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T12:10:37.460 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T12:10:37.535 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:37.532+0000 7f7452ff5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:37.536 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:37.536+0000 7f7452ff5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:37.645 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:10:37.646 INFO:teuthology.orchestra.run.trial131.stderr:dumped all 2026-02-20T12:10:37.654 INFO:teuthology.orchestra.run.trial131.stdout:{"pg_ready":true,"pg_map":{"version":14,"stamp":"2026-02-20T12:10:36.968442+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62917004,"kb_used_data":62917004,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94369348,"statfs":{"total":161061224448,"available":96634212352,"internally_reserved":0,"allocated":64427012096,"data_stored":64427012096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"5.995778"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:35.998808+0000","last_change":"2026-02-20T12:10:36.002294+0000","last_active":"2026-02-20T12:10:35.998808+0000","last_peered":"2026-02-20T12:10:35.998808+0000","last_clean":"2026-02-20T12:10:35.998808+0000","last_became_active":"2026-02-20T12:10:34.013044+0000","last_became_peered":"2026-02-20T12:10:34.013044+0000","last_unstale":"2026-02-20T12:10:35.998808+0000","last_undegraded":"2026-02-20T12:10:35.998808+0000","last_fullsized":"2026-02-20T12:10:35.998808+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_clean_scrub_stamp":"2026-02-20T12:10:32.975486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:35.999763+0000","last_change":"2026-02-20T12:10:36.001510+0000","last_active":"2026-02-20T12:10:35.999763+0000","last_peered":"2026-02-20T12:10:35.999763+0000","last_clean":"2026-02-20T12:10:35.999763+0000","last_became_active":"2026-02-20T12:10:34.010781+0000","last_became_peered":"2026-02-20T12:10:34.010781+0000","last_unstale":"2026-02-20T12:10:35.999763+0000","last_undegraded":"2026-02-20T12:10:35.999763+0000","last_fullsized":"2026-02-20T12:10:35.999763+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_clean_scrub_stamp":"2026-02-20T12:10:32.975486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:35.999724+0000","last_change":"2026-02-20T12:10:36.002769+0000","last_active":"2026-02-20T12:10:35.999724+0000","last_peered":"2026-02-20T12:10:35.999724+0000","last_clean":"2026-02-20T12:10:35.999724+0000","last_became_active":"2026-02-20T12:10:34.016759+0000","last_became_peered":"2026-02-20T12:10:34.016759+0000","last_unstale":"2026-02-20T12:10:35.999724+0000","last_undegraded":"2026-02-20T12:10:35.999724+0000","last_fullsized":"2026-02-20T12:10:35.999724+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_clean_scrub_stamp":"2026-02-20T12:10:32.975486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:36.000280+0000","last_change":"2026-02-20T12:10:36.001808+0000","last_active":"2026-02-20T12:10:36.000280+0000","last_peered":"2026-02-20T12:10:36.000280+0000","last_clean":"2026-02-20T12:10:36.000280+0000","last_became_active":"2026-02-20T12:10:34.012723+0000","last_became_peered":"2026-02-20T12:10:34.012723+0000","last_unstale":"2026-02-20T12:10:36.000280+0000","last_undegraded":"2026-02-20T12:10:36.000280+0000","last_fullsized":"2026-02-20T12:10:36.000280+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_clean_scrub_stamp":"2026-02-20T12:10:32.975486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:35.999581+0000","last_change":"2026-02-20T12:10:36.000892+0000","last_active":"2026-02-20T12:10:35.999581+0000","last_peered":"2026-02-20T12:10:35.999581+0000","last_clean":"2026-02-20T12:10:35.999581+0000","last_became_active":"2026-02-20T12:10:34.021605+0000","last_became_peered":"2026-02-20T12:10:34.021605+0000","last_unstale":"2026-02-20T12:10:35.999581+0000","last_undegraded":"2026-02-20T12:10:35.999581+0000","last_fullsized":"2026-02-20T12:10:35.999581+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_clean_scrub_stamp":"2026-02-20T12:10:32.975486+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:35.998610+0000","last_change":"2026-02-20T12:10:36.000803+0000","last_active":"2026-02-20T12:10:35.998610+0000","last_peered":"2026-02-20T12:10:35.998610+0000","last_clean":"2026-02-20T12:10:35.998610+0000","last_became_active":"2026-02-20T12:10:34.016549+0000","last_became_peered":"2026-02-20T12:10:34.016549+0000","last_unstale":"2026-02-20T12:10:35.998610+0000","last_undegraded":"2026-02-20T12:10:35.998610+0000","last_fullsized":"2026-02-20T12:10:35.998610+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_clean_scrub_stamp":"2026-02-20T12:10:32.975486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:35.996937+0000","last_change":"2026-02-20T12:10:35.998842+0000","last_active":"2026-02-20T12:10:35.996937+0000","last_peered":"2026-02-20T12:10:35.996937+0000","last_clean":"2026-02-20T12:10:35.996937+0000","last_became_active":"2026-02-20T12:10:34.012044+0000","last_became_peered":"2026-02-20T12:10:34.012044+0000","last_unstale":"2026-02-20T12:10:35.996937+0000","last_undegraded":"2026-02-20T12:10:35.996937+0000","last_fullsized":"2026-02-20T12:10:35.996937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_clean_scrub_stamp":"2026-02-20T12:10:32.975486+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:35.999817+0000","last_change":"2026-02-20T12:10:36.002756+0000","last_active":"2026-02-20T12:10:35.999817+0000","last_peered":"2026-02-20T12:10:35.999817+0000","last_clean":"2026-02-20T12:10:35.999817+0000","last_became_active":"2026-02-20T12:10:34.012541+0000","last_became_peered":"2026-02-20T12:10:34.012541+0000","last_unstale":"2026-02-20T12:10:35.999817+0000","last_undegraded":"2026-02-20T12:10:35.999817+0000","last_fullsized":"2026-02-20T12:10:35.999817+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:32.975486+0000","last_clean_scrub_stamp":"2026-02-20T12:10:32.975486+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-20T12:10:35.998592+0000","last_change":"2026-02-20T12:10:30.000224+0000","last_active":"2026-02-20T12:10:35.998592+0000","last_peered":"2026-02-20T12:10:35.998592+0000","last_clean":"2026-02-20T12:10:35.998592+0000","last_became_active":"2026-02-20T12:10:29.993228+0000","last_became_peered":"2026-02-20T12:10:29.993228+0000","last_unstale":"2026-02-20T12:10:35.998592+0000","last_undegraded":"2026-02-20T12:10:35.998592+0000","last_fullsized":"2026-02-20T12:10:35.998592+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T12:10:28.964644+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T12:10:28.964644+0000","last_clean_scrub_stamp":"2026-02-20T12:10:28.964644+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971812,"kb_used_data":20971812,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456972,"statfs":{"total":53687074816,"available":32211939328,"internally_reserved":0,"allocated":21475135488,"data_stored":21475135488,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972572,"kb_used_data":20972572,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456212,"statfs":{"total":53687074816,"available":32211161088,"internally_reserved":0,"allocated":21475913728,"data_stored":21475913728,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972620,"kb_used_data":20972620,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456164,"statfs":{"total":53687074816,"available":32211111936,"internally_reserved":0,"allocated":21475962880,"data_stored":21475962880,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T12:10:37.656 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-20T12:10:37.656 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-20T12:10:37.656 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-20T12:10:37.656 DEBUG:teuthology.orchestra.run.trial131:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-20T12:10:37.732 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:37.730+0000 7fc2d205a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:37.733 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:10:37.733+0000 7fc2d205a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:10:37.870 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:10:37.871 INFO:teuthology.orchestra.run.trial131.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-20T12:10:37.880 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-20T12:10:37.880 INFO:teuthology.run_tasks:Running task ssh_keys... 2026-02-20T12:10:37.978 DEBUG:teuthology.orchestra.run.trial131:> echo 'StrictHostKeyChecking no 2026-02-20T12:10:37.978 DEBUG:teuthology.orchestra.run.trial131:> ' > /home/ubuntu/.ssh/config && echo 'UserKnownHostsFile ' /dev/null >> /home/ubuntu/.ssh/config && chmod 600 /home/ubuntu/.ssh/config 2026-02-20T12:10:37.994 INFO:teuthology.task.ssh_keys:generated public key AAAAB3NzaC1yc2EAAAADAQABAAABAQCv7WfC2KPgi/QE0bZ4p0d68WUepp0frirheynfDKkRNJBucdD692jPwEaInOoLTgLRtsZtZv0Aa515kBmzSHO51CKDgvRko1I3d57vH00jb2+8rjCBHrs8BqU9sDCJXsmikj8Oso3nZ2ND8Yt2bKmtk+qh0G2pOWtkFCVi0v7AHDplnbjxxVF5XtkxAVpX+rlarq5F6mHmW0MOf+l685o0ROOixZQ8e6DfSbujeQ/QpiFEqg+2qDwFFDYnsGdBZHfmfd2p2Mg2TUewHyTDgC7WF1hTJn5WZRCcT2Mk0xHagjM3LHucjrno1UnuAwpolat7mDiMVBUmy6i+d+2ye1gv 2026-02-20T12:10:37.994 INFO:teuthology.task.ssh_keys:pushing keys to trial131.front.sepia.ceph.com for ubuntu 2026-02-20T12:10:37.994 DEBUG:teuthology.orchestra.run.trial131:> rm -f -- /home/ubuntu/.ssh/id_rsa 2026-02-20T12:10:38.050 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:10:38.050 DEBUG:teuthology.orchestra.run.trial131:> dd of=/home/ubuntu/.ssh/id_rsa 2026-02-20T12:10:38.050 DEBUG:teuthology.orchestra.run.trial131:> chmod 0500 /home/ubuntu/.ssh/id_rsa 2026-02-20T12:10:38.106 DEBUG:teuthology.orchestra.run.trial131:> rm -f -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-20T12:10:38.162 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:10:38.162 DEBUG:teuthology.orchestra.run.trial131:> dd of=/home/ubuntu/.ssh/id_rsa.pub 2026-02-20T12:10:38.217 DEBUG:teuthology.orchestra.run.trial131:> cp -v -a /home/ubuntu/.ssh/authorized_keys /home/ubuntu/.ssh/authorized_keys_2026-02-20_12:10:38:217574 2026-02-20T12:10:38.273 INFO:teuthology.orchestra.run.trial131.stdout:'/home/ubuntu/.ssh/authorized_keys' -> '/home/ubuntu/.ssh/authorized_keys_2026-02-20_12:10:38:217574' 2026-02-20T12:10:38.273 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:10:38.273 DEBUG:teuthology.orchestra.run.trial131:> dd of=/home/ubuntu/.ssh/authorized_keys conv=notrunc oflag=append 2026-02-20T12:10:38.329 INFO:teuthology.run_tasks:Running task cbt... 2026-02-20T12:10:38.342 INFO:tasks.cbt:cbt configuration is {'cluster': {'user': 'ubuntu', 'head': 'ubuntu@trial131.front.sepia.ceph.com', 'osds': ['ubuntu@trial131.front.sepia.ceph.com'], 'mons': ['ubuntu@trial131.front.sepia.ceph.com'], 'clients': ['ubuntu@trial131.front.sepia.ceph.com'], 'rgws': {'ubuntu@trial131.front.sepia.ceph.com': None}, 'osds_per_node': 3, 'rebuild_every_test': False, 'use_existing': True, 'is_teuthology': True, 'iterations': 1, 'tmp_dir': '/tmp/cbt', 'pool_profiles': {'rbd': {'pg_size': 128, 'pgp_size': 128, 'replication': 3}}, 'pid_dir': '/var/run/ceph'}, 'benchmarks': {'librbdfio': {'iodepth': [32], 'log_avg_msec': 100, 'mode': ['randrw'], 'norandommap': True, 'op_size': [4096], 'osd_ra': [4096], 'pool_profile': 'rbd', 'procs_per_volume': [1], 'time': 60, 'vol_size': 4096, 'volumes_per_client': [2], 'cmd_path': '/home/ubuntu/cephtest/fio/fio'}}, 'client_endpoints': None, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}} 2026-02-20T12:10:38.343 DEBUG:teuthology.orchestra.run.trial131:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/cbt 2026-02-20T12:10:38.386 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:10:38.386 DEBUG:teuthology.orchestra.run.trial131:> dd of=/home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2026-02-20T12:10:38.440 DEBUG:teuthology.orchestra.run.trial131:> git clone --depth 1 -b master https://github.com/ceph/cbt.git /home/ubuntu/cephtest/cbt 2026-02-20T12:10:38.503 INFO:teuthology.orchestra.run.trial131.stderr:Cloning into '/home/ubuntu/cephtest/cbt'... 2026-02-20T12:10:38.822 INFO:tasks.cbt:Installing collectl 2026-02-20T12:10:38.822 DEBUG:teuthology.orchestra.run.trial131:> sudo mkdir collectl && cd collectl && sudo wget https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download -O collectl.tar.gz && sudo tar -xvf collectl.tar.gz && cd collectl-4.3.1 && sudo ./INSTALL 2026-02-20T12:10:38.898 INFO:teuthology.orchestra.run.trial131.stderr:--2026-02-20 12:10:38-- https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download 2026-02-20T12:10:39.068 INFO:teuthology.orchestra.run.trial131.stderr:Resolving sourceforge.net (sourceforge.net)... 104.18.12.149, 104.18.13.149, 2606:4700::6812:d95, ... 2026-02-20T12:10:39.154 INFO:teuthology.orchestra.run.trial131.stderr:Connecting to sourceforge.net (sourceforge.net)|104.18.12.149|:443... connected. 2026-02-20T12:10:39.528 INFO:teuthology.orchestra.run.trial131.stderr:HTTP request sent, awaiting response... 302 Found 2026-02-20T12:10:39.529 INFO:teuthology.orchestra.run.trial131.stderr:Location: https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABpmE8_pGyALTs-zqXSTh-3F1eFHeKdhYeVpTFPd6ccmyovaKnYrNaPzF1zgneAc88RtFI_rGcvCLh1bKlWGzN8zN63EA%3D%3D&use_mirror=phoenixnap&r= [following] 2026-02-20T12:10:39.529 INFO:teuthology.orchestra.run.trial131.stderr:--2026-02-20 12:10:39-- https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABpmE8_pGyALTs-zqXSTh-3F1eFHeKdhYeVpTFPd6ccmyovaKnYrNaPzF1zgneAc88RtFI_rGcvCLh1bKlWGzN8zN63EA%3D%3D&use_mirror=phoenixnap&r= 2026-02-20T12:10:39.692 INFO:teuthology.orchestra.run.trial131.stderr:Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 104.18.12.149, 104.18.13.149, 2606:4700::6812:d95, ... 2026-02-20T12:10:39.777 INFO:teuthology.orchestra.run.trial131.stderr:Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|104.18.12.149|:443... connected. 2026-02-20T12:10:40.121 INFO:teuthology.orchestra.run.trial131.stderr:HTTP request sent, awaiting response... 302 Found 2026-02-20T12:10:40.122 INFO:teuthology.orchestra.run.trial131.stderr:Location: https://phoenixnap.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 [following] 2026-02-20T12:10:40.122 INFO:teuthology.orchestra.run.trial131.stderr:--2026-02-20 12:10:40-- https://phoenixnap.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 2026-02-20T12:10:40.207 INFO:teuthology.orchestra.run.trial131.stderr:Resolving phoenixnap.dl.sourceforge.net (phoenixnap.dl.sourceforge.net)... 184.164.141.26 2026-02-20T12:10:40.269 INFO:teuthology.orchestra.run.trial131.stderr:Connecting to phoenixnap.dl.sourceforge.net (phoenixnap.dl.sourceforge.net)|184.164.141.26|:443... connected. 2026-02-20T12:10:42.618 INFO:teuthology.orchestra.run.trial131.stderr:HTTP request sent, awaiting response... 200 OK 2026-02-20T12:10:42.618 INFO:teuthology.orchestra.run.trial131.stderr:Length: 620693 (606K) [application/x-gzip] 2026-02-20T12:10:42.618 INFO:teuthology.orchestra.run.trial131.stderr:Saving to: ‘collectl.tar.gz’ 2026-02-20T12:10:42.618 INFO:teuthology.orchestra.run.trial131.stderr: 2026-02-20T12:10:42.680 INFO:teuthology.orchestra.run.trial131.stderr: 0K .......... .......... .......... .......... .......... 8% 802K 1s 2026-02-20T12:10:42.741 INFO:teuthology.orchestra.run.trial131.stderr: 50K .......... .......... .......... .......... .......... 16% 814K 1s 2026-02-20T12:10:42.741 INFO:teuthology.orchestra.run.trial131.stderr: 100K .......... .......... .......... .......... .......... 24% 57.9M 0s 2026-02-20T12:10:42.805 INFO:teuthology.orchestra.run.trial131.stderr: 150K .......... .......... .......... .......... .......... 32% 814K 0s 2026-02-20T12:10:42.805 INFO:teuthology.orchestra.run.trial131.stderr: 200K .......... .......... .......... .......... .......... 41% 154M 0s 2026-02-20T12:10:42.805 INFO:teuthology.orchestra.run.trial131.stderr: 250K .......... .......... .......... .......... .......... 49% 263M 0s 2026-02-20T12:10:42.864 INFO:teuthology.orchestra.run.trial131.stderr: 300K .......... .......... .......... .......... .......... 57% 819K 0s 2026-02-20T12:10:43.316 INFO:teuthology.orchestra.run.trial131.stderr: 350K .......... .......... .......... .......... .......... 65% 111K 0s 2026-02-20T12:10:43.316 INFO:teuthology.orchestra.run.trial131.stderr: 400K .......... .......... .......... .......... .......... 74% 304M 0s 2026-02-20T12:10:43.316 INFO:teuthology.orchestra.run.trial131.stderr: 450K .......... .......... .......... .......... .......... 82% 235M 0s 2026-02-20T12:10:43.379 INFO:teuthology.orchestra.run.trial131.stderr: 500K .......... .......... .......... .......... .......... 90% 815K 0s 2026-02-20T12:10:43.379 INFO:teuthology.orchestra.run.trial131.stderr: 550K .......... .......... .......... .......... .......... 98% 192M 0s 2026-02-20T12:10:43.379 INFO:teuthology.orchestra.run.trial131.stderr: 600K ...... 100% 11.4T=0.8s 2026-02-20T12:10:43.379 INFO:teuthology.orchestra.run.trial131.stderr: 2026-02-20T12:10:43.380 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20 12:10:43 (798 KB/s) - ‘collectl.tar.gz’ saved [620693/620693] 2026-02-20T12:10:43.380 INFO:teuthology.orchestra.run.trial131.stderr: 2026-02-20T12:10:43.406 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/ 2026-02-20T12:10:43.406 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/collectl.conf 2026-02-20T12:10:43.406 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/ 2026-02-20T12:10:43.406 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/SlowProc.html 2026-02-20T12:10:43.406 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/BuddyInfo.html 2026-02-20T12:10:43.406 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/WhySummary.html 2026-02-20T12:10:43.406 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Socket.html 2026-02-20T12:10:43.407 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/colmux.html 2026-02-20T12:10:43.407 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Tutorial.html 2026-02-20T12:10:43.407 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Gexpr.html 2026-02-20T12:10:43.407 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Import.html 2026-02-20T12:10:43.407 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/style.css 2026-02-20T12:10:43.407 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Ganglia3.jpg 2026-02-20T12:10:43.407 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Ganglia2.jpg 2026-02-20T12:10:43.407 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/FileNaming.html 2026-02-20T12:10:43.407 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/TheMath.html 2026-02-20T12:10:43.407 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Environmental.html 2026-02-20T12:10:43.407 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Process.html 2026-02-20T12:10:43.407 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/OutputFiles.html 2026-02-20T12:10:43.407 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/HiResTime.html 2026-02-20T12:10:43.408 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/OutputFormats.html 2026-02-20T12:10:43.408 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/slub.jpg 2026-02-20T12:10:43.408 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Lustre.html 2026-02-20T12:10:43.408 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Documentation.html 2026-02-20T12:10:43.408 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Inodes.html 2026-02-20T12:10:43.408 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Graphite.html 2026-02-20T12:10:43.408 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/InputFiles.html 2026-02-20T12:10:43.408 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Export.html 2026-02-20T12:10:43.408 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Interrupts.html 2026-02-20T12:10:43.408 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Startup.html 2026-02-20T12:10:43.408 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Performance.html 2026-02-20T12:10:43.408 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Messages.html 2026-02-20T12:10:43.408 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Logging.html 2026-02-20T12:10:43.409 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Data-detail.html 2026-02-20T12:10:43.409 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/CPUs.html 2026-02-20T12:10:43.409 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Matrix.html 2026-02-20T12:10:43.409 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Data-brief.html 2026-02-20T12:10:43.409 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/SlabInfo.html 2026-02-20T12:10:43.409 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Misc.html 2026-02-20T12:10:43.409 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Examples.html 2026-02-20T12:10:43.409 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/OpenStack.html 2026-02-20T12:10:43.409 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Plotfiles.html 2026-02-20T12:10:43.409 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/ColmuxCPU.jpg 2026-02-20T12:10:43.409 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Tutorial-Lustre.html 2026-02-20T12:10:43.409 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Data-verbose.html 2026-02-20T12:10:43.409 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Data.html 2026-02-20T12:10:43.410 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Exceptions.html 2026-02-20T12:10:43.410 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/RunningAsAService.html 2026-02-20T12:10:43.410 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/FAQ-collectl.html 2026-02-20T12:10:43.410 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Ganglia.jpg 2026-02-20T12:10:43.410 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/OperationalModes.html 2026-02-20T12:10:43.410 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Playback.html 2026-02-20T12:10:43.410 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Hello.html 2026-02-20T12:10:43.410 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Sockets.html 2026-02-20T12:10:43.410 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Network.html 2026-02-20T12:10:43.410 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Infiniband.html 2026-02-20T12:10:43.410 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Memory.html 2026-02-20T12:10:43.410 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/ColmuxLustre.jpg 2026-02-20T12:10:43.410 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/Disks.html 2026-02-20T12:10:43.411 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/docs/NfsInfo.html 2026-02-20T12:10:43.411 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/service/ 2026-02-20T12:10:43.411 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/service/collectl.service 2026-02-20T12:10:43.411 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/initd/ 2026-02-20T12:10:43.411 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/initd/collectl-suse 2026-02-20T12:10:43.411 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/initd/collectl-debian 2026-02-20T12:10:43.411 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/initd/collectl-generic 2026-02-20T12:10:43.411 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/initd/collectl 2026-02-20T12:10:43.411 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/misc.ph 2026-02-20T12:10:43.411 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/gexpr.ph 2026-02-20T12:10:43.411 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/statsd.ph 2026-02-20T12:10:43.411 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/envrules.std 2026-02-20T12:10:43.412 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/hello.ph 2026-02-20T12:10:43.412 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/vmsum.ph 2026-02-20T12:10:43.412 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/proctree.ph 2026-02-20T12:10:43.412 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/README-WINDOWS 2026-02-20T12:10:43.412 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/formatit.ph 2026-02-20T12:10:43.412 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/colmux 2026-02-20T12:10:43.412 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/INSTALL 2026-02-20T12:10:43.412 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/graphite.ph 2026-02-20T12:10:43.412 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/RELEASE-collectl 2026-02-20T12:10:43.412 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/vmstat.ph 2026-02-20T12:10:43.412 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/README 2026-02-20T12:10:43.413 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/man1/ 2026-02-20T12:10:43.413 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/man1/collectl.1 2026-02-20T12:10:43.413 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/man1/colmux.1 2026-02-20T12:10:43.413 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/lexpr.ph 2026-02-20T12:10:43.413 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/client.pl 2026-02-20T12:10:43.413 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/collectl 2026-02-20T12:10:43.413 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/COPYING 2026-02-20T12:10:43.413 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/vnet.ph 2026-02-20T12:10:43.413 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/GPL 2026-02-20T12:10:43.413 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/UNINSTALL 2026-02-20T12:10:43.413 INFO:teuthology.orchestra.run.trial131.stdout:collectl-4.3.1/ARTISTIC 2026-02-20T12:10:43.446 INFO:teuthology.orchestra.run.trial131.stderr:./INSTALL: line 90: chkconfig: command not found 2026-02-20T12:10:43.448 DEBUG:teuthology.orchestra.run.trial131:> sudo yum -y install librbd-devel pdsh pdsh-rcmd-ssh perf 2026-02-20T12:10:43.800 INFO:teuthology.orchestra.run.trial131.stdout:Last metadata expiration check: 0:01:00 ago on Fri 20 Feb 2026 12:09:43 PM UTC. 2026-02-20T12:10:43.887 INFO:teuthology.orchestra.run.trial131.stdout:Dependencies resolved. 2026-02-20T12:10:43.888 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:10:43.888 INFO:teuthology.orchestra.run.trial131.stdout: Package Arch Version Repository Size 2026-02-20T12:10:43.888 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:10:43.888 INFO:teuthology.orchestra.run.trial131.stdout:Installing: 2026-02-20T12:10:43.888 INFO:teuthology.orchestra.run.trial131.stdout: librbd-devel x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 24 k 2026-02-20T12:10:43.889 INFO:teuthology.orchestra.run.trial131.stdout: pdsh x86_64 2.34-7.el9 epel 118 k 2026-02-20T12:10:43.889 INFO:teuthology.orchestra.run.trial131.stdout: pdsh-rcmd-ssh x86_64 2.34-7.el9 epel 13 k 2026-02-20T12:10:43.889 INFO:teuthology.orchestra.run.trial131.stdout: perf x86_64 5.14.0-681.el9 appstream 3.2 M 2026-02-20T12:10:43.889 INFO:teuthology.orchestra.run.trial131.stdout:Installing dependencies: 2026-02-20T12:10:43.889 INFO:teuthology.orchestra.run.trial131.stdout: libradospp-devel x86_64 2:20.3.0-5312.g26bc9765.el9 ceph 31 k 2026-02-20T12:10:43.889 INFO:teuthology.orchestra.run.trial131.stdout: libtraceevent x86_64 1.8.4-2.el9 baseos 290 k 2026-02-20T12:10:43.889 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:10:43.889 INFO:teuthology.orchestra.run.trial131.stdout:Transaction Summary 2026-02-20T12:10:43.889 INFO:teuthology.orchestra.run.trial131.stdout:================================================================================ 2026-02-20T12:10:43.889 INFO:teuthology.orchestra.run.trial131.stdout:Install 6 Packages 2026-02-20T12:10:43.889 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:10:43.890 INFO:teuthology.orchestra.run.trial131.stdout:Total download size: 3.7 M 2026-02-20T12:10:43.890 INFO:teuthology.orchestra.run.trial131.stdout:Installed size: 14 M 2026-02-20T12:10:43.890 INFO:teuthology.orchestra.run.trial131.stdout:Downloading Packages: 2026-02-20T12:10:44.417 INFO:teuthology.orchestra.run.trial131.stdout:(1/6): libtraceevent-1.8.4-2.el9.x86_64.rpm 2.2 MB/s | 290 kB 00:00 2026-02-20T12:10:44.586 INFO:teuthology.orchestra.run.trial131.stdout:(2/6): perf-5.14.0-681.el9.x86_64.rpm 19 MB/s | 3.2 MB 00:00 2026-02-20T12:10:44.643 INFO:teuthology.orchestra.run.trial131.stdout:(3/6): librbd-devel-20.3.0-5312.g26bc9765.el9.x 67 kB/s | 24 kB 00:00 2026-02-20T12:10:44.859 INFO:teuthology.orchestra.run.trial131.stdout:(4/6): libradospp-devel-20.3.0-5312.g26bc9765.e 55 kB/s | 31 kB 00:00 2026-02-20T12:10:44.980 INFO:teuthology.orchestra.run.trial131.stdout:(5/6): pdsh-rcmd-ssh-2.34-7.el9.x86_64.rpm 39 kB/s | 13 kB 00:00 2026-02-20T12:10:45.051 INFO:teuthology.orchestra.run.trial131.stdout:(6/6): pdsh-2.34-7.el9.x86_64.rpm 255 kB/s | 118 kB 00:00 2026-02-20T12:10:45.052 INFO:teuthology.orchestra.run.trial131.stdout:-------------------------------------------------------------------------------- 2026-02-20T12:10:45.052 INFO:teuthology.orchestra.run.trial131.stdout:Total 3.1 MB/s | 3.7 MB 00:01 2026-02-20T12:10:45.074 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction check 2026-02-20T12:10:45.084 INFO:teuthology.orchestra.run.trial131.stdout:Transaction check succeeded. 2026-02-20T12:10:45.084 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction test 2026-02-20T12:10:45.141 INFO:teuthology.orchestra.run.trial131.stdout:Transaction test succeeded. 2026-02-20T12:10:45.141 INFO:teuthology.orchestra.run.trial131.stdout:Running transaction 2026-02-20T12:10:45.218 INFO:teuthology.orchestra.run.trial131.stdout: Preparing : 1/1 2026-02-20T12:10:45.223 INFO:teuthology.orchestra.run.trial131.stdout: Installing : pdsh-rcmd-ssh-2.34-7.el9.x86_64 1/6 2026-02-20T12:10:45.264 INFO:teuthology.orchestra.run.trial131.stdout: Installing : pdsh-2.34-7.el9.x86_64 2/6 2026-02-20T12:10:45.269 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libtraceevent-1.8.4-2.el9.x86_64 3/6 2026-02-20T12:10:45.271 INFO:teuthology.orchestra.run.trial131.stdout: Installing : libradospp-devel-2:20.3.0-5312.g26bc9765.el9.x86_64 4/6 2026-02-20T12:10:45.434 INFO:teuthology.orchestra.run.trial131.stdout: Installing : librbd-devel-2:20.3.0-5312.g26bc9765.el9.x86_64 5/6 2026-02-20T12:10:45.455 INFO:teuthology.orchestra.run.trial131.stdout: Installing : perf-5.14.0-681.el9.x86_64 6/6 2026-02-20T12:10:45.573 INFO:teuthology.orchestra.run.trial131.stdout: Running scriptlet: perf-5.14.0-681.el9.x86_64 6/6 2026-02-20T12:10:45.573 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libradospp-devel-2:20.3.0-5312.g26bc9765.el9.x86_64 1/6 2026-02-20T12:10:45.573 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : librbd-devel-2:20.3.0-5312.g26bc9765.el9.x86_64 2/6 2026-02-20T12:10:45.573 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : libtraceevent-1.8.4-2.el9.x86_64 3/6 2026-02-20T12:10:45.573 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : perf-5.14.0-681.el9.x86_64 4/6 2026-02-20T12:10:45.573 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : pdsh-2.34-7.el9.x86_64 5/6 2026-02-20T12:10:45.634 INFO:teuthology.orchestra.run.trial131.stdout: Verifying : pdsh-rcmd-ssh-2.34-7.el9.x86_64 6/6 2026-02-20T12:10:45.634 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:10:45.634 INFO:teuthology.orchestra.run.trial131.stdout:Installed: 2026-02-20T12:10:45.634 INFO:teuthology.orchestra.run.trial131.stdout: libradospp-devel-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:10:45.634 INFO:teuthology.orchestra.run.trial131.stdout: librbd-devel-2:20.3.0-5312.g26bc9765.el9.x86_64 2026-02-20T12:10:45.634 INFO:teuthology.orchestra.run.trial131.stdout: libtraceevent-1.8.4-2.el9.x86_64 2026-02-20T12:10:45.634 INFO:teuthology.orchestra.run.trial131.stdout: pdsh-2.34-7.el9.x86_64 2026-02-20T12:10:45.635 INFO:teuthology.orchestra.run.trial131.stdout: pdsh-rcmd-ssh-2.34-7.el9.x86_64 2026-02-20T12:10:45.635 INFO:teuthology.orchestra.run.trial131.stdout: perf-5.14.0-681.el9.x86_64 2026-02-20T12:10:45.635 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:10:45.635 INFO:teuthology.orchestra.run.trial131.stdout:Complete! 2026-02-20T12:10:45.664 DEBUG:teuthology.orchestra.run.trial131:> python3 -m pip install -r /home/ubuntu/cephtest/cbt/requirements.txt 2026-02-20T12:10:45.804 INFO:teuthology.orchestra.run.trial131.stdout:Defaulting to user installation because normal site-packages is not writeable 2026-02-20T12:10:45.815 INFO:teuthology.orchestra.run.trial131.stdout:Requirement already satisfied: pyyaml in /usr/lib64/python3.9/site-packages (from -r /home/ubuntu/cephtest/cbt/requirements.txt (line 1)) (5.4.1) 2026-02-20T12:10:45.815 INFO:teuthology.orchestra.run.trial131.stdout:Requirement already satisfied: lxml in /usr/lib64/python3.9/site-packages (from -r /home/ubuntu/cephtest/cbt/requirements.txt (line 2)) (4.6.5) 2026-02-20T12:10:46.611 INFO:teuthology.orchestra.run.trial131.stdout:Collecting matplotlib 2026-02-20T12:10:46.864 INFO:teuthology.orchestra.run.trial131.stdout: Downloading matplotlib-3.9.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.3 MB) 2026-02-20T12:10:47.528 INFO:teuthology.orchestra.run.trial131.stdout:Requirement already satisfied: packaging>=20.0 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (20.9) 2026-02-20T12:10:47.631 INFO:teuthology.orchestra.run.trial131.stdout:Collecting cycler>=0.10 2026-02-20T12:10:47.724 INFO:teuthology.orchestra.run.trial131.stdout: Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB) 2026-02-20T12:10:47.975 INFO:teuthology.orchestra.run.trial131.stdout:Collecting contourpy>=1.0.1 2026-02-20T12:10:48.050 INFO:teuthology.orchestra.run.trial131.stdout: Downloading contourpy-1.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (321 kB) 2026-02-20T12:10:48.056 INFO:teuthology.orchestra.run.trial131.stdout:Requirement already satisfied: python-dateutil>=2.7 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (2.9.0.post0) 2026-02-20T12:10:48.057 INFO:teuthology.orchestra.run.trial131.stdout:Requirement already satisfied: numpy>=1.23 in /usr/lib64/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (1.23.5) 2026-02-20T12:10:48.166 INFO:teuthology.orchestra.run.trial131.stdout:Collecting importlib-resources>=3.2.0 2026-02-20T12:10:48.240 INFO:teuthology.orchestra.run.trial131.stdout: Downloading importlib_resources-6.5.2-py3-none-any.whl (37 kB) 2026-02-20T12:10:48.688 INFO:teuthology.orchestra.run.trial131.stdout:Collecting fonttools>=4.22.0 2026-02-20T12:10:48.768 INFO:teuthology.orchestra.run.trial131.stdout: Downloading fonttools-4.60.2-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.8 MB) 2026-02-20T12:10:49.337 INFO:teuthology.orchestra.run.trial131.stdout:Collecting pillow>=8 2026-02-20T12:10:49.419 INFO:teuthology.orchestra.run.trial131.stdout: Downloading pillow-11.3.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (6.6 MB) 2026-02-20T12:10:49.548 INFO:teuthology.orchestra.run.trial131.stdout:Requirement already satisfied: pyparsing>=2.3.1 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (2.4.7) 2026-02-20T12:10:49.788 INFO:teuthology.orchestra.run.trial131.stdout:Collecting kiwisolver>=1.3.1 2026-02-20T12:10:49.874 INFO:teuthology.orchestra.run.trial131.stdout: Downloading kiwisolver-1.4.7-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB) 2026-02-20T12:10:50.032 INFO:teuthology.orchestra.run.trial131.stdout:Collecting zipp>=3.1.0 2026-02-20T12:10:50.116 INFO:teuthology.orchestra.run.trial131.stdout: Downloading zipp-3.23.0-py3-none-any.whl (10 kB) 2026-02-20T12:10:50.127 INFO:teuthology.orchestra.run.trial131.stdout:Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (1.15.0) 2026-02-20T12:10:50.242 INFO:teuthology.orchestra.run.trial131.stdout:Installing collected packages: zipp, pillow, kiwisolver, importlib-resources, fonttools, cycler, contourpy, matplotlib 2026-02-20T12:10:51.301 INFO:teuthology.orchestra.run.trial131.stdout:Successfully installed contourpy-1.3.0 cycler-0.12.1 fonttools-4.60.2 importlib-resources-6.5.2 kiwisolver-1.4.7 matplotlib-3.9.4 pillow-11.3.0 zipp-3.23.0 2026-02-20T12:10:51.760 INFO:tasks.cbt:benchmark: librbdfio 2026-02-20T12:10:51.761 DEBUG:teuthology.orchestra.run.trial131:> git clone -b master https://github.com/axboe/fio.git /home/ubuntu/cephtest/fio 2026-02-20T12:10:51.778 INFO:teuthology.orchestra.run.trial131.stderr:Cloning into '/home/ubuntu/cephtest/fio'... 2026-02-20T12:10:54.131 DEBUG:teuthology.orchestra.run.trial131:> cd /home/ubuntu/cephtest/fio && ./configure && make 2026-02-20T12:10:54.304 INFO:teuthology.orchestra.run.trial131.stdout:Operating system Linux 2026-02-20T12:10:54.304 INFO:teuthology.orchestra.run.trial131.stdout:CPU x86_64 2026-02-20T12:10:54.304 INFO:teuthology.orchestra.run.trial131.stdout:Big endian no 2026-02-20T12:10:54.304 INFO:teuthology.orchestra.run.trial131.stdout:Compiler gcc 2026-02-20T12:10:54.305 INFO:teuthology.orchestra.run.trial131.stdout:Cross compile no 2026-02-20T12:10:54.305 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:10:54.305 INFO:teuthology.orchestra.run.trial131.stdout:Static build no 2026-02-20T12:10:54.350 INFO:teuthology.orchestra.run.trial131.stdout:Wordsize 64 2026-02-20T12:10:54.355 INFO:teuthology.orchestra.run.trial131.stdout:zlib no 2026-02-20T12:10:54.366 INFO:teuthology.orchestra.run.trial131.stdout:fcntl(F_FULLFSYNC) no 2026-02-20T12:10:54.375 INFO:teuthology.orchestra.run.trial131.stdout:Linux AIO support no 2026-02-20T12:10:54.375 INFO:teuthology.orchestra.run.trial131.stdout:Linux AIO support rw flags no 2026-02-20T12:10:54.401 INFO:teuthology.orchestra.run.trial131.stdout:POSIX AIO support yes 2026-02-20T12:10:54.401 INFO:teuthology.orchestra.run.trial131.stdout:POSIX AIO support needs -lrt no 2026-02-20T12:10:54.421 INFO:teuthology.orchestra.run.trial131.stdout:POSIX AIO fsync yes 2026-02-20T12:10:54.442 INFO:teuthology.orchestra.run.trial131.stdout:POSIX pshared support yes 2026-02-20T12:10:54.468 INFO:teuthology.orchestra.run.trial131.stdout:pthread_condattr_setclock() yes 2026-02-20T12:10:54.493 INFO:teuthology.orchestra.run.trial131.stdout:pthread_sigmask() yes 2026-02-20T12:10:54.516 INFO:teuthology.orchestra.run.trial131.stdout:pthread_getaffinity_np() yes 2026-02-20T12:10:54.523 INFO:teuthology.orchestra.run.trial131.stdout:Solaris AIO support no 2026-02-20T12:10:54.543 INFO:teuthology.orchestra.run.trial131.stdout:__sync_fetch_and_add yes 2026-02-20T12:10:54.565 INFO:teuthology.orchestra.run.trial131.stdout:__sync_synchronize yes 2026-02-20T12:10:54.585 INFO:teuthology.orchestra.run.trial131.stdout:__sync_val_compare_and_swap yes 2026-02-20T12:10:54.590 INFO:teuthology.orchestra.run.trial131.stdout:libverbs no 2026-02-20T12:10:54.599 INFO:teuthology.orchestra.run.trial131.stdout:rdmacm no 2026-02-20T12:10:54.620 INFO:teuthology.orchestra.run.trial131.stdout:asprintf() yes 2026-02-20T12:10:54.640 INFO:teuthology.orchestra.run.trial131.stdout:vasprintf() yes 2026-02-20T12:10:54.662 INFO:teuthology.orchestra.run.trial131.stdout:Linux fallocate yes 2026-02-20T12:10:54.684 INFO:teuthology.orchestra.run.trial131.stdout:POSIX fadvise yes 2026-02-20T12:10:54.703 INFO:teuthology.orchestra.run.trial131.stdout:POSIX fallocate yes 2026-02-20T12:10:54.725 INFO:teuthology.orchestra.run.trial131.stdout:sched_setaffinity(3 arg) yes 2026-02-20T12:10:54.725 INFO:teuthology.orchestra.run.trial131.stdout:sched_setaffinity(2 arg) no 2026-02-20T12:10:54.746 INFO:teuthology.orchestra.run.trial131.stdout:clock_gettime yes 2026-02-20T12:10:54.768 INFO:teuthology.orchestra.run.trial131.stdout:CLOCK_MONOTONIC yes 2026-02-20T12:10:54.790 INFO:teuthology.orchestra.run.trial131.stdout:gettimeofday yes 2026-02-20T12:10:54.811 INFO:teuthology.orchestra.run.trial131.stdout:fdatasync yes 2026-02-20T12:10:54.832 INFO:teuthology.orchestra.run.trial131.stdout:pipe() yes 2026-02-20T12:10:54.853 INFO:teuthology.orchestra.run.trial131.stdout:pipe2() yes 2026-02-20T12:10:54.873 INFO:teuthology.orchestra.run.trial131.stdout:pread() yes 2026-02-20T12:10:54.896 INFO:teuthology.orchestra.run.trial131.stdout:sync_file_range yes 2026-02-20T12:10:54.902 INFO:teuthology.orchestra.run.trial131.stdout:ASharedMemory_create no 2026-02-20T12:10:54.909 INFO:teuthology.orchestra.run.trial131.stdout:EXT4 move extent yes 2026-02-20T12:10:54.930 INFO:teuthology.orchestra.run.trial131.stdout:Linux splice(2) yes 2026-02-20T12:10:54.936 INFO:teuthology.orchestra.run.trial131.stdout:libnuma no 2026-02-20T12:10:54.956 INFO:teuthology.orchestra.run.trial131.stdout:strsep yes 2026-02-20T12:10:54.975 INFO:teuthology.orchestra.run.trial131.stdout:strcasestr yes 2026-02-20T12:10:54.984 INFO:teuthology.orchestra.run.trial131.stdout:strlcat no 2026-02-20T12:10:55.007 INFO:teuthology.orchestra.run.trial131.stdout:getopt_long_only() yes 2026-02-20T12:10:55.032 INFO:teuthology.orchestra.run.trial131.stdout:inet_aton yes 2026-02-20T12:10:55.053 INFO:teuthology.orchestra.run.trial131.stdout:socklen_t yes 2026-02-20T12:10:55.075 INFO:teuthology.orchestra.run.trial131.stdout:__thread yes 2026-02-20T12:10:55.096 INFO:teuthology.orchestra.run.trial131.stdout:RUSAGE_THREAD yes 2026-02-20T12:10:55.117 INFO:teuthology.orchestra.run.trial131.stdout:SCHED_IDLE yes 2026-02-20T12:10:55.141 INFO:teuthology.orchestra.run.trial131.stdout:TCP_NODELAY yes 2026-02-20T12:10:55.163 INFO:teuthology.orchestra.run.trial131.stdout:vsock yes 2026-02-20T12:10:55.184 INFO:teuthology.orchestra.run.trial131.stdout:Net engine window_size yes 2026-02-20T12:10:55.207 INFO:teuthology.orchestra.run.trial131.stdout:TCP_MAXSEG yes 2026-02-20T12:10:55.227 INFO:teuthology.orchestra.run.trial131.stdout:RLIMIT_MEMLOCK yes 2026-02-20T12:10:55.250 INFO:teuthology.orchestra.run.trial131.stdout:pwritev/preadv yes 2026-02-20T12:10:55.273 INFO:teuthology.orchestra.run.trial131.stdout:pwritev2/preadv2 yes 2026-02-20T12:10:55.296 INFO:teuthology.orchestra.run.trial131.stdout:IPv6 helpers yes 2026-02-20T12:10:55.309 INFO:teuthology.orchestra.run.trial131.stdout:http engine no 2026-02-20T12:10:55.361 INFO:teuthology.orchestra.run.trial131.stdout:Rados engine yes 2026-02-20T12:10:55.425 INFO:teuthology.orchestra.run.trial131.stdout:Rados Block Device engine yes 2026-02-20T12:10:55.489 INFO:teuthology.orchestra.run.trial131.stdout:rbd_poll yes 2026-02-20T12:10:55.551 INFO:teuthology.orchestra.run.trial131.stdout:rbd_invalidate_cache yes 2026-02-20T12:10:55.613 INFO:teuthology.orchestra.run.trial131.stdout:rbd_encryption_load yes 2026-02-20T12:10:55.633 INFO:teuthology.orchestra.run.trial131.stdout:setvbuf yes 2026-02-20T12:10:55.639 INFO:teuthology.orchestra.run.trial131.stdout:Gluster API engine no 2026-02-20T12:10:55.650 INFO:teuthology.orchestra.run.trial131.stdout:s390_z196_facilities no 2026-02-20T12:10:55.650 INFO:teuthology.orchestra.run.trial131.stdout:HDFS engine no 2026-02-20T12:10:55.672 INFO:teuthology.orchestra.run.trial131.stdout:MTD yes 2026-02-20T12:10:55.677 INFO:teuthology.orchestra.run.trial131.stdout:libpmem no 2026-02-20T12:10:55.677 INFO:teuthology.orchestra.run.trial131.stdout:libpmem1_5 no 2026-02-20T12:10:55.682 INFO:teuthology.orchestra.run.trial131.stdout:libpmem2 no 2026-02-20T12:10:55.682 INFO:teuthology.orchestra.run.trial131.stdout:PMDK dev-dax engine no 2026-02-20T12:10:55.682 INFO:teuthology.orchestra.run.trial131.stdout:PMDK libpmem engine no 2026-02-20T12:10:55.687 INFO:teuthology.orchestra.run.trial131.stdout:DDN's Infinite Memory Engine no 2026-02-20T12:10:55.687 INFO:teuthology.orchestra.run.trial131.stdout:iscsi engine no 2026-02-20T12:10:55.687 INFO:teuthology.orchestra.run.trial131.stdout:NBD engine no 2026-02-20T12:10:55.693 INFO:teuthology.orchestra.run.trial131.stdout:DAOS File System (dfs) Engine no 2026-02-20T12:10:55.699 INFO:teuthology.orchestra.run.trial131.stdout:NFS engine no 2026-02-20T12:10:55.699 INFO:teuthology.orchestra.run.trial131.stdout:lex/yacc for arithmetic no 2026-02-20T12:10:55.720 INFO:teuthology.orchestra.run.trial131.stdout:getmntent yes 2026-02-20T12:10:55.732 INFO:teuthology.orchestra.run.trial131.stdout:getmntinfo no 2026-02-20T12:10:55.762 INFO:teuthology.orchestra.run.trial131.stdout:Static Assert yes 2026-02-20T12:10:55.780 INFO:teuthology.orchestra.run.trial131.stdout:bool yes 2026-02-20T12:10:55.801 INFO:teuthology.orchestra.run.trial131.stdout:strndup yes 2026-02-20T12:10:55.807 INFO:teuthology.orchestra.run.trial131.stdout:Valgrind headers no 2026-02-20T12:10:55.825 INFO:teuthology.orchestra.run.trial131.stdout:Zoned block device support yes 2026-02-20T12:10:55.844 INFO:teuthology.orchestra.run.trial131.stdout:Zoned block device capacity yes 2026-02-20T12:10:55.849 INFO:teuthology.orchestra.run.trial131.stdout:libzbc engine no 2026-02-20T12:10:55.869 INFO:teuthology.orchestra.run.trial131.stdout:NVMe uring command support yes 2026-02-20T12:10:55.881 INFO:teuthology.orchestra.run.trial131.stdout:xnvme engine no 2026-02-20T12:10:55.886 INFO:teuthology.orchestra.run.trial131.stdout:isal no 2026-02-20T12:10:55.887 INFO:teuthology.orchestra.run.trial131.stdout:isal CRC64 no 2026-02-20T12:10:55.898 INFO:teuthology.orchestra.run.trial131.stdout:libblkio engine no 2026-02-20T12:10:55.899 INFO:teuthology.orchestra.run.trial131.stdout:march_armv8_a_crc_crypto no 2026-02-20T12:10:55.899 INFO:teuthology.orchestra.run.trial131.stdout:cuda no 2026-02-20T12:10:55.899 INFO:teuthology.orchestra.run.trial131.stdout:libcufile no 2026-02-20T12:10:55.919 INFO:teuthology.orchestra.run.trial131.stdout:Build march=native yes 2026-02-20T12:10:55.926 INFO:teuthology.orchestra.run.trial131.stdout:CUnit no 2026-02-20T12:10:55.947 INFO:teuthology.orchestra.run.trial131.stdout:__kernel_rwf_t yes 2026-02-20T12:10:55.966 INFO:teuthology.orchestra.run.trial131.stdout:-Wimplicit-fallthrough=2 yes 2026-02-20T12:10:55.990 INFO:teuthology.orchestra.run.trial131.stdout:-Wno-stringop-truncation yes 2026-02-20T12:10:56.011 INFO:teuthology.orchestra.run.trial131.stdout:MADV_HUGEPAGE yes 2026-02-20T12:10:56.032 INFO:teuthology.orchestra.run.trial131.stdout:gettid yes 2026-02-20T12:10:56.053 INFO:teuthology.orchestra.run.trial131.stdout:statx(2)/libc yes 2026-02-20T12:10:56.076 INFO:teuthology.orchestra.run.trial131.stdout:statx(2)/syscall yes 2026-02-20T12:10:56.078 INFO:teuthology.orchestra.run.trial131.stdout:Windows PDB generation no 2026-02-20T12:10:56.099 INFO:teuthology.orchestra.run.trial131.stdout:timerfd_create yes 2026-02-20T12:10:56.100 INFO:teuthology.orchestra.run.trial131.stdout:Consider installing zlib1g-dev (zlib-devel) as some fio features depend on it. 2026-02-20T12:10:56.101 INFO:teuthology.orchestra.run.trial131.stdout:Lib-based ioengines dynamic no 2026-02-20T12:10:56.140 INFO:teuthology.orchestra.run.trial131.stdout:TCMalloc support yes 2026-02-20T12:10:56.141 INFO:teuthology.orchestra.run.trial131.stdout:seed_buckets 4 2026-02-20T12:10:56.186 INFO:teuthology.orchestra.run.trial131.stderr:FIO_VERSION = fio-3.41-110-g1a04e5 2026-02-20T12:10:56.210 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/crc16.o 2026-02-20T12:10:56.245 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/crc32.o 2026-02-20T12:10:56.273 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/crc32c-arm64.o 2026-02-20T12:10:56.317 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/crc32c-intel.o 2026-02-20T12:10:56.348 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/crc32c.o 2026-02-20T12:10:56.378 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/crc64.o 2026-02-20T12:10:56.406 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/crc7.o 2026-02-20T12:10:56.431 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/crct10dif_common.o 2026-02-20T12:10:56.455 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/fnv.o 2026-02-20T12:10:56.485 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/md5.o 2026-02-20T12:10:56.542 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/murmur3.o 2026-02-20T12:10:56.573 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/sha1.o 2026-02-20T12:10:56.671 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/sha256.o 2026-02-20T12:10:56.811 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/sha3.o 2026-02-20T12:10:56.892 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/sha512.o 2026-02-20T12:10:56.959 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/test.o 2026-02-20T12:10:57.051 INFO:teuthology.orchestra.run.trial131.stdout: CC crc/xxhash.o 2026-02-20T12:10:57.108 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/axmap.o 2026-02-20T12:10:57.179 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/bloom.o 2026-02-20T12:10:57.229 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/flist_sort.o 2026-02-20T12:10:57.273 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/gauss.o 2026-02-20T12:10:57.324 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/getrusage.o 2026-02-20T12:10:57.351 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/hweight.o 2026-02-20T12:10:57.379 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/ieee754.o 2026-02-20T12:10:57.411 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/lfsr.o 2026-02-20T12:10:57.468 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/memalign.o 2026-02-20T12:10:57.500 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/memcpy.o 2026-02-20T12:10:57.584 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/mountcheck.o 2026-02-20T12:10:57.613 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/num2str.o 2026-02-20T12:10:57.658 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/output_buffer.o 2026-02-20T12:10:57.691 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/pattern.o 2026-02-20T12:10:57.770 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/prio_tree.o 2026-02-20T12:10:57.840 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/rand.o 2026-02-20T12:10:57.913 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/rbtree.o 2026-02-20T12:10:57.982 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/strntol.o 2026-02-20T12:10:58.017 INFO:teuthology.orchestra.run.trial131.stdout: CC lib/zipf.o 2026-02-20T12:10:58.083 INFO:teuthology.orchestra.run.trial131.stdout: CC gettime.o 2026-02-20T12:10:58.235 INFO:teuthology.orchestra.run.trial131.stdout: CC ioengines.o 2026-02-20T12:10:58.370 INFO:teuthology.orchestra.run.trial131.stdout: CC init.o 2026-02-20T12:10:58.753 INFO:teuthology.orchestra.run.trial131.stdout: CC stat.o 2026-02-20T12:10:59.461 INFO:teuthology.orchestra.run.trial131.stdout: CC log.o 2026-02-20T12:10:59.533 INFO:teuthology.orchestra.run.trial131.stdout: CC time.o 2026-02-20T12:10:59.621 INFO:teuthology.orchestra.run.trial131.stdout: CC filesetup.o 2026-02-20T12:10:59.955 INFO:teuthology.orchestra.run.trial131.stdout: CC eta.o 2026-02-20T12:11:00.117 INFO:teuthology.orchestra.run.trial131.stdout: CC verify.o 2026-02-20T12:11:00.453 INFO:teuthology.orchestra.run.trial131.stdout: CC memory.o 2026-02-20T12:11:00.545 INFO:teuthology.orchestra.run.trial131.stdout: CC io_u.o 2026-02-20T12:11:00.927 INFO:teuthology.orchestra.run.trial131.stdout: CC parse.o 2026-02-20T12:11:01.150 INFO:teuthology.orchestra.run.trial131.stdout: CC fio_sem.o 2026-02-20T12:11:01.213 INFO:teuthology.orchestra.run.trial131.stdout: CC rwlock.o 2026-02-20T12:11:01.265 INFO:teuthology.orchestra.run.trial131.stdout: CC pshared.o 2026-02-20T12:11:01.308 INFO:teuthology.orchestra.run.trial131.stdout: CC options.o 2026-02-20T12:11:01.708 INFO:teuthology.orchestra.run.trial131.stdout: CC fio_shared_sem.o 2026-02-20T12:11:01.740 INFO:teuthology.orchestra.run.trial131.stdout: CC smalloc.o 2026-02-20T12:11:01.840 INFO:teuthology.orchestra.run.trial131.stdout: CC filehash.o 2026-02-20T12:11:01.925 INFO:teuthology.orchestra.run.trial131.stdout: CC profile.o 2026-02-20T12:11:01.998 INFO:teuthology.orchestra.run.trial131.stdout: CC debug.o 2026-02-20T12:11:02.028 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/cpu.o 2026-02-20T12:11:02.107 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/mmap.o 2026-02-20T12:11:02.196 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/sync.o 2026-02-20T12:11:02.316 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/null.o 2026-02-20T12:11:02.383 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/net.o 2026-02-20T12:11:02.591 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/ftruncate.o 2026-02-20T12:11:02.651 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/fileoperations.o 2026-02-20T12:11:02.743 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/exec.o 2026-02-20T12:11:02.826 INFO:teuthology.orchestra.run.trial131.stdout: CC server.o 2026-02-20T12:11:03.181 INFO:teuthology.orchestra.run.trial131.stdout: CC client.o 2026-02-20T12:11:03.462 INFO:teuthology.orchestra.run.trial131.stdout: CC iolog.o 2026-02-20T12:11:03.688 INFO:teuthology.orchestra.run.trial131.stdout: CC backend.o 2026-02-20T12:11:04.105 INFO:teuthology.orchestra.run.trial131.stdout: CC libfio.o 2026-02-20T12:11:04.202 INFO:teuthology.orchestra.run.trial131.stdout: CC flow.o 2026-02-20T12:11:04.271 INFO:teuthology.orchestra.run.trial131.stdout: CC cconv.o 2026-02-20T12:11:04.491 INFO:teuthology.orchestra.run.trial131.stdout: CC gettime-thread.o 2026-02-20T12:11:04.553 INFO:teuthology.orchestra.run.trial131.stdout: CC helpers.o 2026-02-20T12:11:04.594 INFO:teuthology.orchestra.run.trial131.stdout: CC json.o 2026-02-20T12:11:04.691 INFO:teuthology.orchestra.run.trial131.stdout: CC idletime.o 2026-02-20T12:11:04.812 INFO:teuthology.orchestra.run.trial131.stdout: CC td_error.o 2026-02-20T12:11:04.874 INFO:teuthology.orchestra.run.trial131.stdout: CC profiles/tiobench.o 2026-02-20T12:11:04.931 INFO:teuthology.orchestra.run.trial131.stdout: CC profiles/act.o 2026-02-20T12:11:05.023 INFO:teuthology.orchestra.run.trial131.stdout: CC io_u_queue.o 2026-02-20T12:11:05.057 INFO:teuthology.orchestra.run.trial131.stdout: CC filelock.o 2026-02-20T12:11:05.132 INFO:teuthology.orchestra.run.trial131.stdout: CC workqueue.o 2026-02-20T12:11:05.229 INFO:teuthology.orchestra.run.trial131.stdout: CC rate-submit.o 2026-02-20T12:11:05.314 INFO:teuthology.orchestra.run.trial131.stdout: CC optgroup.o 2026-02-20T12:11:05.347 INFO:teuthology.orchestra.run.trial131.stdout: CC helper_thread.o 2026-02-20T12:11:05.446 INFO:teuthology.orchestra.run.trial131.stdout: CC steadystate.o 2026-02-20T12:11:05.590 INFO:teuthology.orchestra.run.trial131.stdout: CC zone-dist.o 2026-02-20T12:11:05.667 INFO:teuthology.orchestra.run.trial131.stdout: CC zbd.o 2026-02-20T12:11:06.036 INFO:teuthology.orchestra.run.trial131.stdout: CC dedupe.o 2026-02-20T12:11:06.120 INFO:teuthology.orchestra.run.trial131.stdout: CC dataplacement.o 2026-02-20T12:11:06.215 INFO:teuthology.orchestra.run.trial131.stdout: CC sprandom.o 2026-02-20T12:11:06.379 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/posixaio.o 2026-02-20T12:11:06.454 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/falloc.o 2026-02-20T12:11:06.521 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/e4defrag.o 2026-02-20T12:11:06.596 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/splice.o 2026-02-20T12:11:06.696 INFO:teuthology.orchestra.run.trial131.stdout: CC oslib/asprintf.o 2026-02-20T12:11:06.723 INFO:teuthology.orchestra.run.trial131.stdout: CC oslib/strlcat.o 2026-02-20T12:11:06.756 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/mtd.o 2026-02-20T12:11:06.845 INFO:teuthology.orchestra.run.trial131.stdout: CC oslib/libmtd.o 2026-02-20T12:11:07.038 INFO:teuthology.orchestra.run.trial131.stdout: CC oslib/libmtd_legacy.o 2026-02-20T12:11:07.111 INFO:teuthology.orchestra.run.trial131.stdout: CC diskutil.o 2026-02-20T12:11:07.222 INFO:teuthology.orchestra.run.trial131.stdout: CC fifo.o 2026-02-20T12:11:07.258 INFO:teuthology.orchestra.run.trial131.stdout: CC blktrace.o 2026-02-20T12:11:07.421 INFO:teuthology.orchestra.run.trial131.stdout: CC cgroup.o 2026-02-20T12:11:07.504 INFO:teuthology.orchestra.run.trial131.stdout: CC trim.o 2026-02-20T12:11:07.572 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/sg.o 2026-02-20T12:11:07.763 INFO:teuthology.orchestra.run.trial131.stdout: CC oslib/linux-dev-lookup.o 2026-02-20T12:11:07.802 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/io_uring.o 2026-02-20T12:11:08.033 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/nvme.o 2026-02-20T12:11:08.193 INFO:teuthology.orchestra.run.trial131.stdout: CC oslib/linux-blkzoned.o 2026-02-20T12:11:08.280 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/cmdprio.o 2026-02-20T12:11:08.405 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/rados.o 2026-02-20T12:11:08.504 INFO:teuthology.orchestra.run.trial131.stdout: CC engines/rbd.o 2026-02-20T12:11:08.610 INFO:teuthology.orchestra.run.trial131.stdout: CC fio.o 2026-02-20T12:11:08.666 INFO:teuthology.orchestra.run.trial131.stdout: LINK fio 2026-02-20T12:11:08.786 INFO:teuthology.orchestra.run.trial131.stdout: CC t/log.o 2026-02-20T12:11:08.816 INFO:teuthology.orchestra.run.trial131.stdout: CC t/genzipf.o 2026-02-20T12:11:08.890 INFO:teuthology.orchestra.run.trial131.stdout: CC oslib/strcasestr.o 2026-02-20T12:11:08.911 INFO:teuthology.orchestra.run.trial131.stdout: CC oslib/strndup.o 2026-02-20T12:11:08.931 INFO:teuthology.orchestra.run.trial131.stdout: LINK t/fio-genzipf 2026-02-20T12:11:08.985 INFO:teuthology.orchestra.run.trial131.stdout: CC t/btrace2fio.o 2026-02-20T12:11:09.183 INFO:teuthology.orchestra.run.trial131.stdout: LINK t/fio-btrace2fio 2026-02-20T12:11:09.239 INFO:teuthology.orchestra.run.trial131.stdout: CC t/verify-state.o 2026-02-20T12:11:09.298 INFO:teuthology.orchestra.run.trial131.stdout: CC t/debug.o 2026-02-20T12:11:09.325 INFO:teuthology.orchestra.run.trial131.stdout: LINK t/fio-verify-state 2026-02-20T12:11:09.378 INFO:teuthology.orchestra.run.trial131.stdout: CC t/stest.o 2026-02-20T12:11:09.425 INFO:teuthology.orchestra.run.trial131.stdout: CC t/arch.o 2026-02-20T12:11:09.449 INFO:teuthology.orchestra.run.trial131.stdout: LINK t/stest 2026-02-20T12:11:09.505 INFO:teuthology.orchestra.run.trial131.stdout: CC t/ieee754.o 2026-02-20T12:11:09.535 INFO:teuthology.orchestra.run.trial131.stdout: LINK t/ieee754 2026-02-20T12:11:09.587 INFO:teuthology.orchestra.run.trial131.stdout: CC t/axmap.o 2026-02-20T12:11:09.646 INFO:teuthology.orchestra.run.trial131.stdout: LINK t/axmap 2026-02-20T12:11:09.699 INFO:teuthology.orchestra.run.trial131.stdout: CC t/lfsr-test.o 2026-02-20T12:11:09.753 INFO:teuthology.orchestra.run.trial131.stdout: LINK t/lfsr-test 2026-02-20T12:11:09.809 INFO:teuthology.orchestra.run.trial131.stdout: CC t/gen-rand.o 2026-02-20T12:11:09.864 INFO:teuthology.orchestra.run.trial131.stdout: LINK t/gen-rand 2026-02-20T12:11:09.919 INFO:teuthology.orchestra.run.trial131.stdout: CC t/memlock.o 2026-02-20T12:11:09.959 INFO:teuthology.orchestra.run.trial131.stdout: LINK t/memlock 2026-02-20T12:11:10.013 INFO:teuthology.orchestra.run.trial131.stdout: CC t/read-to-pipe-async.o 2026-02-20T12:11:10.116 INFO:teuthology.orchestra.run.trial131.stdout: LINK t/read-to-pipe-async 2026-02-20T12:11:10.174 INFO:teuthology.orchestra.run.trial131.stdout: CC t/io_uring.o 2026-02-20T12:11:10.448 INFO:teuthology.orchestra.run.trial131.stdout: LINK t/io_uring 2026-02-20T12:11:10.503 DEBUG:teuthology.orchestra.run.trial131:> sudo /sbin/sysctl -q -w kernel.perf_event_paranoid=0 2026-02-20T12:11:10.529 DEBUG:teuthology.orchestra.run.trial131:> /home/ubuntu/cephtest/cbt/cbt.py -a /home/ubuntu/cephtest/archive/cbt /home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2026-02-20T12:11:10.641 INFO:teuthology.orchestra.run.trial131.stderr:12:11:10 - DEBUG - cbt - Settings.cluster: 2026-02-20T12:11:10.641 INFO:teuthology.orchestra.run.trial131.stderr: {'archive_dir': '/home/ubuntu/cephtest/archive/cbt', 2026-02-20T12:11:10.641 INFO:teuthology.orchestra.run.trial131.stderr: 'clients': ['ubuntu@trial131.front.sepia.ceph.com'], 2026-02-20T12:11:10.642 INFO:teuthology.orchestra.run.trial131.stderr: 'conf_file': '/etc/ceph/ceph.conf', 2026-02-20T12:11:10.642 INFO:teuthology.orchestra.run.trial131.stderr: 'head': 'ubuntu@trial131.front.sepia.ceph.com', 2026-02-20T12:11:10.642 INFO:teuthology.orchestra.run.trial131.stderr: 'is_teuthology': True, 2026-02-20T12:11:10.642 INFO:teuthology.orchestra.run.trial131.stderr: 'iterations': 1, 2026-02-20T12:11:10.642 INFO:teuthology.orchestra.run.trial131.stderr: 'mons': ['ubuntu@trial131.front.sepia.ceph.com'], 2026-02-20T12:11:10.642 INFO:teuthology.orchestra.run.trial131.stderr: 'osds': ['ubuntu@trial131.front.sepia.ceph.com'], 2026-02-20T12:11:10.642 INFO:teuthology.orchestra.run.trial131.stderr: 'osds_per_node': 3, 2026-02-20T12:11:10.642 INFO:teuthology.orchestra.run.trial131.stderr: 'pid_dir': '/var/run/ceph', 2026-02-20T12:11:10.642 INFO:teuthology.orchestra.run.trial131.stderr: 'pool_profiles': {'rbd': {'pg_size': 128, 'pgp_size': 128, 'replication': 3}}, 2026-02-20T12:11:10.642 INFO:teuthology.orchestra.run.trial131.stderr: 'rebuild_every_test': False, 2026-02-20T12:11:10.642 INFO:teuthology.orchestra.run.trial131.stderr: 'rgws': {'ubuntu@trial131.front.sepia.ceph.com': None}, 2026-02-20T12:11:10.643 INFO:teuthology.orchestra.run.trial131.stderr: 'tmp_dir': '/tmp/cbt', 2026-02-20T12:11:10.643 INFO:teuthology.orchestra.run.trial131.stderr: 'use_existing': True, 2026-02-20T12:11:10.643 INFO:teuthology.orchestra.run.trial131.stderr: 'user': 'ubuntu'} 2026-02-20T12:11:10.643 INFO:teuthology.orchestra.run.trial131.stderr:12:11:10 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-4c80a9be 2026-02-20T12:11:10.643 INFO:teuthology.orchestra.run.trial131.stderr:12:11:10 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com if [ -d "/tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw" ]; then rm -rf /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw; fi 2026-02-20T12:11:10.805 INFO:teuthology.orchestra.run.trial131.stderr:12:11:10 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw 2026-02-20T12:11:10.966 INFO:teuthology.orchestra.run.trial131.stderr:12:11:10 - INFO - cbt - Pausing for 60s for idle monitoring. 2026-02-20T12:11:10.966 INFO:teuthology.orchestra.run.trial131.stderr:12:11:10 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/idle_monitoring/collectl 2026-02-20T12:11:11.131 INFO:teuthology.orchestra.run.trial131.stderr:12:11:11 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/idle_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-20T12:11:11.133 INFO:teuthology.orchestra.run.trial131.stderr:12:11:11 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/idle_monitoring/perf 2026-02-20T12:11:11.299 INFO:teuthology.orchestra.run.trial131.stderr:12:11:11 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-20T12:11:11.299 INFO:teuthology.orchestra.run.trial131.stderr:12:11:11 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/idle_monitoring/perf/perf_stat.${pid} & done 2026-02-20T12:12:11.340 INFO:teuthology.orchestra.run.trial131.stderr:12:12:11 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-20T12:12:11.523 INFO:teuthology.orchestra.run.trial131.stderr:12:12:11 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-20T12:12:11.703 INFO:teuthology.orchestra.run.trial131.stderr:12:12:11 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/* 2026-02-20T12:12:11.869 INFO:teuthology.orchestra.run.trial131.stderr:12:12:11 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial131.front.sepia.ceph.com -r /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-4c80a9be 2026-02-20T12:12:12.026 INFO:teuthology.orchestra.run.trial131.stderr:12:12:12 - INFO - cbt - Creating fio images... 2026-02-20T12:12:12.026 INFO:teuthology.orchestra.run.trial131.stderr:12:12:12 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/pool_monitoring/collectl 2026-02-20T12:12:12.181 INFO:teuthology.orchestra.run.trial131.stderr:12:12:12 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/pool_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-20T12:12:12.183 INFO:teuthology.orchestra.run.trial131.stderr:12:12:12 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/pool_monitoring/perf 2026-02-20T12:12:12.340 INFO:teuthology.orchestra.run.trial131.stderr:12:12:12 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-20T12:12:12.340 INFO:teuthology.orchestra.run.trial131.stderr:12:12:12 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/pool_monitoring/perf/perf_stat.${pid} & done 2026-02-20T12:12:12.341 INFO:teuthology.orchestra.run.trial131.stderr:12:12:12 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool delete cbt-librbdfio cbt-librbdfio --yes-i-really-really-mean-it 2026-02-20T12:12:12.682 INFO:teuthology.orchestra.run.trial131.stderr:12:12:12 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool create cbt-librbdfio 128 128 2026-02-20T12:12:13.053 INFO:teuthology.orchestra.run.trial131.stderr:12:12:13 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool application enable cbt-librbdfio rbd 2026-02-20T12:12:14.056 INFO:teuthology.orchestra.run.trial131.stderr:12:12:14 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool set cbt-librbdfio size 3 2026-02-20T12:12:15.070 INFO:teuthology.orchestra.run.trial131.stderr:12:12:15 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool set cbt-librbdfio min_size 2 2026-02-20T12:12:16.063 INFO:teuthology.orchestra.run.trial131.stderr:12:12:16 - INFO - cbt - Checking Health after pool creation. 2026-02-20T12:12:21.068 INFO:teuthology.orchestra.run.trial131.stderr:12:12:21 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com /usr/bin/ceph -c /etc/ceph/ceph.conf health 2026-02-20T12:12:21.419 INFO:teuthology.orchestra.run.trial131.stderr:12:12:21 - INFO - cbt - Final Pool Health Check. 2026-02-20T12:12:26.425 INFO:teuthology.orchestra.run.trial131.stderr:12:12:26 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com /usr/bin/ceph -c /etc/ceph/ceph.conf health 2026-02-20T12:12:26.777 INFO:teuthology.orchestra.run.trial131.stderr:12:12:26 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com hostname -f 2026-02-20T12:12:26.945 INFO:teuthology.orchestra.run.trial131.stderr:12:12:26 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com /usr/bin/rbd -c /etc/ceph/ceph.conf create cbt-librbdfio-trial131-0 --size 4096 --pool cbt-librbdfio --order 22 2026-02-20T12:12:29.177 INFO:teuthology.orchestra.run.trial131.stderr:12:12:29 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com /usr/bin/rbd -c /etc/ceph/ceph.conf create cbt-librbdfio-trial131-1 --size 4096 --pool cbt-librbdfio --order 22 2026-02-20T12:12:29.422 INFO:teuthology.orchestra.run.trial131.stderr:12:12:29 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-20T12:12:29.591 INFO:teuthology.orchestra.run.trial131.stderr:12:12:29 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-20T12:12:29.776 INFO:teuthology.orchestra.run.trial131.stderr:12:12:29 - INFO - cbt - Attempting to prefill fio images... 2026-02-20T12:12:29.776 INFO:teuthology.orchestra.run.trial131.stderr:12:12:29 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo /home/ubuntu/cephtest/fio/fio --ioengine=rbd --clientname=admin --pool=cbt-librbdfio --rbdname=cbt-librbdfio-`hostname -f`-0 --invalidate=0 --rw=write --numjobs=1 --bs=4M --size 4096M --name=cbt-librbdfio-`hostname -f`-file-0 --output-format=json,normal > /dev/null 2026-02-20T12:12:29.777 INFO:teuthology.orchestra.run.trial131.stderr:12:12:29 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo /home/ubuntu/cephtest/fio/fio --ioengine=rbd --clientname=admin --pool=cbt-librbdfio --rbdname=cbt-librbdfio-`hostname -f`-1 --invalidate=0 --rw=write --numjobs=1 --bs=4M --size 4096M --name=cbt-librbdfio-`hostname -f`-file-0 --output-format=json,normal > /dev/null 2026-02-20T12:13:08.108 INFO:teuthology.orchestra.run.trial131.stderr:12:13:08 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo /home/ubuntu/cephtest/fio/fio --ioengine=rbd --clientname=admin --pool=cbt-librbdfio --rbdname=cbt-librbdfio-`hostname -f`-0 --invalidate=0 --rw=write --numjobs=1 --bs=4M --size 4096M --name=cbt-librbdfio-`hostname -f`-file-0 --output-format=json,normal > /dev/null 2026-02-20T12:13:08.110 INFO:teuthology.orchestra.run.trial131.stderr:12:13:08 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo /home/ubuntu/cephtest/fio/fio --ioengine=rbd --clientname=admin --pool=cbt-librbdfio --rbdname=cbt-librbdfio-`hostname -f`-1 --invalidate=0 --rw=write --numjobs=1 --bs=4M --size 4096M --name=cbt-librbdfio-`hostname -f`-file-0 --output-format=json,normal > /dev/null 2026-02-20T12:13:30.936 INFO:teuthology.orchestra.run.trial131.stderr:12:13:30 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-4c80a9be 2026-02-20T12:13:30.937 INFO:teuthology.orchestra.run.trial131.stderr:12:13:30 - INFO - cbt - Skipping existing test in /home/ubuntu/cephtest/archive/cbt/results/00000000/id-4c80a9be. 2026-02-20T12:13:30.937 INFO:teuthology.orchestra.run.trial131.stderr:12:13:30 - INFO - cbt - Running benchmark /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw 2026-02-20T12:13:30.937 INFO:teuthology.orchestra.run.trial131.stderr:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-4c80a9be 2026-02-20T12:13:30.937 INFO:teuthology.orchestra.run.trial131.stderr:{'cmd_path': '/home/ubuntu/cephtest/fio/fio', 'log_avg_msec': 100, 'norandommap': True, 'pool_profile': 'rbd', 'time': 60, 'vol_size': 4096, 'iodepth': 32, 'mode': 'randrw', 'op_size': 4096, 'osd_ra': 4096, 'procs_per_volume': 1, 'volumes_per_client': 2, 'benchmark': 'librbdfio', 'iteration': 0} == iteration 0 == 2026-02-20T12:13:30.937 INFO:teuthology.orchestra.run.trial131.stderr:12:13:30 - INFO - cbt - Setting OSD Read Ahead to: 4096 2026-02-20T12:13:30.937 INFO:teuthology.orchestra.run.trial131.stderr:12:13:30 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com find /dev/disk/by-partlabel/osd-device-*data -exec readlink {} \; | cut -d"/" -f 3 | sed "s/[0-9]$//" | xargs -I{} sudo sh -c "echo 4096 > /sys/block/'{}'/queue/read_ahead_kb" 2026-02-20T12:13:30.938 INFO:teuthology.orchestra.run.trial131.stderr:12:13:30 - DEBUG - cbt - Cleaning existing temporary run directory: /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw 2026-02-20T12:13:30.938 INFO:teuthology.orchestra.run.trial131.stderr:12:13:30 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo rm -rf /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw 2026-02-20T12:13:31.106 INFO:teuthology.orchestra.run.trial131.stderr:12:13:31 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sync 2026-02-20T12:13:31.395 INFO:teuthology.orchestra.run.trial131.stderr:12:13:31 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com echo 3 | sudo tee /proc/sys/vm/drop_caches 2026-02-20T12:13:32.540 INFO:teuthology.orchestra.run.trial131.stderr:12:13:32 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw 2026-02-20T12:13:32.732 INFO:teuthology.orchestra.run.trial131.stderr:12:13:32 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf daemon osd.0 config show > /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/ceph_settings.out 2026-02-20T12:13:58.049 INFO:teuthology.orchestra.run.trial131.stderr:12:13:58 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com echo "[`date`] ceph progress" >> /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/pgautoscaler.log;/usr/bin/ceph -c /etc/ceph/ceph.conf progress | tee -a /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/pgautoscaler.log 2026-02-20T12:13:58.412 INFO:teuthology.orchestra.run.trial131.stderr:12:13:58 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/collectl 2026-02-20T12:13:58.572 INFO:teuthology.orchestra.run.trial131.stderr:12:13:58 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-20T12:13:58.573 INFO:teuthology.orchestra.run.trial131.stderr:12:13:58 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/perf 2026-02-20T12:13:58.729 INFO:teuthology.orchestra.run.trial131.stderr:12:13:58 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-20T12:13:58.729 INFO:teuthology.orchestra.run.trial131.stderr:12:13:58 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/perf/perf_stat.${pid} & done 2026-02-20T12:13:58.731 INFO:teuthology.orchestra.run.trial131.stderr:12:13:58 - INFO - cbt - Running rbd fio randrw test. 2026-02-20T12:13:58.731 INFO:teuthology.orchestra.run.trial131.stderr:12:13:58 - DEBUG - cbt - Using rbdname cbt-librbdfio-`hostname -f`-0 2026-02-20T12:13:58.731 INFO:teuthology.orchestra.run.trial131.stderr:12:13:58 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo /home/ubuntu/cephtest/fio/fio --ioengine=rbd --clientname=admin --pool=cbt-librbdfio --rbdname=cbt-librbdfio-`hostname -f`-0 --invalidate=0 --rw=randrw --output-format=json,normal --rwmixread=50 --rwmixwrite=50 --runtime=60 --numjobs=1 --direct=1 --bs=4096B --iodepth=32 --end_fsync=0 --norandommap --write_iops_log=/tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/output.0 --write_bw_log=/tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/output.0 --write_lat_log=/tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/output.0 --log_avg_msec=100 --name=cbt-librbdfio-`hostname -f`-file-0 > /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/output.0 2026-02-20T12:13:58.732 INFO:teuthology.orchestra.run.trial131.stderr:12:13:58 - DEBUG - cbt - Using rbdname cbt-librbdfio-`hostname -f`-1 2026-02-20T12:13:58.733 INFO:teuthology.orchestra.run.trial131.stderr:12:13:58 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo /home/ubuntu/cephtest/fio/fio --ioengine=rbd --clientname=admin --pool=cbt-librbdfio --rbdname=cbt-librbdfio-`hostname -f`-1 --invalidate=0 --rw=randrw --output-format=json,normal --rwmixread=50 --rwmixwrite=50 --runtime=60 --numjobs=1 --direct=1 --bs=4096B --iodepth=32 --end_fsync=0 --norandommap --write_iops_log=/tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/output.1 --write_bw_log=/tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/output.1 --write_lat_log=/tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/output.1 --log_avg_msec=100 --name=cbt-librbdfio-`hostname -f`-file-0 > /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/output.1 2026-02-20T12:14:59.468 INFO:teuthology.orchestra.run.trial131.stderr:12:14:59 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-20T12:14:59.635 INFO:teuthology.orchestra.run.trial131.stderr:12:14:59 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-20T12:14:59.810 INFO:teuthology.orchestra.run.trial131.stderr:12:14:59 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/perf/perf.data 2026-02-20T12:14:59.812 INFO:teuthology.orchestra.run.trial131.stderr:12:14:59 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/perf/perf_stat.* 2026-02-20T12:14:59.814 INFO:teuthology.orchestra.run.trial131.stderr:12:14:59 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com find /var/run/ceph/ceph-osd*.asok -maxdepth 1 -exec sudo /usr/bin/ceph --admin-daemon {} dump_historic_ops \; > /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/historic_ops.out 2026-02-20T12:15:00.165 INFO:teuthology.orchestra.run.trial131.stderr:12:15:00 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial131.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/* 2026-02-20T12:15:00.337 INFO:teuthology.orchestra.run.trial131.stderr:12:15:00 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial131.front.sepia.ceph.com -r /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-4c80a9be 2026-02-20T12:15:00.502 INFO:teuthology.orchestra.run.trial131.stderr:12:15:00 - INFO - cbt - Convert results to json format. 2026-02-20T12:15:00.503 INFO:teuthology.orchestra.run.trial131.stdout:cleaning remote dir /tmp/cbt/00000000/LibrbdFio/osd_ra-00004096/op_size-00004096/concurrent_procs-002/iodepth-032/randrw 2026-02-20T12:15:00.503 INFO:teuthology.orchestra.run.trial131.stdout:trial131: trial131 2026-02-20T12:15:00.503 INFO:teuthology.orchestra.run.trial131.stdout: 2026-02-20T12:15:00.503 INFO:teuthology.orchestra.run.trial131.stdout:['trial131'] 2026-02-20T12:15:00.515 DEBUG:teuthology.run_tasks:Unwinding manager cbt 2026-02-20T12:15:00.532 DEBUG:teuthology.orchestra.run.trial131:> rm --one-file-system -rf -- /home/ubuntu/cephtest/cbt 2026-02-20T12:15:00.560 DEBUG:teuthology.orchestra.run.trial131:> rm --one-file-system -rf -- /home/ubuntu/cephtest/fio 2026-02-20T12:15:00.644 INFO:tasks.cbt_performance:Collecting CBT performance data config 2026-02-20T12:15:00.644 INFO:tasks.cbt_performance:reading cbt preferences 2026-02-20T12:15:00.644 DEBUG:teuthology.orchestra.run.trial131:> find /home/ubuntu/cephtest -name 'json_output.*' 2026-02-20T12:15:00.702 INFO:teuthology.orchestra.run.trial131.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-4c80a9be/json_output.0.trial131.front.sepia.ceph.com 2026-02-20T12:15:00.702 INFO:teuthology.orchestra.run.trial131.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-4c80a9be/json_output.1.trial131.front.sepia.ceph.com 2026-02-20T12:15:00.703 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:15:00.703 DEBUG:teuthology.orchestra.run.trial131:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-4c80a9be/json_output.0.trial131.front.sepia.ceph.com of=/dev/stdout 2026-02-20T12:15:00.763 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-20T12:15:00.763 DEBUG:teuthology.orchestra.run.trial131:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-4c80a9be -name 'perf_stat.*' 2026-02-20T12:15:00.819 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-20T12:15:00.819 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-20T12:15:00.820 DEBUG:teuthology.orchestra.run.trial131:> set -ex 2026-02-20T12:15:00.820 DEBUG:teuthology.orchestra.run.trial131:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-4c80a9be/json_output.1.trial131.front.sepia.ceph.com of=/dev/stdout 2026-02-20T12:15:00.880 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-20T12:15:00.880 DEBUG:teuthology.orchestra.run.trial131:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-4c80a9be -name 'perf_stat.*' 2026-02-20T12:15:00.936 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-20T12:15:00.937 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-20T12:15:00.942 WARNING:tasks.cbt_performance:Could not send performance data to http://mira118.front.sepia.ceph.com:4000/cbt_performance: HTTPConnectionPool(host='mira118.front.sepia.ceph.com', port=4000): Max retries exceeded with url: /cbt_performance (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known')) 2026-02-20T12:15:00.943 WARNING:tasks.cbt_performance:Continuing without performance data collection 2026-02-20T12:15:00.943 DEBUG:teuthology.run_tasks:Unwinding manager ssh_keys 2026-02-20T12:15:00.959 INFO:teuthology.task.ssh_keys:Cleaning up SSH keys 2026-02-20T12:15:00.959 INFO:teuthology.task.ssh_keys:cleaning up keys added for testing 2026-02-20T12:15:00.959 INFO:teuthology.task.ssh_keys: cleaning up keys for user ubuntu on trial131.front.sepia.ceph.com 2026-02-20T12:15:00.959 DEBUG:teuthology.orchestra.run.trial131:> rm -- /home/ubuntu/.ssh/id_rsa 2026-02-20T12:15:00.993 DEBUG:teuthology.orchestra.run.trial131:> rm -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-20T12:15:01.049 DEBUG:teuthology.orchestra.run.trial131:> stat -c '"%a"' /home/ubuntu/.ssh/authorized_keys 2026-02-20T12:15:01.106 INFO:teuthology.orchestra.run.trial131.stdout:"600" 2026-02-20T12:15:01.106 DEBUG:teuthology.orchestra.run.trial131:> mv -- /home/ubuntu/.ssh/authorized_keys_2026-02-20_12:10:38:217574 /home/ubuntu/.ssh/authorized_keys 2026-02-20T12:15:01.161 DEBUG:teuthology.orchestra.run.trial131:> chmod 600 /home/ubuntu/.ssh/authorized_keys 2026-02-20T12:15:01.217 DEBUG:teuthology.orchestra.run.trial131:> rm /home/ubuntu/.ssh/config 2026-02-20T12:15:01.273 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-20T12:15:01.290 DEBUG:teuthology.orchestra.run.trial131:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-20T12:15:01.389 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:15:01.387+0000 7f9e2230b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:15:01.390 INFO:teuthology.orchestra.run.trial131.stderr:2026-02-20T12:15:01.392+0000 7f9e2230b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T12:15:01.515 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-20T12:15:01.515 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-20T12:15:01.515 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-20T12:15:01.516 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T12:15:06.063 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2026-02-20T12:15:07.617 INFO:tasks.ceph.osd.0:Stopped 2026-02-20T12:15:07.618 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-20T12:15:07.618 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T12:15:07.718 INFO:tasks.ceph.osd.1:Stopped 2026-02-20T12:15:07.718 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-20T12:15:07.719 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T12:15:11.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2026-02-20T12:15:11.264 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 has been restored 2026-02-20T12:15:13.820 INFO:tasks.ceph.osd.2:Stopped 2026-02-20T12:15:13.820 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-20T12:15:13.821 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-20T12:15:13.821 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T12:15:13.858 INFO:tasks.ceph.mgr.x:Stopped 2026-02-20T12:15:13.858 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-20T12:15:13.858 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-20T12:15:13.858 INFO:teuthology.orchestra.run:waiting for 300 2026-02-20T12:15:13.907 INFO:tasks.ceph.mon.a:Stopped 2026-02-20T12:15:13.907 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-20T12:15:13.908 DEBUG:teuthology.orchestra.run.trial131:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v '\(PG_' | grep -E -v '\(OSD_' | grep -E -v '\(OBJECT_' | grep -E -v 'overall HEALTH' | head -n 1 2026-02-20T12:15:13.973 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial131.front.sepia.ceph.com 2026-02-20T12:15:13.973 DEBUG:teuthology.orchestra.run.trial131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-20T12:15:14.042 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial131.front.sepia.ceph.com 2026-02-20T12:15:14.042 DEBUG:teuthology.orchestra.run.trial131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-20T12:15:14.115 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial131.front.sepia.ceph.com 2026-02-20T12:15:14.115 DEBUG:teuthology.orchestra.run.trial131:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-20T12:15:14.184 INFO:tasks.ceph:Archiving mon data... 2026-02-20T12:15:14.186 DEBUG:teuthology.misc:Transferring archived files from trial131:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:29:19-crimson-rados-wip-perezjos-crimson-only-12-02-2026-PR66245-distro-debug-trial/60625/data/mon.a.tgz 2026-02-20T12:15:14.186 DEBUG:teuthology.orchestra.run.trial131:> mktemp 2026-02-20T12:15:14.239 INFO:teuthology.orchestra.run.trial131.stdout:/tmp/tmp.azXaHar2tT 2026-02-20T12:15:14.239 DEBUG:teuthology.orchestra.run.trial131:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.azXaHar2tT 2026-02-20T12:15:14.333 DEBUG:teuthology.orchestra.run.trial131:> sudo chmod 0666 /tmp/tmp.azXaHar2tT 2026-02-20T12:15:14.415 DEBUG:teuthology.orchestra.remote:trial131:/tmp/tmp.azXaHar2tT is 208KB 2026-02-20T12:15:14.507 DEBUG:teuthology.orchestra.run.trial131:> rm -fr /tmp/tmp.azXaHar2tT 2026-02-20T12:15:14.524 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-20T12:15:14.524 DEBUG:teuthology.orchestra.run.trial131:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-20T12:15:14.643 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-20T12:15:14.643 INFO:tasks.ceph:Archiving crash dumps... 2026-02-20T12:15:14.645 DEBUG:teuthology.misc:Transferring archived files from trial131:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:29:19-crimson-rados-wip-perezjos-crimson-only-12-02-2026-PR66245-distro-debug-trial/60625/remote/trial131/crash 2026-02-20T12:15:14.646 DEBUG:teuthology.orchestra.run.trial131:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-20T12:15:14.706 INFO:tasks.ceph:Compressing logs... 2026-02-20T12:15:14.707 DEBUG:teuthology.orchestra.run.trial131:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-20T12:15:14.729 INFO:teuthology.orchestra.run.trial131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-20T12:15:14.730 INFO:teuthology.orchestra.run.trial131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-20T12:15:14.730 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47249.log 2026-02-20T12:15:14.730 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40495.log 2026-02-20T12:15:14.730 INFO:teuthology.orchestra.run.trial131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-20T12:15:14.730 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.47249.log: /var/log/ceph/ceph-client.admin.40495.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40950.log 2026-02-20T12:15:14.731 INFO:teuthology.orchestra.run.trial131.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40495.log.gz 2026-02-20T12:15:14.731 INFO:teuthology.orchestra.run.trial131.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.47249.log.gz 2026-02-20T12:15:14.731 INFO:teuthology.orchestra.run.trial131.stderr: 91.0% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-20T12:15:14.731 INFO:teuthology.orchestra.run.trial131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40672.log 2026-02-20T12:15:14.731 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-osd.0.log: /var/log/ceph/ceph-client.admin.40950.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47251.log 2026-02-20T12:15:14.731 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40950.log.gz 2026-02-20T12:15:14.731 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.40672.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41297.log 2026-02-20T12:15:14.731 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40672.log.gz 2026-02-20T12:15:14.732 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.47251.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40045.log 2026-02-20T12:15:14.732 INFO:teuthology.orchestra.run.trial131.stderr: 50.8% -- replaced with /var/log/ceph/ceph-client.admin.47251.log.gz 2026-02-20T12:15:14.732 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.41297.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.38614.log 2026-02-20T12:15:14.732 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41297.log.gz 2026-02-20T12:15:14.732 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.40045.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40182.log 2026-02-20T12:15:14.732 INFO:teuthology.orchestra.run.trial131.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40045.log.gz 2026-02-20T12:15:14.732 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph.tmp-client.admin.38614.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40901.log 2026-02-20T12:15:14.733 INFO:teuthology.orchestra.run.trial131.stderr: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.38614.log.gz 2026-02-20T12:15:14.733 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.40182.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46620.log 2026-02-20T12:15:14.733 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40182.log.gz 2026-02-20T12:15:14.733 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.40901.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39903.log 2026-02-20T12:15:14.733 INFO:teuthology.orchestra.run.trial131.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40901.log.gz 2026-02-20T12:15:14.733 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.46620.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40999.log 2026-02-20T12:15:14.733 INFO:teuthology.orchestra.run.trial131.stderr: 51.2% -- replaced with /var/log/ceph/ceph-client.admin.46620.log.gz 2026-02-20T12:15:14.734 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.39903.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41440.log 2026-02-20T12:15:14.734 INFO:teuthology.orchestra.run.trial131.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39903.log.gz 2026-02-20T12:15:14.734 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.40999.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46939.log 2026-02-20T12:15:14.734 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40999.log.gz 2026-02-20T12:15:14.734 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.41440.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47429.log 2026-02-20T12:15:14.734 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41440.log.gz 2026-02-20T12:15:14.734 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.46939.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46884.log 2026-02-20T12:15:14.735 INFO:teuthology.orchestra.run.trial131.stderr: 51.2% -- replaced with /var/log/ceph/ceph-client.admin.46939.log.gz 2026-02-20T12:15:14.735 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.47429.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40526.log 2026-02-20T12:15:14.735 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47429.log.gz 2026-02-20T12:15:14.735 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.46884.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41298.log 2026-02-20T12:15:14.735 INFO:teuthology.orchestra.run.trial131.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.46884.log.gz 2026-02-20T12:15:14.735 INFO:teuthology.orchestra.run.trial131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47997.log 2026-02-20T12:15:14.736 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.40526.log: /var/log/ceph/ceph-client.admin.41298.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40526.log.gz 2026-02-20T12:15:14.736 INFO:teuthology.orchestra.run.trial131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40740.log 2026-02-20T12:15:14.736 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41298.log.gz 2026-02-20T12:15:14.736 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.47997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47025.log 2026-02-20T12:15:14.736 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47997.log.gz 2026-02-20T12:15:14.736 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.40740.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46852.log 2026-02-20T12:15:14.736 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40740.log.gz 2026-02-20T12:15:14.737 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.47025.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39998.log 2026-02-20T12:15:14.737 INFO:teuthology.orchestra.run.trial131.stderr: 67.2% -- replaced with /var/log/ceph/ceph-client.admin.47025.log.gz 2026-02-20T12:15:14.737 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.46852.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40810.log 2026-02-20T12:15:14.737 INFO:teuthology.orchestra.run.trial131.stderr: 50.8% -- replaced with /var/log/ceph/ceph-client.admin.46852.log.gz 2026-02-20T12:15:14.737 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.39998.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41150.log 2026-02-20T12:15:14.737 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39998.log.gz 2026-02-20T12:15:14.738 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.40810.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47427.log 2026-02-20T12:15:14.738 INFO:teuthology.orchestra.run.trial131.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40810.log.gz 2026-02-20T12:15:14.738 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.41150.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-20T12:15:14.738 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41150.log.gz 2026-02-20T12:15:14.738 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.47427.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41296.log 2026-02-20T12:15:14.738 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47427.log.gz 2026-02-20T12:15:14.738 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40835.log 2026-02-20T12:15:14.738 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.41296.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47995.log 2026-02-20T12:15:14.739 INFO:teuthology.orchestra.run.trial131.stderr: 86.5% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-20T12:15:14.739 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.40835.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41296.log.gz 2026-02-20T12:15:14.739 INFO:teuthology.orchestra.run.trial131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47761.log 2026-02-20T12:15:14.739 INFO:teuthology.orchestra.run.trial131.stderr: 66.9% -- replaced with /var/log/ceph/ceph-client.admin.40835.log.gz 2026-02-20T12:15:14.739 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.47995.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-20T12:15:14.739 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47995.log.gz 2026-02-20T12:15:14.739 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.47761.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41489.log 2026-02-20T12:15:14.739 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.47761.log.gz 2026-02-20T12:15:14.740 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46735.log 2026-02-20T12:15:14.740 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.41489.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46793.log 2026-02-20T12:15:14.740 INFO:teuthology.orchestra.run.trial131.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41489.log.gz 2026-02-20T12:15:14.740 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.46735.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40276.log 2026-02-20T12:15:14.740 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.46793.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.46735.log.gz 2026-02-20T12:15:14.740 INFO:teuthology.orchestra.run.trial131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.48657.log 2026-02-20T12:15:14.741 INFO:teuthology.orchestra.run.trial131.stderr: 50.8% -- replaced with /var/log/ceph/ceph-client.admin.46793.log.gz 2026-02-20T12:15:14.741 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.40276.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-20T12:15:14.741 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40276.log.gz 2026-02-20T12:15:14.741 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.48657.log: gzip 50.0% -- replaced with /var/log/ceph/ceph-client.admin.48657.log.gz 2026-02-20T12:15:14.741 INFO:teuthology.orchestra.run.trial131.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.46676.log 2026-02-20T12:15:14.741 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-mgr.x.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41391.log 2026-02-20T12:15:14.742 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.46676.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41149.log 2026-02-20T12:15:14.742 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.41391.log: 50.8% -- replaced with /var/log/ceph/ceph-client.admin.46676.log.gz 2026-02-20T12:15:14.742 INFO:teuthology.orchestra.run.trial131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41151.log 2026-02-20T12:15:14.742 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41391.log.gz 2026-02-20T12:15:14.742 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.41149.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-20T12:15:14.742 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41149.log.gz 2026-02-20T12:15:14.742 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.41151.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41048.log 2026-02-20T12:15:14.743 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41151.log.gz 2026-02-20T12:15:14.743 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40229.log 2026-02-20T12:15:14.743 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.41048.log: 86.8% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-20T12:15:14.743 INFO:teuthology.orchestra.run.trial131.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.47073.log 2026-02-20T12:15:14.743 INFO:teuthology.orchestra.run.trial131.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41048.log.gz 2026-02-20T12:15:14.743 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.40229.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40229.log.gz 2026-02-20T12:15:14.744 INFO:teuthology.orchestra.run.trial131.stderr:/var/log/ceph/ceph-client.admin.47073.log: 67.2% -- replaced with /var/log/ceph/ceph-client.admin.47073.log.gz 2026-02-20T12:15:14.752 INFO:teuthology.orchestra.run.trial131.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-20T12:15:16.265 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 has been restored 2026-02-20T12:15:23.348 INFO:teuthology.orchestra.run.trial131.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-20T12:15:23.348 INFO:teuthology.orchestra.run.trial131.stderr: 91.6% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-20T12:15:23.711 INFO:teuthology.orchestra.run.trial131.stderr: 91.4% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-20T12:15:23.713 INFO:teuthology.orchestra.run.trial131.stderr: 2026-02-20T12:15:23.713 INFO:teuthology.orchestra.run.trial131.stderr:real 0m8.992s 2026-02-20T12:15:23.713 INFO:teuthology.orchestra.run.trial131.stderr:user 0m24.745s 2026-02-20T12:15:23.713 INFO:teuthology.orchestra.run.trial131.stderr:sys 0m1.449s 2026-02-20T12:15:23.714 INFO:tasks.ceph:Archiving logs... 2026-02-20T12:15:23.715 DEBUG:teuthology.misc:Transferring archived files from trial131:/var/log/ceph to /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:29:19-crimson-rados-wip-perezjos-crimson-only-12-02-2026-PR66245-distro-debug-trial/60625/remote/trial131/log 2026-02-20T12:15:23.716 DEBUG:teuthology.orchestra.run.trial131:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-20T12:15:25.916 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-20T12:15:25.985 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-20T12:15:25.996 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-20T12:15:25.996 DEBUG:teuthology.orchestra.run.trial131:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-20T12:15:26.023 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-20T12:15:26.034 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-20T12:15:26.034 DEBUG:teuthology.orchestra.run.trial131:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-20T12:15:26.079 INFO:teuthology.orchestra.run.trial131.stderr:bash: line 1: ntpq: command not found 2026-02-20T12:15:26.418 INFO:teuthology.orchestra.run.trial131.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-20T12:15:26.418 INFO:teuthology.orchestra.run.trial131.stdout:=============================================================================== 2026-02-20T12:15:26.418 INFO:teuthology.orchestra.run.trial131.stdout:^+ 23.186.168.133 2 6 377 10 +6630us[+6639us] +/- 56ms 2026-02-20T12:15:26.418 INFO:teuthology.orchestra.run.trial131.stdout:^+ ntp.maxhost.io 2 6 377 70 +6128us[+6140us] +/- 56ms 2026-02-20T12:15:26.418 INFO:teuthology.orchestra.run.trial131.stdout:^* as393746.mci.trtnw.net 2 6 377 8 +4792us[+4800us] +/- 23ms 2026-02-20T12:15:26.418 INFO:teuthology.orchestra.run.trial131.stdout:^+ t2.time.gq1.yahoo.com 2 6 377 138 -30ms[ -30ms] +/- 68ms 2026-02-20T12:15:26.419 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-20T12:15:26.435 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-20T12:15:26.436 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-20T12:15:26.451 DEBUG:teuthology.orchestra.run.trial131:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-20T12:15:26.522 DEBUG:teuthology.orchestra.run.trial131:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-20T12:15:26.548 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T12:15:26.549 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-20T12:15:26.565 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-20T12:15:26.583 INFO:teuthology.task.internal:Duration was 701.799231 seconds 2026-02-20T12:15:26.583 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-20T12:15:26.599 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-20T12:15:26.599 DEBUG:teuthology.orchestra.run.trial131:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-20T12:15:26.637 INFO:teuthology.orchestra.run.trial131.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-20T12:15:27.013 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-20T12:15:27.013 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial131.front.sepia.ceph.com 2026-02-20T12:15:27.013 DEBUG:teuthology.orchestra.run.trial131:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-20T12:15:27.071 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-20T12:15:27.072 DEBUG:teuthology.orchestra.run.trial131:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-20T12:15:27.177 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-20T12:15:27.178 DEBUG:teuthology.orchestra.run.trial131:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-20T12:15:27.205 INFO:teuthology.orchestra.run.trial131.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T12:15:27.205 INFO:teuthology.orchestra.run.trial131.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T12:15:27.205 INFO:teuthology.orchestra.run.trial131.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-20T12:15:27.206 INFO:teuthology.orchestra.run.trial131.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-20T12:15:27.206 INFO:teuthology.orchestra.run.trial131.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-20T12:15:27.215 INFO:teuthology.orchestra.run.trial131.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.6% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-20T12:15:27.217 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-20T12:15:27.238 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-20T12:15:27.238 DEBUG:teuthology.orchestra.run.trial131:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-20T12:15:27.286 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-20T12:15:27.299 DEBUG:teuthology.orchestra.run.trial131:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-20T12:15:27.349 INFO:teuthology.orchestra.run.trial131.stdout:kernel.core_pattern = core 2026-02-20T12:15:27.360 DEBUG:teuthology.orchestra.run.trial131:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-20T12:15:27.415 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T12:15:27.416 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-20T12:15:27.428 INFO:teuthology.task.internal:Transferring archived files... 2026-02-20T12:15:27.428 DEBUG:teuthology.misc:Transferring archived files from trial131:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:29:19-crimson-rados-wip-perezjos-crimson-only-12-02-2026-PR66245-distro-debug-trial/60625/remote/trial131 2026-02-20T12:15:27.428 DEBUG:teuthology.orchestra.run.trial131:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-20T12:15:27.521 INFO:teuthology.task.internal:Removing archive directory... 2026-02-20T12:15:27.521 DEBUG:teuthology.orchestra.run.trial131:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-20T12:15:27.546 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-20T12:15:27.564 INFO:teuthology.task.internal:Not uploading archives. 2026-02-20T12:15:27.564 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-20T12:15:27.587 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-20T12:15:27.587 DEBUG:teuthology.orchestra.run.trial131:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-20T12:15:27.604 INFO:teuthology.orchestra.run.trial131.stdout: 917576 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 20 12:15 /home/ubuntu/cephtest 2026-02-20T12:15:27.605 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-20T12:15:27.617 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-20T12:15:27.644 INFO:teuthology.run:Summary data: description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm} perf_ignorelist settings/optimized workloads/fio_4K_rand_rw} duration: 701.7992308139801 flavor: debug owner: scheduled_jjperez@soko04.front.sepia.ceph.com success: true 2026-02-20T12:15:27.644 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-20T12:15:27.816 INFO:teuthology.run:pass