2026-02-20T11:16:24.215 INFO:root:teuthology version: 1.2.4.dev11+ge6d9099ce 2026-02-20T11:16:24.231 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-20T11:16:24.313 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60503 branch: wip-perezjos-crimson-only-19-02-2026-PR64439 description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} perf_ignorelist settings/optimized workloads/radosbench_4M_seq_read} email: null first_in_suite: false flavor: debug job_id: '60503' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-perezjos-crimson-only-19-02-2026-PR64439 ceph: conf: global: auth client required: none auth cluster required: none auth service required: none auth supported: none debug asok: 0/0 debug auth: 0/0 debug bdev: 0/0 debug bluefs: 0/0 debug bluestore: 0/0 debug buffer: 0/0 debug civetweb: 0/0 debug client: 0/0 debug compressor: 0/0 debug context: 0/0 debug crush: 0/0 debug crypto: 0/0 debug dpdk: 0/0 debug eventtrace: 0/0 debug filer: 0/0 debug filestore: 0/0 debug finisher: 0/0 debug fuse: 0/0 debug heartbeatmap: 0/0 debug javaclient: 0/0 debug journal: 0/0 debug journaler: 0/0 debug kstore: 0/0 debug leveldb: 0/0 debug lockdep: 0/0 debug mds: 0/0 debug mds balancer: 0/0 debug mds locker: 0/0 debug mds log: 0/0 debug mds log expire: 0/0 debug mds migrator: 0/0 debug memdb: 0/0 debug mgr: 0/0 debug mgrc: 0/0 debug mon: 0/0 debug monc: 0/0 debug ms: 0/0 debug objclass: 0/0 debug objectcacher: 0/0 debug objecter: 0/0 debug optracker: 0/0 debug osd: 0/0 debug paxos: 0/0 debug perfcounter: 0/0 debug rados: 0/0 debug rbd: 0/0 debug rbd mirror: 0/0 debug rbd replay: 0/0 debug refs: 0/0 debug rgw: 0/0 debug rgw lifecycle: 0/0 debug rgw notification: 0/0 debug rgw sync: 0/0 debug rocksdb: 0/0 debug striper: 0/0 debug throttle: 0/0 debug timer: 0/0 debug tp: 0/0 enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 0/0 debug ms: 0/0 debug paxos: 0/0 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug filestore: 0/0 debug journal: 0/0 debug ms: 0/0 debug osd: 0/0 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - \(PG_ - \(OSD_ - \(OBJECT_ - overall HEALTH sha1: 0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: flavor: debug sha1: 0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: 445d5c9ba18a4040e70e25303230c36c77773acb owner: scheduled_jjperez@soko04.front.sepia.ceph.com priority: 101 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 1736 sha1: 0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: 445d5c9ba18a4040e70e25303230c36c77773acb targets: trial038.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI9Tb0uNtlqm3jR0v/dEugkpUoH0vqC8/9JBwkiq7f0M1eYWLFMugxzKmPHpwuYg1K5iUFFf69EvmZ2s846qAYo= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: osd: debug monc: 20 wait-for-scrub: false - ssh_keys: null - cbt: benchmarks: radosbench: concurrent_ops: 4 concurrent_procs: 2 op_size: - 4194304 pool_monitoring_list: - collectl pool_profile: replicated run_monitoring_list: - collectl time: 60 write_only: false cluster: iterations: 1 osds_per_node: 3 pool_profiles: replicated: pg_size: 256 pgp_size: 256 replication: replicated user: ubuntu monitoring_profiles: perf: args: stat -p {pid} -o {perf_dir}/perf_stat.{pid} nodes: - osds perf_cmd: perf pid_glob: crimson-osd.*.pid teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: reimage-ssh-down teuthology_sha1: e6d9099ce03dc21193fda653159ec82465c3a94a timestamp: 2026-02-20_10:21:31 tube: trial user: jjperez verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699 2026-02-20T11:16:24.313 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa; will attempt to use it 2026-02-20T11:16:24.314 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa/tasks 2026-02-20T11:16:24.314 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-20T11:16:24.315 INFO:teuthology.task.internal:Checking packages... 2026-02-20T11:16:24.316 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4' 2026-02-20T11:16:24.316 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-20T11:16:24.316 INFO:teuthology.packaging:ref: None 2026-02-20T11:16:24.316 INFO:teuthology.packaging:tag: None 2026-02-20T11:16:24.316 INFO:teuthology.packaging:branch: wip-perezjos-crimson-only-19-02-2026-PR64439 2026-02-20T11:16:24.316 INFO:teuthology.packaging:sha1: 0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:16:24.316 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=wip-perezjos-crimson-only-19-02-2026-PR64439 2026-02-20T11:16:24.430 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5357.g0c3148ca 2026-02-20T11:16:24.431 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-20T11:16:24.442 INFO:teuthology.task.internal:no buildpackages task found 2026-02-20T11:16:24.442 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-20T11:16:24.456 INFO:teuthology.task.internal:Saving configuration 2026-02-20T11:16:24.470 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-20T11:16:24.482 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-20T11:16:24.556 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial038.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60503', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-20 11:12:58.830023', 'locked_by': 'scheduled_jjperez@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI9Tb0uNtlqm3jR0v/dEugkpUoH0vqC8/9JBwkiq7f0M1eYWLFMugxzKmPHpwuYg1K5iUFFf69EvmZ2s846qAYo='} 2026-02-20T11:16:24.556 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-20T11:16:24.568 INFO:teuthology.task.internal:roles: ubuntu@trial038.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-20T11:16:24.568 INFO:teuthology.run_tasks:Running task console_log... 2026-02-20T11:16:24.651 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x715a969332e0>, signals=[15]) 2026-02-20T11:16:24.651 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-20T11:16:24.662 INFO:teuthology.task.internal:Opening connections... 2026-02-20T11:16:24.662 DEBUG:teuthology.task.internal:connecting to ubuntu@trial038.front.sepia.ceph.com 2026-02-20T11:16:24.663 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T11:16:24.726 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-20T11:16:24.737 DEBUG:teuthology.orchestra.run.trial038:> uname -m 2026-02-20T11:16:24.801 INFO:teuthology.orchestra.run.trial038.stdout:x86_64 2026-02-20T11:16:24.801 DEBUG:teuthology.orchestra.run.trial038:> cat /etc/os-release 2026-02-20T11:16:24.856 INFO:teuthology.orchestra.run.trial038.stdout:NAME="CentOS Stream" 2026-02-20T11:16:24.857 INFO:teuthology.orchestra.run.trial038.stdout:VERSION="9" 2026-02-20T11:16:24.857 INFO:teuthology.orchestra.run.trial038.stdout:ID="centos" 2026-02-20T11:16:24.857 INFO:teuthology.orchestra.run.trial038.stdout:ID_LIKE="rhel fedora" 2026-02-20T11:16:24.857 INFO:teuthology.orchestra.run.trial038.stdout:VERSION_ID="9" 2026-02-20T11:16:24.857 INFO:teuthology.orchestra.run.trial038.stdout:PLATFORM_ID="platform:el9" 2026-02-20T11:16:24.857 INFO:teuthology.orchestra.run.trial038.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-20T11:16:24.857 INFO:teuthology.orchestra.run.trial038.stdout:ANSI_COLOR="0;31" 2026-02-20T11:16:24.857 INFO:teuthology.orchestra.run.trial038.stdout:LOGO="fedora-logo-icon" 2026-02-20T11:16:24.857 INFO:teuthology.orchestra.run.trial038.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-20T11:16:24.857 INFO:teuthology.orchestra.run.trial038.stdout:HOME_URL="https://centos.org/" 2026-02-20T11:16:24.857 INFO:teuthology.orchestra.run.trial038.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-20T11:16:24.857 INFO:teuthology.orchestra.run.trial038.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-20T11:16:24.857 INFO:teuthology.orchestra.run.trial038.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-20T11:16:24.857 INFO:teuthology.lock.ops:Updating trial038.front.sepia.ceph.com on lock server 2026-02-20T11:16:24.920 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-20T11:16:24.932 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-20T11:16:24.944 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-20T11:16:24.944 DEBUG:teuthology.orchestra.run.trial038:> test '!' -e /home/ubuntu/cephtest 2026-02-20T11:16:24.959 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-20T11:16:24.970 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-20T11:16:24.971 DEBUG:teuthology.orchestra.run.trial038:> test -z $(ls -A /var/lib/ceph) 2026-02-20T11:16:25.016 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-20T11:16:25.112 INFO:teuthology.run_tasks:Running task kernel... 2026-02-20T11:16:25.143 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-20T11:16:25.143 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-20T11:16:25.144 DEBUG:teuthology.orchestra.run.trial038:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-20T11:16:25.158 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T11:16:25.158 DEBUG:teuthology.orchestra.run.trial038:> uname -r 2026-02-20T11:16:25.214 INFO:teuthology.orchestra.run.trial038.stdout:5.14.0-681.el9.x86_64 2026-02-20T11:16:25.214 INFO:teuthology.task.kernel:Running kernel on trial038: 5.14.0-681.el9.x86_64 2026-02-20T11:16:25.214 DEBUG:teuthology.orchestra.run.trial038:> sudo yum install -y kernel 2026-02-20T11:16:34.856 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - BaseOS 984 kB/s | 8.9 MB 00:09 2026-02-20T11:16:38.079 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 27 MB 00:02 2026-02-20T11:16:43.552 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 7.8 MB 00:03 2026-02-20T11:16:45.219 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - Extras packages 21 kB/s | 20 kB 00:00 2026-02-20T11:16:48.866 INFO:teuthology.orchestra.run.trial038.stdout:Extra Packages for Enterprise Linux 5.7 MB/s | 20 MB 00:03 2026-02-20T11:16:52.028 INFO:teuthology.orchestra.run.trial038.stdout:lab-extras 2.3 MB/s | 50 kB 00:00 2026-02-20T11:16:53.033 INFO:teuthology.orchestra.run.trial038.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-20T11:16:53.033 INFO:teuthology.orchestra.run.trial038.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-20T11:16:53.052 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:16:53.053 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:16:53.053 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:16:53.077 DEBUG:teuthology.orchestra.run.trial038:> echo no | sudo yum reinstall kernel || true 2026-02-20T11:16:53.379 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:01 ago on Fri 20 Feb 2026 11:16:52 AM UTC. 2026-02-20T11:16:53.459 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:16:53.460 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:16:53.460 INFO:teuthology.orchestra.run.trial038.stdout: Package Architecture Version Repository Size 2026-02-20T11:16:53.460 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:16:53.460 INFO:teuthology.orchestra.run.trial038.stdout:Reinstalling: 2026-02-20T11:16:53.460 INFO:teuthology.orchestra.run.trial038.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T11:16:53.460 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:16:53.460 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-20T11:16:53.460 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:16:53.461 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:16:53.461 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 803 k 2026-02-20T11:16:53.461 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 0 2026-02-20T11:16:53.461 INFO:teuthology.orchestra.run.trial038.stderr:Operation aborted. 2026-02-20T11:16:53.480 INFO:teuthology.orchestra.run.trial038.stdout:Is this ok [y/N]: 2026-02-20T11:16:53.480 DEBUG:teuthology.orchestra.run.trial038:> sudo yum reinstall -y kernel || true 2026-02-20T11:16:53.782 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:01 ago on Fri 20 Feb 2026 11:16:52 AM UTC. 2026-02-20T11:16:53.863 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:16:53.863 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:16:53.863 INFO:teuthology.orchestra.run.trial038.stdout: Package Architecture Version Repository Size 2026-02-20T11:16:53.863 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:16:53.863 INFO:teuthology.orchestra.run.trial038.stdout:Reinstalling: 2026-02-20T11:16:53.863 INFO:teuthology.orchestra.run.trial038.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-20T11:16:53.863 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:16:53.864 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-20T11:16:53.864 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:16:53.864 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:16:53.864 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 803 k 2026-02-20T11:16:53.864 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 0 2026-02-20T11:16:53.864 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-20T11:16:55.490 INFO:teuthology.orchestra.run.trial038.stdout:kernel-5.14.0-681.el9.x86_64.rpm 524 kB/s | 803 kB 00:01 2026-02-20T11:16:55.490 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:16:55.490 INFO:teuthology.orchestra.run.trial038.stdout:Total 494 kB/s | 803 kB 00:01 2026-02-20T11:16:55.502 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-20T11:16:55.536 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-20T11:16:55.536 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-20T11:16:55.541 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-20T11:16:55.541 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-20T11:16:55.565 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-20T11:16:55.568 INFO:teuthology.orchestra.run.trial038.stdout: Reinstalling : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T11:16:55.569 INFO:teuthology.orchestra.run.trial038.stdout: Cleanup : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T11:16:55.676 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T11:16:55.676 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/2 2026-02-20T11:16:55.732 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 2/2 2026-02-20T11:16:55.733 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:16:55.733 INFO:teuthology.orchestra.run.trial038.stdout:Reinstalled: 2026-02-20T11:16:55.733 INFO:teuthology.orchestra.run.trial038.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-20T11:16:55.733 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:16:55.733 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:16:55.759 DEBUG:teuthology.orchestra.run.trial038:> rpm -q kernel | sort -rV | head -n 1 2026-02-20T11:16:55.825 INFO:teuthology.orchestra.run.trial038.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-20T11:16:55.826 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-20T11:16:55.827 INFO:teuthology.task.kernel:Newest distro kernel installed and running 2026-02-20T11:16:55.827 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-20T11:16:55.827 DEBUG:teuthology.orchestra.run.trial038:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-20T11:16:55.892 INFO:teuthology.orchestra.run.trial038.stdout:ttyS1 2026-02-20T11:16:55.917 DEBUG:teuthology.parallel:result is None 2026-02-20T11:16:55.918 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-20T11:16:55.924 INFO:teuthology.task.internal:Creating test directory... 2026-02-20T11:16:55.925 DEBUG:teuthology.orchestra.run.trial038:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-20T11:16:55.975 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-20T11:16:55.987 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-20T11:16:55.999 INFO:teuthology.task.internal:Creating archive directory... 2026-02-20T11:16:55.999 DEBUG:teuthology.orchestra.run.trial038:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-20T11:16:56.038 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-20T11:16:56.050 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-20T11:16:56.050 DEBUG:teuthology.orchestra.run.trial038:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-20T11:16:56.098 INFO:teuthology.orchestra.run.trial038.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T11:16:56.106 INFO:teuthology.orchestra.run.trial038.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-20T11:16:56.108 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-20T11:16:56.120 INFO:teuthology.task.internal:Configuring sudo... 2026-02-20T11:16:56.120 DEBUG:teuthology.orchestra.run.trial038:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-20T11:16:56.173 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-20T11:16:56.188 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-20T11:16:56.188 DEBUG:teuthology.orchestra.run.trial038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-20T11:16:56.228 DEBUG:teuthology.orchestra.run.trial038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T11:16:56.379 DEBUG:teuthology.orchestra.run.trial038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-20T11:16:56.404 DEBUG:teuthology.orchestra.run.trial038:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T11:16:56.535 DEBUG:teuthology.orchestra.run.trial038:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-20T11:16:56.559 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:16:56.559 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-20T11:16:56.624 DEBUG:teuthology.orchestra.run.trial038:> sudo service rsyslog restart 2026-02-20T11:16:56.691 INFO:teuthology.orchestra.run.trial038.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-20T11:16:56.987 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-20T11:16:57.000 INFO:teuthology.task.internal:Starting timer... 2026-02-20T11:16:57.000 INFO:teuthology.run_tasks:Running task pcp... 2026-02-20T11:16:57.020 INFO:teuthology.run_tasks:Running task selinux... 2026-02-20T11:16:57.041 DEBUG:teuthology.orchestra.run.trial038:> sudo service auditd rotate 2026-02-20T11:16:57.069 INFO:teuthology.orchestra.run.trial038.stdout:Rotating logs: 2026-02-20T11:16:57.071 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-20T11:16:57.071 DEBUG:teuthology.orchestra.run.trial038:> /usr/sbin/getenforce 2026-02-20T11:16:57.130 INFO:teuthology.orchestra.run.trial038.stdout:Permissive 2026-02-20T11:16:57.130 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial038.front.sepia.ceph.com': 'permissive'} 2026-02-20T11:16:57.131 DEBUG:teuthology.orchestra.run.trial038:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-20T11:16:57.193 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-20T11:16:57.194 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-20T11:16:57.194 DEBUG:teuthology.orchestra.run.trial038:> sudo /usr/sbin/setenforce permissive 2026-02-20T11:16:57.258 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-20T11:16:57.270 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-20T11:16:57.280 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-20T11:16:57.281 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial038.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-20T11:19:42.833 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial038.front.sepia.ceph.com')] 2026-02-20T11:19:42.835 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial038.front.sepia.ceph.com' 2026-02-20T11:19:42.837 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial038.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-20T11:19:42.945 DEBUG:teuthology.orchestra.run.trial038:> true 2026-02-20T11:19:43.021 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial038.front.sepia.ceph.com' 2026-02-20T11:19:43.021 INFO:teuthology.run_tasks:Running task clock... 2026-02-20T11:19:43.038 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-20T11:19:43.039 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-20T11:19:43.039 DEBUG:teuthology.orchestra.run.trial038:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-20T11:19:43.090 INFO:teuthology.orchestra.run.trial038.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-20T11:19:43.102 INFO:teuthology.orchestra.run.trial038.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-20T11:19:43.122 INFO:teuthology.orchestra.run.trial038.stderr:sudo: ntpd: command not found 2026-02-20T11:19:43.131 INFO:teuthology.orchestra.run.trial038.stdout:506 Cannot talk to daemon 2026-02-20T11:19:43.143 INFO:teuthology.orchestra.run.trial038.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-20T11:19:43.154 INFO:teuthology.orchestra.run.trial038.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-20T11:19:43.201 INFO:teuthology.orchestra.run.trial038.stderr:bash: line 1: ntpq: command not found 2026-02-20T11:19:43.554 INFO:teuthology.orchestra.run.trial038.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-20T11:19:43.554 INFO:teuthology.orchestra.run.trial038.stdout:=============================================================================== 2026-02-20T11:19:43.554 INFO:teuthology.orchestra.run.trial038.stdout:^? time.richie.net 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-20T11:19:43.554 INFO:teuthology.orchestra.run.trial038.stdout:^? 144.202.41.38.vultruserc> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-20T11:19:43.555 INFO:teuthology.orchestra.run.trial038.stdout:^? 144.202.66.214.vultruser> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-20T11:19:43.555 INFO:teuthology.orchestra.run.trial038.stdout:^? 72-14-186-59.ip.linodeus> 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-20T11:19:43.556 INFO:teuthology.run_tasks:Running task install... 2026-02-20T11:19:43.569 DEBUG:teuthology.task.install:project ceph 2026-02-20T11:19:43.569 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}} 2026-02-20T11:19:43.570 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'} 2026-02-20T11:19:43.570 INFO:teuthology.task.install:Using flavor: debug 2026-02-20T11:19:43.579 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-20T11:19:43.579 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-20T11:19:43.580 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4', 'tag': None, 'wait_for_package': False} 2026-02-20T11:19:43.581 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:19:43.740 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/ 2026-02-20T11:19:43.740 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5357.g0c3148ca 2026-02-20T11:19:44.098 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-20T11:19:44.099 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:19:44.099 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-20T11:19:44.125 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-20T11:19:44.126 DEBUG:teuthology.orchestra.run.trial038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-20T11:19:44.192 DEBUG:teuthology.orchestra.run.trial038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-20T11:19:44.265 DEBUG:teuthology.orchestra.run.trial038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-20T11:19:44.330 INFO:teuthology.orchestra.run.trial038.stdout:check_obsoletes = 1 2026-02-20T11:19:44.332 DEBUG:teuthology.orchestra.run.trial038:> sudo yum clean all 2026-02-20T11:19:44.493 INFO:teuthology.orchestra.run.trial038.stdout:41 files removed 2026-02-20T11:19:44.508 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-radosgw 2026-02-20T11:19:45.363 INFO:teuthology.orchestra.run.trial038.stdout:ceph packages for x86_64 131 kB/s | 94 kB 00:00 2026-02-20T11:19:45.932 INFO:teuthology.orchestra.run.trial038.stdout:ceph noarch packages 45 kB/s | 24 kB 00:00 2026-02-20T11:19:46.468 INFO:teuthology.orchestra.run.trial038.stdout:ceph source packages 4.0 kB/s | 2.1 kB 00:00 2026-02-20T11:19:48.178 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - BaseOS 5.3 MB/s | 8.9 MB 00:01 2026-02-20T11:19:50.281 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - AppStream 17 MB/s | 27 MB 00:01 2026-02-20T11:19:54.774 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - CRB 3.8 MB/s | 7.8 MB 00:02 2026-02-20T11:19:55.933 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - Extras packages 49 kB/s | 20 kB 00:00 2026-02-20T11:19:56.903 INFO:teuthology.orchestra.run.trial038.stdout:Extra Packages for Enterprise Linux 22 MB/s | 20 MB 00:00 2026-02-20T11:20:00.029 INFO:teuthology.orchestra.run.trial038.stdout:lab-extras 2.8 MB/s | 50 kB 00:00 2026-02-20T11:20:01.237 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:20:01.238 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:20:01.238 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repository Size 2026-02-20T11:20:01.238 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:20:01.238 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-20T11:20:01.238 INFO:teuthology.orchestra.run.trial038.stdout: ceph-radosgw x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 11 M 2026-02-20T11:20:01.238 INFO:teuthology.orchestra.run.trial038.stdout:Upgrading: 2026-02-20T11:20:01.239 INFO:teuthology.orchestra.run.trial038.stdout: librados2 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 3.3 M 2026-02-20T11:20:01.239 INFO:teuthology.orchestra.run.trial038.stdout: librbd1 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 2.7 M 2026-02-20T11:20:01.239 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-20T11:20:01.239 INFO:teuthology.orchestra.run.trial038.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-20T11:20:01.239 INFO:teuthology.orchestra.run.trial038.stdout: ceph-base x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 5.1 M 2026-02-20T11:20:01.239 INFO:teuthology.orchestra.run.trial038.stdout: ceph-common x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 21 M 2026-02-20T11:20:01.239 INFO:teuthology.orchestra.run.trial038.stdout: ceph-selinux x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 25 k 2026-02-20T11:20:01.239 INFO:teuthology.orchestra.run.trial038.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-20T11:20:01.239 INFO:teuthology.orchestra.run.trial038.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-20T11:20:01.239 INFO:teuthology.orchestra.run.trial038.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-20T11:20:01.239 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs2 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 860 k 2026-02-20T11:20:01.239 INFO:teuthology.orchestra.run.trial038.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-20T11:20:01.240 INFO:teuthology.orchestra.run.trial038.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-20T11:20:01.240 INFO:teuthology.orchestra.run.trial038.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-20T11:20:01.240 INFO:teuthology.orchestra.run.trial038.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-20T11:20:01.240 INFO:teuthology.orchestra.run.trial038.stdout: libradosstriper1 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 263 k 2026-02-20T11:20:01.240 INFO:teuthology.orchestra.run.trial038.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-20T11:20:01.240 INFO:teuthology.orchestra.run.trial038.stdout: librgw2 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 5.6 M 2026-02-20T11:20:01.240 INFO:teuthology.orchestra.run.trial038.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-20T11:20:01.240 INFO:teuthology.orchestra.run.trial038.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-20T11:20:01.240 INFO:teuthology.orchestra.run.trial038.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-20T11:20:01.240 INFO:teuthology.orchestra.run.trial038.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-20T11:20:01.240 INFO:teuthology.orchestra.run.trial038.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-20T11:20:01.240 INFO:teuthology.orchestra.run.trial038.stdout: python3-ceph-argparse x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 45 k 2026-02-20T11:20:01.241 INFO:teuthology.orchestra.run.trial038.stdout: python3-ceph-common x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 185 k 2026-02-20T11:20:01.241 INFO:teuthology.orchestra.run.trial038.stdout: python3-cephfs x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 204 k 2026-02-20T11:20:01.241 INFO:teuthology.orchestra.run.trial038.stdout: python3-rados x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 324 k 2026-02-20T11:20:01.241 INFO:teuthology.orchestra.run.trial038.stdout: python3-rbd x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 305 k 2026-02-20T11:20:01.241 INFO:teuthology.orchestra.run.trial038.stdout: python3-rgw x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 100 k 2026-02-20T11:20:01.241 INFO:teuthology.orchestra.run.trial038.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-20T11:20:01.241 INFO:teuthology.orchestra.run.trial038.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-20T11:20:01.241 INFO:teuthology.orchestra.run.trial038.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-20T11:20:01.241 INFO:teuthology.orchestra.run.trial038.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-20T11:20:01.241 INFO:teuthology.orchestra.run.trial038.stdout:Installing weak dependencies: 2026-02-20T11:20:01.241 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs-daemon x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 37 k 2026-02-20T11:20:01.241 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 26 k 2026-02-20T11:20:01.241 INFO:teuthology.orchestra.run.trial038.stdout: nvme-cli x86_64 2.16-1.el9 baseos 1.2 M 2026-02-20T11:20:01.242 INFO:teuthology.orchestra.run.trial038.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-20T11:20:01.242 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:20:01.242 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-20T11:20:01.242 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:20:01.242 INFO:teuthology.orchestra.run.trial038.stdout:Install 35 Packages 2026-02-20T11:20:01.242 INFO:teuthology.orchestra.run.trial038.stdout:Upgrade 2 Packages 2026-02-20T11:20:01.242 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:20:01.243 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 61 M 2026-02-20T11:20:01.243 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-20T11:20:03.493 INFO:teuthology.orchestra.run.trial038.stdout:(1/37): ceph-base-20.3.0-5357.g0c3148ca.el9.x86 3.5 MB/s | 5.1 MB 00:01 2026-02-20T11:20:03.563 INFO:teuthology.orchestra.run.trial038.stdout:(2/37): ceph-selinux-20.3.0-5357.g0c3148ca.el9. 357 kB/s | 25 kB 00:00 2026-02-20T11:20:03.635 INFO:teuthology.orchestra.run.trial038.stdout:(3/37): libcephfs-daemon-20.3.0-5357.g0c3148ca. 518 kB/s | 37 kB 00:00 2026-02-20T11:20:03.705 INFO:teuthology.orchestra.run.trial038.stdout:(4/37): libcephfs-proxy2-20.3.0-5357.g0c3148ca. 361 kB/s | 26 kB 00:00 2026-02-20T11:20:04.107 INFO:teuthology.orchestra.run.trial038.stdout:(5/37): libcephfs2-20.3.0-5357.g0c3148ca.el9.x8 2.1 MB/s | 860 kB 00:00 2026-02-20T11:20:04.228 INFO:teuthology.orchestra.run.trial038.stdout:(6/37): libradosstriper1-20.3.0-5357.g0c3148ca. 2.1 MB/s | 263 kB 00:00 2026-02-20T11:20:05.662 INFO:teuthology.orchestra.run.trial038.stdout:(7/37): librgw2-20.3.0-5357.g0c3148ca.el9.x86_6 3.9 MB/s | 5.6 MB 00:01 2026-02-20T11:20:05.718 INFO:teuthology.orchestra.run.trial038.stdout:(8/37): python3-ceph-argparse-20.3.0-5357.g0c31 797 kB/s | 45 kB 00:00 2026-02-20T11:20:05.779 INFO:teuthology.orchestra.run.trial038.stdout:(9/37): python3-ceph-common-20.3.0-5357.g0c3148 3.0 MB/s | 185 kB 00:00 2026-02-20T11:20:05.830 INFO:teuthology.orchestra.run.trial038.stdout:(10/37): python3-cephfs-20.3.0-5357.g0c3148ca.e 3.9 MB/s | 204 kB 00:00 2026-02-20T11:20:05.930 INFO:teuthology.orchestra.run.trial038.stdout:(11/37): python3-rados-20.3.0-5357.g0c3148ca.el 3.2 MB/s | 324 kB 00:00 2026-02-20T11:20:06.028 INFO:teuthology.orchestra.run.trial038.stdout:(12/37): python3-rbd-20.3.0-5357.g0c3148ca.el9. 3.0 MB/s | 305 kB 00:00 2026-02-20T11:20:06.078 INFO:teuthology.orchestra.run.trial038.stdout:(13/37): python3-rgw-20.3.0-5357.g0c3148ca.el9. 2.0 MB/s | 100 kB 00:00 2026-02-20T11:20:06.247 INFO:teuthology.orchestra.run.trial038.stdout:(14/37): mailcap-2.1.49-5.el9.noarch.rpm 196 kB/s | 33 kB 00:00 2026-02-20T11:20:06.398 INFO:teuthology.orchestra.run.trial038.stdout:(15/37): nvme-cli-2.16-1.el9.x86_64.rpm 7.7 MB/s | 1.2 MB 00:00 2026-02-20T11:20:06.435 INFO:teuthology.orchestra.run.trial038.stdout:(16/37): pciutils-3.7.0-7.el9.x86_64.rpm 2.5 MB/s | 93 kB 00:00 2026-02-20T11:20:06.616 INFO:teuthology.orchestra.run.trial038.stdout:(17/37): boost-program-options-1.75.0-13.el9.x8 575 kB/s | 104 kB 00:00 2026-02-20T11:20:06.661 INFO:teuthology.orchestra.run.trial038.stdout:(18/37): libnbd-1.20.3-4.el9.x86_64.rpm 3.6 MB/s | 164 kB 00:00 2026-02-20T11:20:06.686 INFO:teuthology.orchestra.run.trial038.stdout:(19/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 6.3 MB/s | 160 kB 00:00 2026-02-20T11:20:06.749 INFO:teuthology.orchestra.run.trial038.stdout:(20/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 718 kB/s | 45 kB 00:00 2026-02-20T11:20:06.794 INFO:teuthology.orchestra.run.trial038.stdout:(21/37): librdkafka-1.6.1-102.el9.x86_64.rpm 14 MB/s | 662 kB 00:00 2026-02-20T11:20:06.819 INFO:teuthology.orchestra.run.trial038.stdout:(22/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 12 MB/s | 292 kB 00:00 2026-02-20T11:20:06.843 INFO:teuthology.orchestra.run.trial038.stdout:(23/37): qatlib-25.08.0-2.el9.x86_64.rpm 9.7 MB/s | 240 kB 00:00 2026-02-20T11:20:06.865 INFO:teuthology.orchestra.run.trial038.stdout:(24/37): qatlib-service-25.08.0-2.el9.x86_64.rp 1.6 MB/s | 37 kB 00:00 2026-02-20T11:20:06.887 INFO:teuthology.orchestra.run.trial038.stdout:(25/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 2.9 MB/s | 66 kB 00:00 2026-02-20T11:20:07.193 INFO:teuthology.orchestra.run.trial038.stdout:(26/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.0 MB/s | 308 kB 00:00 2026-02-20T11:20:07.354 INFO:teuthology.orchestra.run.trial038.stdout:(27/37): libarrow-9.0.0-15.el9.x86_64.rpm 27 MB/s | 4.4 MB 00:00 2026-02-20T11:20:07.384 INFO:teuthology.orchestra.run.trial038.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 822 kB/s | 25 kB 00:00 2026-02-20T11:20:07.415 INFO:teuthology.orchestra.run.trial038.stdout:(29/37): liboath-2.6.12-1.el9.x86_64.rpm 1.6 MB/s | 49 kB 00:00 2026-02-20T11:20:07.445 INFO:teuthology.orchestra.run.trial038.stdout:(30/37): libunwind-1.6.2-1.el9.x86_64.rpm 2.2 MB/s | 67 kB 00:00 2026-02-20T11:20:07.480 INFO:teuthology.orchestra.run.trial038.stdout:(31/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 24 MB/s | 838 kB 00:00 2026-02-20T11:20:07.511 INFO:teuthology.orchestra.run.trial038.stdout:(32/37): re2-20211101-20.el9.x86_64.rpm 6.1 MB/s | 191 kB 00:00 2026-02-20T11:20:07.549 INFO:teuthology.orchestra.run.trial038.stdout:(33/37): thrift-0.15.0-4.el9.x86_64.rpm 41 MB/s | 1.6 MB 00:00 2026-02-20T11:20:08.552 INFO:teuthology.orchestra.run.trial038.stdout:(34/37): librados2-20.3.0-5357.g0c3148ca.el9.x8 3.3 MB/s | 3.3 MB 00:01 2026-02-20T11:20:08.990 INFO:teuthology.orchestra.run.trial038.stdout:(35/37): ceph-common-20.3.0-5357.g0c3148ca.el9. 3.0 MB/s | 21 MB 00:06 2026-02-20T11:20:09.306 INFO:teuthology.orchestra.run.trial038.stdout:(36/37): librbd1-20.3.0-5357.g0c3148ca.el9.x86_ 3.6 MB/s | 2.7 MB 00:00 2026-02-20T11:20:12.039 INFO:teuthology.orchestra.run.trial038.stdout:(37/37): ceph-radosgw-20.3.0-5357.g0c3148ca.el9 1.1 MB/s | 11 MB 00:09 2026-02-20T11:20:12.042 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:20:12.042 INFO:teuthology.orchestra.run.trial038.stdout:Total 5.7 MB/s | 61 MB 00:10 2026-02-20T11:20:12.137 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-20T11:20:12.157 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-20T11:20:12.157 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-20T11:20:12.410 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-20T11:20:12.410 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-20T11:20:12.701 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-20T11:20:12.842 INFO:teuthology.orchestra.run.trial038.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-20T11:20:12.848 INFO:teuthology.orchestra.run.trial038.stdout: Upgrading : librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/39 2026-02-20T11:20:12.895 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/39 2026-02-20T11:20:12.896 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 3/39 2026-02-20T11:20:12.907 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 3/39 2026-02-20T11:20:12.909 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libcephfs-daemon-2:20.3.0-5357.g0c3148ca.el9.x86_6 4/39 2026-02-20T11:20:12.910 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libcephfs-proxy2-2:20.3.0-5357.g0c3148ca.el9.x86_6 5/39 2026-02-20T11:20:12.938 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5357.g0c3148ca.el9.x86_6 5/39 2026-02-20T11:20:12.940 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 6/39 2026-02-20T11:20:12.948 INFO:teuthology.orchestra.run.trial038.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-20T11:20:12.950 INFO:teuthology.orchestra.run.trial038.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-20T11:20:12.953 INFO:teuthology.orchestra.run.trial038.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-20T11:20:12.957 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-20T11:20:12.969 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-ceph-argparse-2:20.3.0-5357.g0c3148ca.el9. 11/39 2026-02-20T11:20:12.984 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 12/39 2026-02-20T11:20:12.984 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libradosstriper1-2:20.3.0-5357.g0c3148ca.el9.x86_6 13/39 2026-02-20T11:20:13.006 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5357.g0c3148ca.el9.x86_6 13/39 2026-02-20T11:20:13.010 INFO:teuthology.orchestra.run.trial038.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-20T11:20:13.040 INFO:teuthology.orchestra.run.trial038.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-20T11:20:13.043 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-20T11:20:13.050 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-20T11:20:13.052 INFO:teuthology.orchestra.run.trial038.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-20T11:20:13.057 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-20T11:20:13.174 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-20T11:20:13.175 INFO:teuthology.orchestra.run.trial038.stdout: Upgrading : librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 21/39 2026-02-20T11:20:13.200 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 21/39 2026-02-20T11:20:13.203 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 22/39 2026-02-20T11:20:13.209 INFO:teuthology.orchestra.run.trial038.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-20T11:20:13.457 INFO:teuthology.orchestra.run.trial038.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-20T11:20:13.459 INFO:teuthology.orchestra.run.trial038.stdout: Installing : librgw2-2:20.3.0-5357.g0c3148ca.el9.x86_64 25/39 2026-02-20T11:20:13.475 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librgw2-2:20.3.0-5357.g0c3148ca.el9.x86_64 25/39 2026-02-20T11:20:13.479 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 26/39 2026-02-20T11:20:13.483 INFO:teuthology.orchestra.run.trial038.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-20T11:20:13.486 INFO:teuthology.orchestra.run.trial038.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-20T11:20:13.487 INFO:teuthology.orchestra.run.trial038.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-20T11:20:13.505 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-20T11:20:13.623 INFO:teuthology.orchestra.run.trial038.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-20T11:20:13.626 INFO:teuthology.orchestra.run.trial038.stdout: Installing : nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-20T11:20:13.917 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: nvme-cli-2.16-1.el9.x86_64 31/39 2026-02-20T11:20:13.917 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-20T11:20:13.918 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:20:13.961 INFO:teuthology.orchestra.run.trial038.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-20T11:20:13.963 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-ceph-common-2:20.3.0-5357.g0c3148ca.el9.x8 33/39 2026-02-20T11:20:14.905 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 34/39 2026-02-20T11:20:14.907 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 34/39 2026-02-20T11:20:15.167 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 34/39 2026-02-20T11:20:15.171 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 35/39 2026-02-20T11:20:15.211 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 35/39 2026-02-20T11:20:15.211 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-20T11:20:15.212 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-20T11:20:15.212 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:20:15.215 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-selinux-2:20.3.0-5357.g0c3148ca.el9.x86_64 36/39 2026-02-20T11:20:19.784 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5357.g0c3148ca.el9.x86_64 36/39 2026-02-20T11:20:19.784 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /sys 2026-02-20T11:20:19.784 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /proc 2026-02-20T11:20:19.784 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /mnt 2026-02-20T11:20:19.784 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /var/tmp 2026-02-20T11:20:19.785 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /home 2026-02-20T11:20:19.785 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /root 2026-02-20T11:20:19.785 INFO:teuthology.orchestra.run.trial038.stdout:skipping the directory /tmp 2026-02-20T11:20:19.785 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:20:20.282 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 37/39 2026-02-20T11:20:20.306 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 37/39 2026-02-20T11:20:20.306 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:20:20.306 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-20T11:20:20.307 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T11:20:20.307 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-20T11:20:20.307 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:20:20.308 INFO:teuthology.orchestra.run.trial038.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-20T11:20:20.321 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-20T11:20:20.321 INFO:teuthology.orchestra.run.trial038.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-20T11:20:20.378 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-20T11:20:20.378 INFO:teuthology.orchestra.run.trial038.stdout:Creating group 'qat' with GID 992. 2026-02-20T11:20:20.378 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:20:20.680 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/39 2026-02-20T11:20:20.681 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/39 2026-02-20T11:20:20.681 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 3/39 2026-02-20T11:20:20.681 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-selinux-2:20.3.0-5357.g0c3148ca.el9.x86_64 4/39 2026-02-20T11:20:20.681 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libcephfs-daemon-2:20.3.0-5357.g0c3148ca.el9.x86_6 5/39 2026-02-20T11:20:20.681 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5357.g0c3148ca.el9.x86_6 6/39 2026-02-20T11:20:20.681 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 7/39 2026-02-20T11:20:20.681 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libradosstriper1-2:20.3.0-5357.g0c3148ca.el9.x86_6 8/39 2026-02-20T11:20:20.681 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librgw2-2:20.3.0-5357.g0c3148ca.el9.x86_64 9/39 2026-02-20T11:20:20.682 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5357.g0c3148ca.el9. 10/39 2026-02-20T11:20:20.682 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-ceph-common-2:20.3.0-5357.g0c3148ca.el9.x8 11/39 2026-02-20T11:20:20.682 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 12/39 2026-02-20T11:20:20.682 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 13/39 2026-02-20T11:20:20.682 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 14/39 2026-02-20T11:20:20.682 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 15/39 2026-02-20T11:20:20.682 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-20T11:20:20.682 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : nvme-cli-2.16-1.el9.x86_64 17/39 2026-02-20T11:20:20.682 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-20T11:20:20.682 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-20T11:20:20.682 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-20T11:20:20.683 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-20T11:20:20.683 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-20T11:20:20.683 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-20T11:20:20.683 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-20T11:20:20.683 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-20T11:20:20.683 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-20T11:20:20.683 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-20T11:20:20.683 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-20T11:20:20.683 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-20T11:20:20.683 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-20T11:20:20.683 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-20T11:20:20.683 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-20T11:20:20.684 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-20T11:20:20.684 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-20T11:20:20.684 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-20T11:20:20.684 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 36/39 2026-02-20T11:20:20.684 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-20T11:20:20.684 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 38/39 2026-02-20T11:20:20.743 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-20T11:20:20.743 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:20:20.743 INFO:teuthology.orchestra.run.trial038.stdout:Upgraded: 2026-02-20T11:20:20.744 INFO:teuthology.orchestra.run.trial038.stdout: librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.744 INFO:teuthology.orchestra.run.trial038.stdout: librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.744 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-20T11:20:20.744 INFO:teuthology.orchestra.run.trial038.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-20T11:20:20.744 INFO:teuthology.orchestra.run.trial038.stdout: ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.744 INFO:teuthology.orchestra.run.trial038.stdout: ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.744 INFO:teuthology.orchestra.run.trial038.stdout: ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.744 INFO:teuthology.orchestra.run.trial038.stdout: ceph-selinux-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.744 INFO:teuthology.orchestra.run.trial038.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-20T11:20:20.744 INFO:teuthology.orchestra.run.trial038.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-20T11:20:20.744 INFO:teuthology.orchestra.run.trial038.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-20T11:20:20.744 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs-daemon-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.745 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs-proxy2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.745 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.745 INFO:teuthology.orchestra.run.trial038.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-20T11:20:20.745 INFO:teuthology.orchestra.run.trial038.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-20T11:20:20.745 INFO:teuthology.orchestra.run.trial038.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-20T11:20:20.745 INFO:teuthology.orchestra.run.trial038.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-20T11:20:20.745 INFO:teuthology.orchestra.run.trial038.stdout: libradosstriper1-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.745 INFO:teuthology.orchestra.run.trial038.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-20T11:20:20.745 INFO:teuthology.orchestra.run.trial038.stdout: librgw2-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.745 INFO:teuthology.orchestra.run.trial038.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-20T11:20:20.745 INFO:teuthology.orchestra.run.trial038.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-20T11:20:20.745 INFO:teuthology.orchestra.run.trial038.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-20T11:20:20.746 INFO:teuthology.orchestra.run.trial038.stdout: nvme-cli-2.16-1.el9.x86_64 2026-02-20T11:20:20.746 INFO:teuthology.orchestra.run.trial038.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-20T11:20:20.746 INFO:teuthology.orchestra.run.trial038.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-20T11:20:20.746 INFO:teuthology.orchestra.run.trial038.stdout: python3-ceph-argparse-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.746 INFO:teuthology.orchestra.run.trial038.stdout: python3-ceph-common-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.746 INFO:teuthology.orchestra.run.trial038.stdout: python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.746 INFO:teuthology.orchestra.run.trial038.stdout: python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.746 INFO:teuthology.orchestra.run.trial038.stdout: python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.746 INFO:teuthology.orchestra.run.trial038.stdout: python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:20:20.746 INFO:teuthology.orchestra.run.trial038.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-20T11:20:20.746 INFO:teuthology.orchestra.run.trial038.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-20T11:20:20.747 INFO:teuthology.orchestra.run.trial038.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-20T11:20:20.747 INFO:teuthology.orchestra.run.trial038.stdout: re2-1:20211101-20.el9.x86_64 2026-02-20T11:20:20.747 INFO:teuthology.orchestra.run.trial038.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-20T11:20:20.747 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:20:20.747 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:20:20.784 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-test 2026-02-20T11:20:21.094 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:21 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:20:21.178 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:20:21.179 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:20:21.179 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repository Size 2026-02-20T11:20:21.179 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:20:21.179 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-20T11:20:21.179 INFO:teuthology.orchestra.run.trial038.stdout: ceph-test x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 127 M 2026-02-20T11:20:21.179 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-20T11:20:21.179 INFO:teuthology.orchestra.run.trial038.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-20T11:20:21.180 INFO:teuthology.orchestra.run.trial038.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-20T11:20:21.180 INFO:teuthology.orchestra.run.trial038.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-20T11:20:21.180 INFO:teuthology.orchestra.run.trial038.stdout: libcephsqlite x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 153 k 2026-02-20T11:20:21.180 INFO:teuthology.orchestra.run.trial038.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-20T11:20:21.180 INFO:teuthology.orchestra.run.trial038.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-20T11:20:21.180 INFO:teuthology.orchestra.run.trial038.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-20T11:20:21.180 INFO:teuthology.orchestra.run.trial038.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-20T11:20:21.180 INFO:teuthology.orchestra.run.trial038.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-20T11:20:21.180 INFO:teuthology.orchestra.run.trial038.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-20T11:20:21.180 INFO:teuthology.orchestra.run.trial038.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-20T11:20:21.180 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:20:21.181 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-20T11:20:21.181 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:20:21.181 INFO:teuthology.orchestra.run.trial038.stdout:Install 12 Packages 2026-02-20T11:20:21.181 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:20:21.181 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 132 M 2026-02-20T11:20:21.181 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 989 M 2026-02-20T11:20:21.181 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-20T11:20:21.912 INFO:teuthology.orchestra.run.trial038.stdout:(1/12): libcephsqlite-20.3.0-5357.g0c3148ca.el9 320 kB/s | 153 kB 00:00 2026-02-20T11:20:22.438 INFO:teuthology.orchestra.run.trial038.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 43 kB/s | 23 kB 00:00 2026-02-20T11:20:22.570 INFO:teuthology.orchestra.run.trial038.stdout:(3/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 1.8 MB/s | 2.1 MB 00:01 2026-02-20T11:20:22.740 INFO:teuthology.orchestra.run.trial038.stdout:(4/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 311 kB/s | 94 kB 00:00 2026-02-20T11:20:22.750 INFO:teuthology.orchestra.run.trial038.stdout:(5/12): libasan8-15.2.1-7.el9.x86_64.rpm 2.8 MB/s | 525 kB 00:00 2026-02-20T11:20:22.855 INFO:teuthology.orchestra.run.trial038.stdout:(6/12): libxslt-1.1.34-12.el9.x86_64.rpm 2.2 MB/s | 233 kB 00:00 2026-02-20T11:20:22.923 INFO:teuthology.orchestra.run.trial038.stdout:(7/12): protobuf-3.14.0-17.el9.x86_64.rpm 14 MB/s | 1.0 MB 00:00 2026-02-20T11:20:22.956 INFO:teuthology.orchestra.run.trial038.stdout:(8/12): libubsan-11.5.0-14.el9.x86_64.rpm 818 kB/s | 176 kB 00:00 2026-02-20T11:20:22.991 INFO:teuthology.orchestra.run.trial038.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 1.7 MB/s | 64 kB 00:00 2026-02-20T11:20:22.995 INFO:teuthology.orchestra.run.trial038.stdout:(10/12): socat-1.7.4.1-8.el9.x86_64.rpm 4.1 MB/s | 303 kB 00:00 2026-02-20T11:20:23.227 INFO:teuthology.orchestra.run.trial038.stdout:(11/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 517 kB/s | 122 kB 00:00 2026-02-20T11:20:52.301 INFO:teuthology.orchestra.run.trial038.stdout:(12/12): ceph-test-20.3.0-5357.g0c3148ca.el9.x8 4.1 MB/s | 127 MB 00:30 2026-02-20T11:20:52.303 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:20:52.303 INFO:teuthology.orchestra.run.trial038.stdout:Total 4.2 MB/s | 132 MB 00:31 2026-02-20T11:20:52.381 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-20T11:20:52.392 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-20T11:20:52.392 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-20T11:20:52.822 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-20T11:20:52.822 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-20T11:20:53.278 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-20T11:20:53.287 INFO:teuthology.orchestra.run.trial038.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-20T11:20:53.296 INFO:teuthology.orchestra.run.trial038.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-20T11:20:53.300 INFO:teuthology.orchestra.run.trial038.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-20T11:20:53.305 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-20T11:20:53.308 INFO:teuthology.orchestra.run.trial038.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-20T11:20:53.313 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-20T11:20:53.314 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-20T11:20:53.331 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-20T11:20:53.334 INFO:teuthology.orchestra.run.trial038.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-20T11:20:53.345 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-20T11:20:53.354 INFO:teuthology.orchestra.run.trial038.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-20T11:20:53.354 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libcephsqlite-2:20.3.0-5357.g0c3148ca.el9.x86_64 11/12 2026-02-20T11:21:00.284 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5357.g0c3148ca.el9.x86_64 11/12 2026-02-20T11:21:00.303 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 12/12 2026-02-20T11:21:01.121 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 12/12 2026-02-20T11:21:01.121 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/12 2026-02-20T11:21:01.122 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libcephsqlite-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/12 2026-02-20T11:21:01.122 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-20T11:21:01.122 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-20T11:21:01.122 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-20T11:21:01.122 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-20T11:21:01.122 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-20T11:21:01.122 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-20T11:21:01.122 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-20T11:21:01.122 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-20T11:21:01.122 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-20T11:21:01.176 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-20T11:21:01.176 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:01.176 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-20T11:21:01.176 INFO:teuthology.orchestra.run.trial038.stdout: ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:21:01.176 INFO:teuthology.orchestra.run.trial038.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-20T11:21:01.177 INFO:teuthology.orchestra.run.trial038.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-20T11:21:01.177 INFO:teuthology.orchestra.run.trial038.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-20T11:21:01.177 INFO:teuthology.orchestra.run.trial038.stdout: libcephsqlite-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:21:01.177 INFO:teuthology.orchestra.run.trial038.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-20T11:21:01.177 INFO:teuthology.orchestra.run.trial038.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-20T11:21:01.177 INFO:teuthology.orchestra.run.trial038.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-20T11:21:01.177 INFO:teuthology.orchestra.run.trial038.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-20T11:21:01.177 INFO:teuthology.orchestra.run.trial038.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-20T11:21:01.177 INFO:teuthology.orchestra.run.trial038.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-20T11:21:01.177 INFO:teuthology.orchestra.run.trial038.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-20T11:21:01.177 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:01.177 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:21:01.220 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph 2026-02-20T11:21:01.522 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:01 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:21:01.608 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:21:01.611 INFO:teuthology.orchestra.run.trial038.stdout:======================================================================================= 2026-02-20T11:21:01.611 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repository Size 2026-02-20T11:21:01.611 INFO:teuthology.orchestra.run.trial038.stdout:======================================================================================= 2026-02-20T11:21:01.611 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-20T11:21:01.611 INFO:teuthology.orchestra.run.trial038.stdout: ceph x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 6.5 k 2026-02-20T11:21:01.611 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-20T11:21:01.611 INFO:teuthology.orchestra.run.trial038.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-20T11:21:01.612 INFO:teuthology.orchestra.run.trial038.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 57 k 2026-02-20T11:21:01.612 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mds x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 2.0 M 2026-02-20T11:21:01.612 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 937 k 2026-02-20T11:21:01.612 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 297 k 2026-02-20T11:21:01.612 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mon x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 4.9 M 2026-02-20T11:21:01.612 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 163 k 2026-02-20T11:21:01.612 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-classic x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 15 M 2026-02-20T11:21:01.612 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-crimson x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 88 M 2026-02-20T11:21:01.612 INFO:teuthology.orchestra.run.trial038.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 17 k 2026-02-20T11:21:01.612 INFO:teuthology.orchestra.run.trial038.stdout: cephadm noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 1.0 M 2026-02-20T11:21:01.612 INFO:teuthology.orchestra.run.trial038.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-20T11:21:01.613 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-20T11:21:01.613 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-20T11:21:01.613 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-20T11:21:01.613 INFO:teuthology.orchestra.run.trial038.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-20T11:21:01.613 INFO:teuthology.orchestra.run.trial038.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-20T11:21:01.613 INFO:teuthology.orchestra.run.trial038.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-20T11:21:01.613 INFO:teuthology.orchestra.run.trial038.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-20T11:21:01.613 INFO:teuthology.orchestra.run.trial038.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-20T11:21:01.613 INFO:teuthology.orchestra.run.trial038.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-20T11:21:01.613 INFO:teuthology.orchestra.run.trial038.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-20T11:21:01.613 INFO:teuthology.orchestra.run.trial038.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-20T11:21:01.613 INFO:teuthology.orchestra.run.trial038.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-20T11:21:01.614 INFO:teuthology.orchestra.run.trial038.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-20T11:21:01.614 INFO:teuthology.orchestra.run.trial038.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-20T11:21:01.614 INFO:teuthology.orchestra.run.trial038.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-20T11:21:01.614 INFO:teuthology.orchestra.run.trial038.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-20T11:21:01.614 INFO:teuthology.orchestra.run.trial038.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-20T11:21:01.614 INFO:teuthology.orchestra.run.trial038.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-20T11:21:01.614 INFO:teuthology.orchestra.run.trial038.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-20T11:21:01.614 INFO:teuthology.orchestra.run.trial038.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-20T11:21:01.614 INFO:teuthology.orchestra.run.trial038.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-20T11:21:01.614 INFO:teuthology.orchestra.run.trial038.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-20T11:21:01.614 INFO:teuthology.orchestra.run.trial038.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-20T11:21:01.614 INFO:teuthology.orchestra.run.trial038.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-20T11:21:01.615 INFO:teuthology.orchestra.run.trial038.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-20T11:21:01.615 INFO:teuthology.orchestra.run.trial038.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-20T11:21:01.615 INFO:teuthology.orchestra.run.trial038.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-20T11:21:01.615 INFO:teuthology.orchestra.run.trial038.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-20T11:21:01.615 INFO:teuthology.orchestra.run.trial038.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-20T11:21:01.615 INFO:teuthology.orchestra.run.trial038.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-20T11:21:01.615 INFO:teuthology.orchestra.run.trial038.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-20T11:21:01.615 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-20T11:21:01.615 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-20T11:21:01.615 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-20T11:21:01.615 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-20T11:21:01.615 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-20T11:21:01.616 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-20T11:21:01.616 INFO:teuthology.orchestra.run.trial038.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-20T11:21:01.616 INFO:teuthology.orchestra.run.trial038.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-20T11:21:01.616 INFO:teuthology.orchestra.run.trial038.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-20T11:21:01.616 INFO:teuthology.orchestra.run.trial038.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-20T11:21:01.616 INFO:teuthology.orchestra.run.trial038.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-20T11:21:01.616 INFO:teuthology.orchestra.run.trial038.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-20T11:21:01.616 INFO:teuthology.orchestra.run.trial038.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-20T11:21:01.616 INFO:teuthology.orchestra.run.trial038.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-20T11:21:01.616 INFO:teuthology.orchestra.run.trial038.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-20T11:21:01.616 INFO:teuthology.orchestra.run.trial038.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-20T11:21:01.616 INFO:teuthology.orchestra.run.trial038.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-20T11:21:01.617 INFO:teuthology.orchestra.run.trial038.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-20T11:21:01.617 INFO:teuthology.orchestra.run.trial038.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-20T11:21:01.617 INFO:teuthology.orchestra.run.trial038.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-20T11:21:01.617 INFO:teuthology.orchestra.run.trial038.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-20T11:21:01.617 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-20T11:21:01.617 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-20T11:21:01.617 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-20T11:21:01.617 INFO:teuthology.orchestra.run.trial038.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-20T11:21:01.617 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-20T11:21:01.617 INFO:teuthology.orchestra.run.trial038.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-20T11:21:01.617 INFO:teuthology.orchestra.run.trial038.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-20T11:21:01.617 INFO:teuthology.orchestra.run.trial038.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-20T11:21:01.618 INFO:teuthology.orchestra.run.trial038.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-20T11:21:01.618 INFO:teuthology.orchestra.run.trial038.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-20T11:21:01.618 INFO:teuthology.orchestra.run.trial038.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-20T11:21:01.618 INFO:teuthology.orchestra.run.trial038.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-20T11:21:01.618 INFO:teuthology.orchestra.run.trial038.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-20T11:21:01.618 INFO:teuthology.orchestra.run.trial038.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-20T11:21:01.618 INFO:teuthology.orchestra.run.trial038.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-20T11:21:01.618 INFO:teuthology.orchestra.run.trial038.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-20T11:21:01.618 INFO:teuthology.orchestra.run.trial038.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-20T11:21:01.618 INFO:teuthology.orchestra.run.trial038.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-20T11:21:01.618 INFO:teuthology.orchestra.run.trial038.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-20T11:21:01.618 INFO:teuthology.orchestra.run.trial038.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-20T11:21:01.619 INFO:teuthology.orchestra.run.trial038.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-20T11:21:01.619 INFO:teuthology.orchestra.run.trial038.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-20T11:21:01.619 INFO:teuthology.orchestra.run.trial038.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-20T11:21:01.619 INFO:teuthology.orchestra.run.trial038.stdout:Installing weak dependencies: 2026-02-20T11:21:01.619 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 184 k 2026-02-20T11:21:01.619 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 51 M 2026-02-20T11:21:01.619 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 7.4 M 2026-02-20T11:21:01.619 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 22 k 2026-02-20T11:21:01.619 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-rook noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 50 k 2026-02-20T11:21:01.619 INFO:teuthology.orchestra.run.trial038.stdout: ceph-volume noarch 2:20.3.0-5357.g0c3148ca.el9 ceph-noarch 301 k 2026-02-20T11:21:01.619 INFO:teuthology.orchestra.run.trial038.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-20T11:21:01.619 INFO:teuthology.orchestra.run.trial038.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-20T11:21:01.619 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:01.620 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-20T11:21:01.620 INFO:teuthology.orchestra.run.trial038.stdout:======================================================================================= 2026-02-20T11:21:01.620 INFO:teuthology.orchestra.run.trial038.stdout:Install 96 Packages 2026-02-20T11:21:01.620 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:01.621 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 226 M 2026-02-20T11:21:01.621 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 1.5 G 2026-02-20T11:21:01.621 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-20T11:21:02.157 INFO:teuthology.orchestra.run.trial038.stdout:(1/96): ceph-20.3.0-5357.g0c3148ca.el9.x86_64.r 25 kB/s | 6.5 kB 00:00 2026-02-20T11:21:02.846 INFO:teuthology.orchestra.run.trial038.stdout:(2/96): ceph-mds-20.3.0-5357.g0c3148ca.el9.x86_ 2.1 MB/s | 2.0 MB 00:00 2026-02-20T11:21:02.850 INFO:teuthology.orchestra.run.trial038.stdout:(3/96): ceph-mgr-20.3.0-5357.g0c3148ca.el9.x86_ 982 kB/s | 937 kB 00:00 2026-02-20T11:21:02.914 INFO:teuthology.orchestra.run.trial038.stdout:(4/96): ceph-osd-20.3.0-5357.g0c3148ca.el9.x86_ 2.4 MB/s | 163 kB 00:00 2026-02-20T11:21:04.916 INFO:teuthology.orchestra.run.trial038.stdout:(5/96): ceph-mon-20.3.0-5357.g0c3148ca.el9.x86_ 1.8 MB/s | 4.9 MB 00:02 2026-02-20T11:21:04.978 INFO:teuthology.orchestra.run.trial038.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5357.g0c 930 kB/s | 57 kB 00:00 2026-02-20T11:21:05.105 INFO:teuthology.orchestra.run.trial038.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5357.g0c3148ca. 1.4 MB/s | 184 kB 00:00 2026-02-20T11:21:11.219 INFO:teuthology.orchestra.run.trial038.stdout:(8/96): ceph-osd-classic-20.3.0-5357.g0c3148ca. 1.8 MB/s | 15 MB 00:08 2026-02-20T11:21:14.741 INFO:teuthology.orchestra.run.trial038.stdout:(9/96): ceph-mgr-diskprediction-local-20.3.0-53 2.1 MB/s | 7.4 MB 00:03 2026-02-20T11:21:14.805 INFO:teuthology.orchestra.run.trial038.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5357.g0c3148 348 kB/s | 22 kB 00:00 2026-02-20T11:21:14.945 INFO:teuthology.orchestra.run.trial038.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5357.g0c3 2.1 MB/s | 297 kB 00:00 2026-02-20T11:21:15.015 INFO:teuthology.orchestra.run.trial038.stdout:(12/96): ceph-mgr-rook-20.3.0-5357.g0c3148ca.el 714 kB/s | 50 kB 00:00 2026-02-20T11:21:15.081 INFO:teuthology.orchestra.run.trial038.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5357.g0c 263 kB/s | 17 kB 00:00 2026-02-20T11:21:15.222 INFO:teuthology.orchestra.run.trial038.stdout:(14/96): ceph-volume-20.3.0-5357.g0c3148ca.el9. 2.1 MB/s | 301 kB 00:00 2026-02-20T11:21:15.809 INFO:teuthology.orchestra.run.trial038.stdout:(15/96): cephadm-20.3.0-5357.g0c3148ca.el9.noar 1.7 MB/s | 1.0 MB 00:00 2026-02-20T11:21:15.942 INFO:teuthology.orchestra.run.trial038.stdout:(16/96): cryptsetup-2.8.1-3.el9.x86_64.rpm 2.6 MB/s | 351 kB 00:00 2026-02-20T11:21:15.965 INFO:teuthology.orchestra.run.trial038.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 1.7 MB/s | 40 kB 00:00 2026-02-20T11:21:15.988 INFO:teuthology.orchestra.run.trial038.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 3.0 MB/s | 72 kB 00:00 2026-02-20T11:21:16.098 INFO:teuthology.orchestra.run.trial038.stdout:(19/96): libgfortran-11.5.0-14.el9.x86_64.rpm 7.1 MB/s | 794 kB 00:00 2026-02-20T11:21:16.124 INFO:teuthology.orchestra.run.trial038.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 6.9 MB/s | 184 kB 00:00 2026-02-20T11:21:16.150 INFO:teuthology.orchestra.run.trial038.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 9.6 MB/s | 253 kB 00:00 2026-02-20T11:21:16.212 INFO:teuthology.orchestra.run.trial038.stdout:(22/96): python3-cryptography-36.0.1-5.el9.x86_ 20 MB/s | 1.2 MB 00:00 2026-02-20T11:21:16.237 INFO:teuthology.orchestra.run.trial038.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 4.2 MB/s | 106 kB 00:00 2026-02-20T11:21:16.268 INFO:teuthology.orchestra.run.trial038.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 4.3 MB/s | 135 kB 00:00 2026-02-20T11:21:16.293 INFO:teuthology.orchestra.run.trial038.stdout:(25/96): python3-pyparsing-2.4.7-9.el9.noarch.r 5.8 MB/s | 150 kB 00:00 2026-02-20T11:21:16.319 INFO:teuthology.orchestra.run.trial038.stdout:(26/96): python3-requests-2.25.1-10.el9.noarch. 4.8 MB/s | 126 kB 00:00 2026-02-20T11:21:16.345 INFO:teuthology.orchestra.run.trial038.stdout:(27/96): python3-urllib3-1.26.5-7.el9.noarch.rp 8.3 MB/s | 218 kB 00:00 2026-02-20T11:21:16.371 INFO:teuthology.orchestra.run.trial038.stdout:(28/96): unzip-6.0-59.el9.x86_64.rpm 6.9 MB/s | 182 kB 00:00 2026-02-20T11:21:16.396 INFO:teuthology.orchestra.run.trial038.stdout:(29/96): zip-3.0-35.el9.x86_64.rpm 10 MB/s | 266 kB 00:00 2026-02-20T11:21:16.688 INFO:teuthology.orchestra.run.trial038.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 101 kB/s | 30 kB 00:00 2026-02-20T11:21:17.281 INFO:teuthology.orchestra.run.trial038.stdout:(31/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 5.0 MB/s | 3.0 MB 00:00 2026-02-20T11:21:17.383 INFO:teuthology.orchestra.run.trial038.stdout:(32/96): flexiblas-openblas-openmp-3.0.4-9.el9. 146 kB/s | 15 kB 00:00 2026-02-20T11:21:17.491 INFO:teuthology.orchestra.run.trial038.stdout:(33/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 2.2 MB/s | 246 kB 00:00 2026-02-20T11:21:17.599 INFO:teuthology.orchestra.run.trial038.stdout:(34/96): lua-5.4.4-4.el9.x86_64.rpm 1.7 MB/s | 188 kB 00:00 2026-02-20T11:21:17.700 INFO:teuthology.orchestra.run.trial038.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 416 kB/s | 42 kB 00:00 2026-02-20T11:21:17.934 INFO:teuthology.orchestra.run.trial038.stdout:(36/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 23 MB/s | 5.3 MB 00:00 2026-02-20T11:21:18.142 INFO:teuthology.orchestra.run.trial038.stdout:(37/96): python3-babel-2.9.1-2.el9.noarch.rpm 29 MB/s | 6.0 MB 00:00 2026-02-20T11:21:18.256 INFO:teuthology.orchestra.run.trial038.stdout:(38/96): python3-devel-3.9.25-3.el9.x86_64.rpm 2.1 MB/s | 244 kB 00:00 2026-02-20T11:21:18.355 INFO:teuthology.orchestra.run.trial038.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 2.5 MB/s | 249 kB 00:00 2026-02-20T11:21:18.456 INFO:teuthology.orchestra.run.trial038.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 474 kB/s | 48 kB 00:00 2026-02-20T11:21:18.559 INFO:teuthology.orchestra.run.trial038.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 1.7 MB/s | 177 kB 00:00 2026-02-20T11:21:18.673 INFO:teuthology.orchestra.run.trial038.stdout:(42/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 11 MB/s | 1.2 MB 00:00 2026-02-20T11:21:18.765 INFO:teuthology.orchestra.run.trial038.stdout:(43/96): python3-markupsafe-1.1.1-12.el9.x86_64 375 kB/s | 35 kB 00:00 2026-02-20T11:21:18.955 INFO:teuthology.orchestra.run.trial038.stdout:(44/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 32 MB/s | 6.1 MB 00:00 2026-02-20T11:21:19.050 INFO:teuthology.orchestra.run.trial038.stdout:(45/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 4.5 MB/s | 442 kB 00:00 2026-02-20T11:21:19.149 INFO:teuthology.orchestra.run.trial038.stdout:(46/96): python3-packaging-20.9-5.el9.noarch.rp 779 kB/s | 77 kB 00:00 2026-02-20T11:21:19.245 INFO:teuthology.orchestra.run.trial038.stdout:(47/96): python3-protobuf-3.14.0-17.el9.noarch. 2.7 MB/s | 267 kB 00:00 2026-02-20T11:21:19.337 INFO:teuthology.orchestra.run.trial038.stdout:(48/96): python3-pyasn1-0.4.8-6.el9.noarch.rpm 1.7 MB/s | 159 kB 00:00 2026-02-20T11:21:19.435 INFO:teuthology.orchestra.run.trial038.stdout:(49/96): python3-pyasn1-modules-0.4.8-6.el9.noa 2.8 MB/s | 279 kB 00:00 2026-02-20T11:21:19.529 INFO:teuthology.orchestra.run.trial038.stdout:(50/96): python3-requests-oauthlib-1.3.0-12.el9 569 kB/s | 54 kB 00:00 2026-02-20T11:21:20.021 INFO:teuthology.orchestra.run.trial038.stdout:(51/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 39 MB/s | 19 MB 00:00 2026-02-20T11:21:20.124 INFO:teuthology.orchestra.run.trial038.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 407 kB/s | 42 kB 00:00 2026-02-20T11:21:20.220 INFO:teuthology.orchestra.run.trial038.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 1.9 MB/s | 189 kB 00:00 2026-02-20T11:21:20.322 INFO:teuthology.orchestra.run.trial038.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 885 kB/s | 90 kB 00:00 2026-02-20T11:21:20.347 INFO:teuthology.orchestra.run.trial038.stdout:(55/96): lua-devel-5.4.4-4.el9.x86_64.rpm 908 kB/s | 22 kB 00:00 2026-02-20T11:21:20.389 INFO:teuthology.orchestra.run.trial038.stdout:(56/96): protobuf-compiler-3.14.0-17.el9.x86_64 20 MB/s | 862 kB 00:00 2026-02-20T11:21:20.687 INFO:teuthology.orchestra.run.trial038.stdout:(57/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 1.8 MB/s | 551 kB 00:00 2026-02-20T11:21:20.715 INFO:teuthology.orchestra.run.trial038.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 695 kB/s | 19 kB 00:00 2026-02-20T11:21:20.745 INFO:teuthology.orchestra.run.trial038.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 5.0 MB/s | 151 kB 00:00 2026-02-20T11:21:20.777 INFO:teuthology.orchestra.run.trial038.stdout:(60/96): python3-asyncssh-2.13.2-5.el9.noarch.r 17 MB/s | 548 kB 00:00 2026-02-20T11:21:20.805 INFO:teuthology.orchestra.run.trial038.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 1.0 MB/s | 29 kB 00:00 2026-02-20T11:21:20.834 INFO:teuthology.orchestra.run.trial038.stdout:(62/96): python3-backports-tarfile-1.2.0-1.el9. 2.0 MB/s | 60 kB 00:00 2026-02-20T11:21:20.863 INFO:teuthology.orchestra.run.trial038.stdout:(63/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.5 MB/s | 43 kB 00:00 2026-02-20T11:21:20.891 INFO:teuthology.orchestra.run.trial038.stdout:(64/96): python3-cachetools-4.2.4-1.el9.noarch. 1.1 MB/s | 32 kB 00:00 2026-02-20T11:21:20.919 INFO:teuthology.orchestra.run.trial038.stdout:(65/96): python3-certifi-2023.05.07-4.el9.noarc 505 kB/s | 14 kB 00:00 2026-02-20T11:21:20.949 INFO:teuthology.orchestra.run.trial038.stdout:(66/96): python3-cheroot-10.0.1-4.el9.noarch.rp 5.6 MB/s | 173 kB 00:00 2026-02-20T11:21:20.984 INFO:teuthology.orchestra.run.trial038.stdout:(67/96): python3-cherrypy-18.6.1-2.el9.noarch.r 10 MB/s | 358 kB 00:00 2026-02-20T11:21:21.015 INFO:teuthology.orchestra.run.trial038.stdout:(68/96): python3-google-auth-2.45.0-1.el9.noarc 8.1 MB/s | 254 kB 00:00 2026-02-20T11:21:21.108 INFO:teuthology.orchestra.run.trial038.stdout:(69/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 22 MB/s | 2.0 MB 00:00 2026-02-20T11:21:21.139 INFO:teuthology.orchestra.run.trial038.stdout:(70/96): python3-grpcio-tools-1.46.7-10.el9.x86 4.7 MB/s | 144 kB 00:00 2026-02-20T11:21:21.169 INFO:teuthology.orchestra.run.trial038.stdout:(71/96): python3-influxdb-5.3.1-1.el9.noarch.rp 4.5 MB/s | 139 kB 00:00 2026-02-20T11:21:21.198 INFO:teuthology.orchestra.run.trial038.stdout:(72/96): python3-isodate-0.6.1-3.el9.noarch.rpm 1.9 MB/s | 56 kB 00:00 2026-02-20T11:21:21.225 INFO:teuthology.orchestra.run.trial038.stdout:(73/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 385 kB/s | 11 kB 00:00 2026-02-20T11:21:21.254 INFO:teuthology.orchestra.run.trial038.stdout:(74/96): python3-jaraco-classes-3.2.1-5.el9.noa 629 kB/s | 18 kB 00:00 2026-02-20T11:21:21.282 INFO:teuthology.orchestra.run.trial038.stdout:(75/96): python3-jaraco-collections-3.0.0-8.el9 813 kB/s | 23 kB 00:00 2026-02-20T11:21:21.310 INFO:teuthology.orchestra.run.trial038.stdout:(76/96): python3-jaraco-context-6.0.1-3.el9.noa 699 kB/s | 20 kB 00:00 2026-02-20T11:21:21.338 INFO:teuthology.orchestra.run.trial038.stdout:(77/96): python3-jaraco-functools-3.5.0-2.el9.n 696 kB/s | 19 kB 00:00 2026-02-20T11:21:21.366 INFO:teuthology.orchestra.run.trial038.stdout:(78/96): python3-jaraco-text-4.0.0-2.el9.noarch 941 kB/s | 26 kB 00:00 2026-02-20T11:21:21.399 INFO:teuthology.orchestra.run.trial038.stdout:(79/96): python3-kubernetes-26.1.0-3.el9.noarch 32 MB/s | 1.0 MB 00:00 2026-02-20T11:21:21.428 INFO:teuthology.orchestra.run.trial038.stdout:(80/96): python3-more-itertools-8.12.0-2.el9.no 2.6 MB/s | 79 kB 00:00 2026-02-20T11:21:21.457 INFO:teuthology.orchestra.run.trial038.stdout:(81/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 2.9 MB/s | 86 kB 00:00 2026-02-20T11:21:21.486 INFO:teuthology.orchestra.run.trial038.stdout:(82/96): python3-natsort-7.1.1-5.el9.noarch.rpm 2.0 MB/s | 58 kB 00:00 2026-02-20T11:21:21.514 INFO:teuthology.orchestra.run.trial038.stdout:(83/96): python3-portend-3.1.0-2.el9.noarch.rpm 595 kB/s | 16 kB 00:00 2026-02-20T11:21:21.542 INFO:teuthology.orchestra.run.trial038.stdout:(84/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.1 MB/s | 90 kB 00:00 2026-02-20T11:21:21.571 INFO:teuthology.orchestra.run.trial038.stdout:(85/96): python3-repoze-lru-0.7-16.el9.noarch.r 1.1 MB/s | 31 kB 00:00 2026-02-20T11:21:21.600 INFO:teuthology.orchestra.run.trial038.stdout:(86/96): python3-routes-2.5.1-5.el9.noarch.rpm 6.3 MB/s | 188 kB 00:00 2026-02-20T11:21:21.629 INFO:teuthology.orchestra.run.trial038.stdout:(87/96): python3-rsa-4.9-2.el9.noarch.rpm 2.0 MB/s | 59 kB 00:00 2026-02-20T11:21:21.658 INFO:teuthology.orchestra.run.trial038.stdout:(88/96): python3-saml-1.16.0-1.el9.noarch.rpm 4.2 MB/s | 125 kB 00:00 2026-02-20T11:21:21.686 INFO:teuthology.orchestra.run.trial038.stdout:(89/96): python3-tempora-5.0.0-2.el9.noarch.rpm 1.2 MB/s | 36 kB 00:00 2026-02-20T11:21:21.715 INFO:teuthology.orchestra.run.trial038.stdout:(90/96): python3-typing-extensions-4.15.0-1.el9 2.9 MB/s | 86 kB 00:00 2026-02-20T11:21:21.744 INFO:teuthology.orchestra.run.trial038.stdout:(91/96): python3-websocket-client-1.2.3-2.el9.n 3.1 MB/s | 90 kB 00:00 2026-02-20T11:21:21.772 INFO:teuthology.orchestra.run.trial038.stdout:(92/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 1.7 MB/s | 48 kB 00:00 2026-02-20T11:21:21.800 INFO:teuthology.orchestra.run.trial038.stdout:(93/96): python3-xmltodict-0.12.0-15.el9.noarch 795 kB/s | 22 kB 00:00 2026-02-20T11:21:21.828 INFO:teuthology.orchestra.run.trial038.stdout:(94/96): python3-zc-lockfile-2.0-10.el9.noarch. 716 kB/s | 20 kB 00:00 2026-02-20T11:21:34.270 INFO:teuthology.orchestra.run.trial038.stdout:(95/96): ceph-osd-crimson-20.3.0-5357.g0c3148ca 2.8 MB/s | 88 MB 00:31 2026-02-20T11:21:34.697 INFO:teuthology.orchestra.run.trial038.stdout:(96/96): ceph-mgr-dashboard-20.3.0-5357.g0c3148 1.7 MB/s | 51 MB 00:29 2026-02-20T11:21:34.700 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:21:34.700 INFO:teuthology.orchestra.run.trial038.stdout:Total 6.8 MB/s | 226 MB 00:33 2026-02-20T11:21:35.095 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-20T11:21:35.132 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-20T11:21:35.132 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-20T11:21:35.933 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-20T11:21:35.933 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-20T11:21:36.777 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-20T11:21:36.779 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-20T11:21:36.782 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-20T11:21:36.803 INFO:teuthology.orchestra.run.trial038.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-20T11:21:36.836 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 4/96 2026-02-20T11:21:36.842 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-20T11:21:36.845 INFO:teuthology.orchestra.run.trial038.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-20T11:21:36.849 INFO:teuthology.orchestra.run.trial038.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-20T11:21:36.872 INFO:teuthology.orchestra.run.trial038.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-20T11:21:36.885 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-20T11:21:36.888 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-20T11:21:36.894 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-20T11:21:36.896 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-20T11:21:36.907 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-20T11:21:36.914 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-20T11:21:36.937 INFO:teuthology.orchestra.run.trial038.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-20T11:21:36.940 INFO:teuthology.orchestra.run.trial038.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-20T11:21:36.989 INFO:teuthology.orchestra.run.trial038.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-20T11:21:37.002 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 18/96 2026-02-20T11:21:37.004 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-20T11:21:37.009 INFO:teuthology.orchestra.run.trial038.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-20T11:21:37.014 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-20T11:21:37.019 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-20T11:21:37.022 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-20T11:21:37.036 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-20T11:21:37.040 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-20T11:21:37.045 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-20T11:21:37.056 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-20T11:21:37.065 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-20T11:21:37.072 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-20T11:21:37.091 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-20T11:21:37.101 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-20T11:21:37.117 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-20T11:21:37.121 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-20T11:21:37.129 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-20T11:21:37.168 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-20T11:21:37.486 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-20T11:21:37.496 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-20T11:21:37.500 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-20T11:21:37.506 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-20T11:21:37.507 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-20T11:21:37.534 INFO:teuthology.orchestra.run.trial038.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-20T11:21:37.577 INFO:teuthology.orchestra.run.trial038.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-20T11:21:37.587 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-20T11:21:37.593 INFO:teuthology.orchestra.run.trial038.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-20T11:21:37.596 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-20T11:21:37.602 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-20T11:21:37.605 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-20T11:21:37.612 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-20T11:21:37.616 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-20T11:21:37.643 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-20T11:21:37.652 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-20T11:21:37.658 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-20T11:21:37.665 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-20T11:21:37.698 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-20T11:21:37.918 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 55/96 2026-02-20T11:21:37.941 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-20T11:21:37.945 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-20T11:21:38.004 INFO:teuthology.orchestra.run.trial038.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-20T11:21:38.006 INFO:teuthology.orchestra.run.trial038.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-20T11:21:38.027 INFO:teuthology.orchestra.run.trial038.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-20T11:21:38.346 INFO:teuthology.orchestra.run.trial038.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-20T11:21:38.418 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-20T11:21:39.077 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-20T11:21:39.093 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-20T11:21:39.103 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-20T11:21:39.110 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-20T11:21:39.123 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-20T11:21:39.138 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-20T11:21:39.213 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-20T11:21:39.221 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-20T11:21:39.244 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-20T11:21:39.273 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-20T11:21:39.323 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-20T11:21:39.329 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-20T11:21:39.332 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-20T11:21:39.334 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-20T11:21:39.351 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-20T11:21:39.351 INFO:teuthology.orchestra.run.trial038.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-20T11:21:39.351 INFO:teuthology.orchestra.run.trial038.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-20T11:21:39.352 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:39.359 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-20T11:21:39.384 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-20T11:21:39.384 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-20T11:21:39.384 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:39.421 INFO:teuthology.orchestra.run.trial038.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-20T11:21:40.018 INFO:teuthology.orchestra.run.trial038.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 79/96 2026-02-20T11:21:45.504 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_6 80/96 2026-02-20T11:21:45.566 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_6 81/96 2026-02-20T11:21:45.569 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch 82/96 2026-02-20T11:21:45.583 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch 82/96 2026-02-20T11:21:45.583 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:21:45.583 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-20T11:21:45.583 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:45.593 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 83/96 2026-02-20T11:21:45.617 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 83/96 2026-02-20T11:21:45.618 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:21:45.618 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-20T11:21:45.618 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T11:21:45.618 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-20T11:21:45.618 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:45.676 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 84/96 2026-02-20T11:21:45.678 INFO:teuthology.orchestra.run.trial038.stdout: Installing : cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 84/96 2026-02-20T11:21:45.687 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5357.g0c3148ca.el9 85/96 2026-02-20T11:21:45.710 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5357.g0c3148ca.el 86/96 2026-02-20T11:21:45.711 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarc 87/96 2026-02-20T11:21:48.634 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarc 87/96 2026-02-20T11:21:48.641 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noa 88/96 2026-02-20T11:21:49.098 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noa 88/96 2026-02-20T11:21:49.099 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c314 89/96 2026-02-20T11:21:49.111 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c314 89/96 2026-02-20T11:21:49.112 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5357.g0c3148ca.el9.noa 90/96 2026-02-20T11:21:49.175 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5357.g0c3148ca.el9.noa 90/96 2026-02-20T11:21:49.215 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5357.g0c3148ca.el9. 91/96 2026-02-20T11:21:49.216 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 92/96 2026-02-20T11:21:49.238 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 92/96 2026-02-20T11:21:49.238 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:21:49.238 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-20T11:21:49.238 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T11:21:49.238 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-20T11:21:49.238 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:49.249 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch 93/96 2026-02-20T11:21:49.471 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch 93/96 2026-02-20T11:21:49.472 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mon-2:20.3.0-5357.g0c3148ca.el9.x86_64 94/96 2026-02-20T11:21:49.493 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mon-2:20.3.0-5357.g0c3148ca.el9.x86_64 94/96 2026-02-20T11:21:49.493 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:21:49.493 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-20T11:21:49.493 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T11:21:49.493 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-20T11:21:49.493 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:49.576 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-mds-2:20.3.0-5357.g0c3148ca.el9.x86_64 95/96 2026-02-20T11:21:49.596 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-mds-2:20.3.0-5357.g0c3148ca.el9.x86_64 95/96 2026-02-20T11:21:49.597 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:21:49.597 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-20T11:21:49.597 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T11:21:49.597 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-20T11:21:49.597 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:49.598 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 96/96 2026-02-20T11:21:49.604 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_6 96/96 2026-02-20T11:21:49.628 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_6 96/96 2026-02-20T11:21:51.047 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 96/96 2026-02-20T11:21:51.048 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/96 2026-02-20T11:21:51.048 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mds-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/96 2026-02-20T11:21:51.048 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 3/96 2026-02-20T11:21:51.048 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mon-2:20.3.0-5357.g0c3148ca.el9.x86_64 4/96 2026-02-20T11:21:51.048 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 5/96 2026-02-20T11:21:51.048 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_6 6/96 2026-02-20T11:21:51.048 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_6 7/96 2026-02-20T11:21:51.048 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5357.g0c3148ca.el 8/96 2026-02-20T11:21:51.049 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarc 9/96 2026-02-20T11:21:51.049 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noa 10/96 2026-02-20T11:21:51.049 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c314 11/96 2026-02-20T11:21:51.049 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5357.g0c3148ca.el9.noa 12/96 2026-02-20T11:21:51.049 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5357.g0c3148ca.el9. 13/96 2026-02-20T11:21:51.049 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch 14/96 2026-02-20T11:21:51.049 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5357.g0c3148ca.el9 15/96 2026-02-20T11:21:51.049 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch 16/96 2026-02-20T11:21:51.049 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 17/96 2026-02-20T11:21:51.049 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 18/96 2026-02-20T11:21:51.050 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-20T11:21:51.050 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-20T11:21:51.050 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-20T11:21:51.050 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-20T11:21:51.050 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-20T11:21:51.050 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-20T11:21:51.050 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-20T11:21:51.050 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-20T11:21:51.050 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-20T11:21:51.050 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-20T11:21:51.050 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-20T11:21:51.051 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-20T11:21:51.051 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-20T11:21:51.051 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-20T11:21:51.051 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-20T11:21:51.051 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-20T11:21:51.051 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-20T11:21:51.051 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-20T11:21:51.051 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-20T11:21:51.051 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-20T11:21:51.051 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-20T11:21:51.051 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 40/96 2026-02-20T11:21:51.051 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-20T11:21:51.052 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-20T11:21:51.052 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-20T11:21:51.052 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-20T11:21:51.052 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-20T11:21:51.052 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-20T11:21:51.052 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-20T11:21:51.052 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-20T11:21:51.052 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-20T11:21:51.052 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 50/96 2026-02-20T11:21:51.052 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 51/96 2026-02-20T11:21:51.052 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-20T11:21:51.052 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-20T11:21:51.053 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-20T11:21:51.053 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-20T11:21:51.053 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-20T11:21:51.053 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-20T11:21:51.053 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-20T11:21:51.053 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-20T11:21:51.053 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-20T11:21:51.053 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-20T11:21:51.053 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-20T11:21:51.053 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-20T11:21:51.054 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-20T11:21:51.054 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-20T11:21:51.054 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-20T11:21:51.054 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-20T11:21:51.054 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-20T11:21:51.054 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-20T11:21:51.054 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-20T11:21:51.054 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-20T11:21:51.054 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-20T11:21:51.054 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-20T11:21:51.054 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-20T11:21:51.055 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-20T11:21:51.055 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-20T11:21:51.055 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-20T11:21:51.055 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-20T11:21:51.055 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-20T11:21:51.055 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-20T11:21:51.055 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-20T11:21:51.055 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-20T11:21:51.055 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-20T11:21:51.055 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-20T11:21:51.055 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-20T11:21:51.055 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-20T11:21:51.055 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-20T11:21:51.056 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-20T11:21:51.056 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-20T11:21:51.056 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-20T11:21:51.056 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-20T11:21:51.056 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-20T11:21:51.056 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-20T11:21:51.056 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-20T11:21:51.056 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-20T11:21:51.110 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-20T11:21:51.111 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:51.111 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-20T11:21:51.111 INFO:teuthology.orchestra.run.trial038.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-20T11:21:51.111 INFO:teuthology.orchestra.run.trial038.stdout: ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:21:51.111 INFO:teuthology.orchestra.run.trial038.stdout: ceph-grafana-dashboards-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:21:51.111 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mds-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:21:51.111 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:21:51.111 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:21:51.112 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:21:51.112 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:21:51.112 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-k8sevents-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:21:51.112 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-modules-core-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:21:51.112 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:21:51.112 INFO:teuthology.orchestra.run.trial038.stdout: ceph-mon-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:21:51.112 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:21:51.112 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:21:51.112 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:21:51.112 INFO:teuthology.orchestra.run.trial038.stdout: ceph-prometheus-alerts-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:21:51.112 INFO:teuthology.orchestra.run.trial038.stdout: ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:21:51.113 INFO:teuthology.orchestra.run.trial038.stdout: cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch 2026-02-20T11:21:51.113 INFO:teuthology.orchestra.run.trial038.stdout: cryptsetup-2.8.1-3.el9.x86_64 2026-02-20T11:21:51.113 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-20T11:21:51.113 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-20T11:21:51.113 INFO:teuthology.orchestra.run.trial038.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-20T11:21:51.113 INFO:teuthology.orchestra.run.trial038.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-20T11:21:51.113 INFO:teuthology.orchestra.run.trial038.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-20T11:21:51.113 INFO:teuthology.orchestra.run.trial038.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-20T11:21:51.113 INFO:teuthology.orchestra.run.trial038.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-20T11:21:51.113 INFO:teuthology.orchestra.run.trial038.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-20T11:21:51.113 INFO:teuthology.orchestra.run.trial038.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T11:21:51.114 INFO:teuthology.orchestra.run.trial038.stdout: lua-5.4.4-4.el9.x86_64 2026-02-20T11:21:51.114 INFO:teuthology.orchestra.run.trial038.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-20T11:21:51.114 INFO:teuthology.orchestra.run.trial038.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-20T11:21:51.114 INFO:teuthology.orchestra.run.trial038.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-20T11:21:51.114 INFO:teuthology.orchestra.run.trial038.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-20T11:21:51.114 INFO:teuthology.orchestra.run.trial038.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-20T11:21:51.114 INFO:teuthology.orchestra.run.trial038.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-20T11:21:51.114 INFO:teuthology.orchestra.run.trial038.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-20T11:21:51.114 INFO:teuthology.orchestra.run.trial038.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-20T11:21:51.114 INFO:teuthology.orchestra.run.trial038.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-20T11:21:51.114 INFO:teuthology.orchestra.run.trial038.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-20T11:21:51.114 INFO:teuthology.orchestra.run.trial038.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-20T11:21:51.115 INFO:teuthology.orchestra.run.trial038.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-20T11:21:51.115 INFO:teuthology.orchestra.run.trial038.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-20T11:21:51.115 INFO:teuthology.orchestra.run.trial038.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-20T11:21:51.115 INFO:teuthology.orchestra.run.trial038.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-20T11:21:51.115 INFO:teuthology.orchestra.run.trial038.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-20T11:21:51.115 INFO:teuthology.orchestra.run.trial038.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-20T11:21:51.115 INFO:teuthology.orchestra.run.trial038.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-20T11:21:51.115 INFO:teuthology.orchestra.run.trial038.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-20T11:21:51.115 INFO:teuthology.orchestra.run.trial038.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-20T11:21:51.115 INFO:teuthology.orchestra.run.trial038.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-20T11:21:51.116 INFO:teuthology.orchestra.run.trial038.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-20T11:21:51.116 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-20T11:21:51.116 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-20T11:21:51.116 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-20T11:21:51.116 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-20T11:21:51.116 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-20T11:21:51.116 INFO:teuthology.orchestra.run.trial038.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-20T11:21:51.116 INFO:teuthology.orchestra.run.trial038.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-20T11:21:51.116 INFO:teuthology.orchestra.run.trial038.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-20T11:21:51.116 INFO:teuthology.orchestra.run.trial038.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-20T11:21:51.116 INFO:teuthology.orchestra.run.trial038.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-20T11:21:51.117 INFO:teuthology.orchestra.run.trial038.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-20T11:21:51.117 INFO:teuthology.orchestra.run.trial038.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-20T11:21:51.117 INFO:teuthology.orchestra.run.trial038.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-20T11:21:51.117 INFO:teuthology.orchestra.run.trial038.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-20T11:21:51.117 INFO:teuthology.orchestra.run.trial038.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-20T11:21:51.117 INFO:teuthology.orchestra.run.trial038.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-20T11:21:51.117 INFO:teuthology.orchestra.run.trial038.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-20T11:21:51.117 INFO:teuthology.orchestra.run.trial038.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-20T11:21:51.117 INFO:teuthology.orchestra.run.trial038.stdout: python3-ply-3.11-14.el9.noarch 2026-02-20T11:21:51.117 INFO:teuthology.orchestra.run.trial038.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-20T11:21:51.117 INFO:teuthology.orchestra.run.trial038.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-20T11:21:51.117 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-20T11:21:51.117 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-20T11:21:51.118 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-20T11:21:51.118 INFO:teuthology.orchestra.run.trial038.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-20T11:21:51.118 INFO:teuthology.orchestra.run.trial038.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-20T11:21:51.118 INFO:teuthology.orchestra.run.trial038.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-20T11:21:51.118 INFO:teuthology.orchestra.run.trial038.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-20T11:21:51.118 INFO:teuthology.orchestra.run.trial038.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-20T11:21:51.118 INFO:teuthology.orchestra.run.trial038.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-20T11:21:51.118 INFO:teuthology.orchestra.run.trial038.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-20T11:21:51.118 INFO:teuthology.orchestra.run.trial038.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-20T11:21:51.118 INFO:teuthology.orchestra.run.trial038.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-20T11:21:51.118 INFO:teuthology.orchestra.run.trial038.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-20T11:21:51.118 INFO:teuthology.orchestra.run.trial038.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-20T11:21:51.118 INFO:teuthology.orchestra.run.trial038.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-20T11:21:51.119 INFO:teuthology.orchestra.run.trial038.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-20T11:21:51.119 INFO:teuthology.orchestra.run.trial038.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-20T11:21:51.119 INFO:teuthology.orchestra.run.trial038.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-20T11:21:51.119 INFO:teuthology.orchestra.run.trial038.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-20T11:21:51.119 INFO:teuthology.orchestra.run.trial038.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-20T11:21:51.119 INFO:teuthology.orchestra.run.trial038.stdout: unzip-6.0-59.el9.x86_64 2026-02-20T11:21:51.119 INFO:teuthology.orchestra.run.trial038.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-20T11:21:51.119 INFO:teuthology.orchestra.run.trial038.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-20T11:21:51.119 INFO:teuthology.orchestra.run.trial038.stdout: zip-3.0-35.el9.x86_64 2026-02-20T11:21:51.119 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:51.119 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:21:51.181 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-base 2026-02-20T11:21:51.488 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:51 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:21:51.552 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:21:51.570 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:21:51.571 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:21:51.571 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:21:51.592 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install cephadm 2026-02-20T11:21:51.906 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:51 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:21:51.970 INFO:teuthology.orchestra.run.trial038.stdout:Package cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:21:51.989 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:21:51.989 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:21:51.990 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:21:52.012 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-immutable-object-cache 2026-02-20T11:21:52.321 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:52 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:21:52.404 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:21:52.405 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:21:52.405 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-20T11:21:52.405 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:21:52.405 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-20T11:21:52.405 INFO:teuthology.orchestra.run.trial038.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 149 k 2026-02-20T11:21:52.405 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:52.405 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-20T11:21:52.405 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:21:52.406 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-20T11:21:52.406 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:52.406 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 149 k 2026-02-20T11:21:52.406 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 455 k 2026-02-20T11:21:52.406 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-20T11:21:52.874 INFO:teuthology.orchestra.run.trial038.stdout:ceph-immutable-object-cache-20.3.0-5357.g0c3148 319 kB/s | 149 kB 00:00 2026-02-20T11:21:52.875 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:21:52.875 INFO:teuthology.orchestra.run.trial038.stdout:Total 318 kB/s | 149 kB 00:00 2026-02-20T11:21:52.875 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-20T11:21:52.879 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-20T11:21:52.880 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-20T11:21:52.911 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-20T11:21:52.911 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-20T11:21:52.966 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-20T11:21:52.967 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca. 1/1 2026-02-20T11:21:52.990 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca. 1/1 2026-02-20T11:21:52.990 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:21:52.990 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-20T11:21:52.990 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:53.312 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca. 1/1 2026-02-20T11:21:53.312 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:53.312 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-20T11:21:53.312 INFO:teuthology.orchestra.run.trial038.stdout: ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:21:53.312 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:53.312 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:21:53.340 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr 2026-02-20T11:21:53.648 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:53 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:21:53.713 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:21:53.732 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:21:53.733 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:21:53.733 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:21:53.754 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-dashboard 2026-02-20T11:21:54.066 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:54 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:21:54.130 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:21:54.148 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:21:54.149 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:21:54.149 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:21:54.171 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-20T11:21:54.476 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:54 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:21:54.541 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:21:54.559 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:21:54.560 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:21:54.560 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:21:54.581 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-rook 2026-02-20T11:21:54.888 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:54 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:21:54.955 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:21:54.973 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:21:54.974 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:21:54.974 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:21:54.995 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-cephadm 2026-02-20T11:21:55.297 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:55 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:21:55.362 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:21:55.381 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:21:55.381 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:21:55.381 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:21:55.402 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-osd 2026-02-20T11:21:55.714 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:55 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:21:55.778 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:21:55.797 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:21:55.797 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:21:55.798 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:21:55.819 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-osd-classic 2026-02-20T11:21:56.124 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:56 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:21:56.187 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:21:56.206 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:21:56.206 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:21:56.206 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:21:56.228 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-fuse 2026-02-20T11:21:56.532 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:56 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:21:56.616 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:21:56.616 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:21:56.616 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-20T11:21:56.616 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:21:56.616 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-20T11:21:56.616 INFO:teuthology.orchestra.run.trial038.stdout: ceph-fuse x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 914 k 2026-02-20T11:21:56.616 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:56.616 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-20T11:21:56.616 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:21:56.617 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-20T11:21:56.617 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:56.617 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 914 k 2026-02-20T11:21:56.617 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 2.9 M 2026-02-20T11:21:56.617 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-20T11:21:57.387 INFO:teuthology.orchestra.run.trial038.stdout:ceph-fuse-20.3.0-5357.g0c3148ca.el9.x86_64.rpm 1.2 MB/s | 914 kB 00:00 2026-02-20T11:21:57.388 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:21:57.388 INFO:teuthology.orchestra.run.trial038.stdout:Total 1.2 MB/s | 914 kB 00:00 2026-02-20T11:21:57.388 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-20T11:21:57.393 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-20T11:21:57.393 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-20T11:21:57.426 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-20T11:21:57.426 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-20T11:21:57.516 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-20T11:21:57.533 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:21:57.776 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:21:57.834 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:21:57.834 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:57.834 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-20T11:21:57.834 INFO:teuthology.orchestra.run.trial038.stdout: ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:21:57.834 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:57.834 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:21:57.861 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-volume 2026-02-20T11:21:58.167 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:58 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:21:58.234 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:21:58.252 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:21:58.253 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:21:58.253 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:21:58.274 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librados-devel 2026-02-20T11:21:58.585 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:58 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:21:58.669 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:21:58.669 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:21:58.669 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-20T11:21:58.670 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:21:58.670 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-20T11:21:58.670 INFO:teuthology.orchestra.run.trial038.stdout: librados-devel x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 124 k 2026-02-20T11:21:58.670 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:58.670 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-20T11:21:58.670 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:21:58.670 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-20T11:21:58.670 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:58.670 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 124 k 2026-02-20T11:21:58.670 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 434 k 2026-02-20T11:21:58.670 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-20T11:21:59.206 INFO:teuthology.orchestra.run.trial038.stdout:librados-devel-20.3.0-5357.g0c3148ca.el9.x86_64 231 kB/s | 124 kB 00:00 2026-02-20T11:21:59.206 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:21:59.207 INFO:teuthology.orchestra.run.trial038.stdout:Total 230 kB/s | 124 kB 00:00 2026-02-20T11:21:59.207 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-20T11:21:59.209 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-20T11:21:59.209 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-20T11:21:59.244 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-20T11:21:59.244 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-20T11:21:59.300 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-20T11:21:59.317 INFO:teuthology.orchestra.run.trial038.stdout: Installing : librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:21:59.394 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:21:59.452 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:21:59.452 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:59.452 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-20T11:21:59.452 INFO:teuthology.orchestra.run.trial038.stdout: librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:21:59.452 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:21:59.452 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:21:59.482 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install libcephfs2 2026-02-20T11:21:59.799 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:01:59 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:21:59.864 INFO:teuthology.orchestra.run.trial038.stdout:Package libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:21:59.882 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:21:59.882 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:21:59.883 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:21:59.904 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install libcephfs-devel 2026-02-20T11:22:00.218 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:00 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:22:00.300 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:22:00.300 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:22:00.301 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-20T11:22:00.301 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:22:00.301 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-20T11:22:00.301 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs-devel x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 37 k 2026-02-20T11:22:00.301 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:00.301 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-20T11:22:00.301 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:22:00.301 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-20T11:22:00.301 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:00.301 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 37 k 2026-02-20T11:22:00.301 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 171 k 2026-02-20T11:22:00.301 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-20T11:22:00.542 INFO:teuthology.orchestra.run.trial038.stdout:libcephfs-devel-20.3.0-5357.g0c3148ca.el9.x86_6 155 kB/s | 37 kB 00:00 2026-02-20T11:22:00.542 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:22:00.542 INFO:teuthology.orchestra.run.trial038.stdout:Total 155 kB/s | 37 kB 00:00 2026-02-20T11:22:00.542 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-20T11:22:00.545 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-20T11:22:00.545 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-20T11:22:00.547 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-20T11:22:00.547 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-20T11:22:00.568 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-20T11:22:00.585 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:22:00.642 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:22:00.700 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:22:00.701 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:00.701 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-20T11:22:00.701 INFO:teuthology.orchestra.run.trial038.stdout: libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:22:00.701 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:00.702 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:22:00.730 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librados2 2026-02-20T11:22:01.040 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:01 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:22:01.105 INFO:teuthology.orchestra.run.trial038.stdout:Package librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:22:01.123 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:22:01.124 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:22:01.124 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:22:01.146 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librbd1 2026-02-20T11:22:01.461 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:01 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:22:01.526 INFO:teuthology.orchestra.run.trial038.stdout:Package librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:22:01.544 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:22:01.544 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:22:01.545 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:22:01.566 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rados 2026-02-20T11:22:01.875 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:01 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:22:01.941 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:22:01.960 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:22:01.960 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:22:01.960 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:22:01.982 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rgw 2026-02-20T11:22:02.290 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:02 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:22:02.356 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:22:02.374 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:22:02.375 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:22:02.375 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:22:02.397 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-cephfs 2026-02-20T11:22:02.711 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:02 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:22:02.775 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:22:02.793 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:22:02.793 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:22:02.794 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:22:02.815 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rbd 2026-02-20T11:22:03.130 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:03 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:22:03.200 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:22:03.218 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:22:03.218 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:22:03.219 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:22:03.240 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-fuse 2026-02-20T11:22:03.543 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:03 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:22:03.627 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:22:03.627 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:22:03.627 INFO:teuthology.orchestra.run.trial038.stdout: Package Architecture Version Repository Size 2026-02-20T11:22:03.627 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:22:03.627 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-20T11:22:03.628 INFO:teuthology.orchestra.run.trial038.stdout: rbd-fuse x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 86 k 2026-02-20T11:22:03.628 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:03.628 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-20T11:22:03.628 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:22:03.628 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-20T11:22:03.628 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:03.628 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 86 k 2026-02-20T11:22:03.628 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 236 k 2026-02-20T11:22:03.628 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-20T11:22:04.592 INFO:teuthology.orchestra.run.trial038.stdout:rbd-fuse-20.3.0-5357.g0c3148ca.el9.x86_64.rpm 90 kB/s | 86 kB 00:00 2026-02-20T11:22:04.593 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:22:04.593 INFO:teuthology.orchestra.run.trial038.stdout:Total 89 kB/s | 86 kB 00:00 2026-02-20T11:22:04.593 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-20T11:22:04.597 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-20T11:22:04.597 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-20T11:22:04.628 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-20T11:22:04.628 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-20T11:22:04.680 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-20T11:22:04.696 INFO:teuthology.orchestra.run.trial038.stdout: Installing : rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:22:04.774 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:22:04.833 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:22:04.833 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:04.833 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-20T11:22:04.833 INFO:teuthology.orchestra.run.trial038.stdout: rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:22:04.833 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:04.833 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:22:04.863 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-mirror 2026-02-20T11:22:05.177 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:05 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:22:05.264 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:22:05.265 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:22:05.265 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-20T11:22:05.265 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:22:05.265 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-20T11:22:05.265 INFO:teuthology.orchestra.run.trial038.stdout: rbd-mirror x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 2.6 M 2026-02-20T11:22:05.265 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:05.265 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-20T11:22:05.265 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:22:05.265 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-20T11:22:05.266 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:05.266 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 2.6 M 2026-02-20T11:22:05.266 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 11 M 2026-02-20T11:22:05.266 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-20T11:22:07.174 INFO:teuthology.orchestra.run.trial038.stdout:rbd-mirror-20.3.0-5357.g0c3148ca.el9.x86_64.rpm 1.4 MB/s | 2.6 MB 00:01 2026-02-20T11:22:07.174 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:22:07.174 INFO:teuthology.orchestra.run.trial038.stdout:Total 1.4 MB/s | 2.6 MB 00:01 2026-02-20T11:22:07.174 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-20T11:22:07.179 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-20T11:22:07.180 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-20T11:22:07.219 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-20T11:22:07.219 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-20T11:22:07.382 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-20T11:22:07.383 INFO:teuthology.orchestra.run.trial038.stdout: Installing : rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:22:07.408 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:22:07.409 INFO:teuthology.orchestra.run.trial038.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-20T11:22:07.409 INFO:teuthology.orchestra.run.trial038.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-20T11:22:07.409 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T11:22:07.409 INFO:teuthology.orchestra.run.trial038.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-20T11:22:07.409 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:07.708 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:22:07.708 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:07.708 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-20T11:22:07.709 INFO:teuthology.orchestra.run.trial038.stdout: rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:22:07.709 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:07.709 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:22:07.737 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-nbd 2026-02-20T11:22:08.053 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:08 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:22:08.135 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:22:08.136 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:22:08.136 INFO:teuthology.orchestra.run.trial038.stdout: Package Architecture Version Repository Size 2026-02-20T11:22:08.136 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:22:08.136 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-20T11:22:08.136 INFO:teuthology.orchestra.run.trial038.stdout: rbd-nbd x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 167 k 2026-02-20T11:22:08.136 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:08.136 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-20T11:22:08.136 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:22:08.136 INFO:teuthology.orchestra.run.trial038.stdout:Install 1 Package 2026-02-20T11:22:08.136 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:08.136 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 167 k 2026-02-20T11:22:08.136 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 496 k 2026-02-20T11:22:08.136 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-20T11:22:08.530 INFO:teuthology.orchestra.run.trial038.stdout:rbd-nbd-20.3.0-5357.g0c3148ca.el9.x86_64.rpm 424 kB/s | 167 kB 00:00 2026-02-20T11:22:08.531 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:22:08.531 INFO:teuthology.orchestra.run.trial038.stdout:Total 423 kB/s | 167 kB 00:00 2026-02-20T11:22:08.531 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-20T11:22:08.535 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-20T11:22:08.536 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-20T11:22:08.569 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-20T11:22:08.569 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-20T11:22:08.627 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-20T11:22:08.644 INFO:teuthology.orchestra.run.trial038.stdout: Installing : rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:22:08.721 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:22:08.778 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/1 2026-02-20T11:22:08.779 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:08.779 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-20T11:22:08.779 INFO:teuthology.orchestra.run.trial038.stdout: rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:22:08.779 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:08.779 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:22:08.807 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-osd-crimson 2026-02-20T11:22:09.127 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:09 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:22:09.191 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-osd-crimson-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:22:09.209 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:22:09.210 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:22:09.211 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:22:09.231 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-20T11:22:09.544 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:02:09 ago on Fri 20 Feb 2026 11:20:00 AM UTC. 2026-02-20T11:22:09.630 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:22:09.630 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:22:09.631 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repo Size 2026-02-20T11:22:09.631 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:22:09.631 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-20T11:22:09.631 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 433 M 2026-02-20T11:22:09.631 INFO:teuthology.orchestra.run.trial038.stdout:Installing weak dependencies: 2026-02-20T11:22:09.631 INFO:teuthology.orchestra.run.trial038.stdout: ceph-debugsource x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 12 M 2026-02-20T11:22:09.631 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:09.631 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-20T11:22:09.631 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:22:09.631 INFO:teuthology.orchestra.run.trial038.stdout:Install 2 Packages 2026-02-20T11:22:09.631 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:22:09.632 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 445 M 2026-02-20T11:22:09.632 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 2.7 G 2026-02-20T11:22:09.632 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-20T11:22:11.165 INFO:teuthology.orchestra.run.trial038.stdout:(1/2): ceph-debugsource-20.3.0-5357.g0c3148ca.e 7.8 MB/s | 12 MB 00:01 2026-02-20T11:23:46.102 INFO:teuthology.orchestra.run.trial038.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5357.g 4.5 MB/s | 433 MB 01:36 2026-02-20T11:23:46.103 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:23:46.104 INFO:teuthology.orchestra.run.trial038.stdout:Total 4.6 MB/s | 445 MB 01:36 2026-02-20T11:23:46.104 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-20T11:23:46.109 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-20T11:23:46.109 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-20T11:23:47.489 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-20T11:23:47.489 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-20T11:23:51.090 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-20T11:24:13.443 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-debugsource-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/2 2026-02-20T11:24:13.460 INFO:teuthology.orchestra.run.trial038.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5357.g0c3148ca.e 2/2 2026-02-20T11:24:15.477 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5357.g0c3148ca.e 2/2 2026-02-20T11:24:15.477 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-debugsource-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/2 2026-02-20T11:24:15.536 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5357.g0c3148ca.e 2/2 2026-02-20T11:24:15.536 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:15.536 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-20T11:24:15.537 INFO:teuthology.orchestra.run.trial038.stdout: ceph-debugsource-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:24:15.537 INFO:teuthology.orchestra.run.trial038.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:24:15.537 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:15.537 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:15.607 DEBUG:teuthology.parallel:result is None 2026-02-20T11:24:15.607 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:24:15.778 DEBUG:teuthology.orchestra.run.trial038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-20T11:24:15.800 INFO:teuthology.orchestra.run.trial038.stdout:20.3.0-5357.g0c3148ca.el9 2026-02-20T11:24:15.801 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5357.g0c3148ca.el9 2026-02-20T11:24:15.801 INFO:teuthology.task.install:The correct ceph version 20.3.0-5357.g0c3148ca is installed. 2026-02-20T11:24:15.803 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-20T11:24:15.803 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:24:15.803 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-20T11:24:15.865 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-20T11:24:15.865 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:24:15.865 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/usr/bin/daemon-helper 2026-02-20T11:24:15.930 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-20T11:24:15.995 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-20T11:24:15.995 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:24:15.995 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-20T11:24:16.059 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-20T11:24:16.122 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-20T11:24:16.122 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:24:16.122 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/usr/bin/stdin-killer 2026-02-20T11:24:16.187 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-20T11:24:16.253 INFO:teuthology.run_tasks:Running task pexec... 2026-02-20T11:24:16.266 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-20T11:24:16.267 DEBUG:teuthology.orchestra.run.trial038:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-20T11:24:16.295 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial038.front.sepia.ceph.com 2026-02-20T11:24:16.295 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-20T11:24:16.345 DEBUG:teuthology.parallel:result is None 2026-02-20T11:24:16.346 INFO:teuthology.run_tasks:Running task install... 2026-02-20T11:24:16.364 DEBUG:teuthology.task.install:project ceph 2026-02-20T11:24:16.364 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}} 2026-02-20T11:24:16.364 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'} 2026-02-20T11:24:16.364 INFO:teuthology.task.install:Using flavor: debug 2026-02-20T11:24:16.373 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-20T11:24:16.373 INFO:teuthology.task.install:extra packages: [] 2026-02-20T11:24:16.373 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4', 'tag': None, 'wait_for_package': False} 2026-02-20T11:24:16.374 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:24:16.539 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/ 2026-02-20T11:24:16.539 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5357.g0c3148ca 2026-02-20T11:24:16.859 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-perezjos-crimson-only-19-02-2026-PR64439/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-20T11:24:16.859 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:24:16.859 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-20T11:24:16.883 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-20T11:24:16.883 DEBUG:teuthology.orchestra.run.trial038:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/0c3148ca19c827ca2a7a9b92c2554d99507fd1f4/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-20T11:24:16.948 DEBUG:teuthology.orchestra.run.trial038:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-20T11:24:17.013 DEBUG:teuthology.orchestra.run.trial038:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-20T11:24:17.069 INFO:teuthology.orchestra.run.trial038.stdout:check_obsoletes = 1 2026-02-20T11:24:17.079 DEBUG:teuthology.orchestra.run.trial038:> sudo yum clean all 2026-02-20T11:24:17.241 INFO:teuthology.orchestra.run.trial038.stdout:56 files removed 2026-02-20T11:24:17.256 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-radosgw 2026-02-20T11:24:17.901 INFO:teuthology.orchestra.run.trial038.stdout:ceph packages for x86_64 190 kB/s | 94 kB 00:00 2026-02-20T11:24:18.388 INFO:teuthology.orchestra.run.trial038.stdout:ceph noarch packages 53 kB/s | 24 kB 00:00 2026-02-20T11:24:18.821 INFO:teuthology.orchestra.run.trial038.stdout:ceph source packages 5.0 kB/s | 2.1 kB 00:00 2026-02-20T11:24:20.474 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - BaseOS 5.4 MB/s | 8.9 MB 00:01 2026-02-20T11:24:22.547 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - AppStream 17 MB/s | 27 MB 00:01 2026-02-20T11:24:26.607 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - CRB 4.6 MB/s | 7.8 MB 00:01 2026-02-20T11:24:28.055 INFO:teuthology.orchestra.run.trial038.stdout:CentOS Stream 9 - Extras packages 28 kB/s | 20 kB 00:00 2026-02-20T11:24:29.006 INFO:teuthology.orchestra.run.trial038.stdout:Extra Packages for Enterprise Linux 23 MB/s | 20 MB 00:00 2026-02-20T11:24:32.197 INFO:teuthology.orchestra.run.trial038.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-20T11:24:33.379 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-radosgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:33.398 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:33.399 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:33.399 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:33.425 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-test 2026-02-20T11:24:33.739 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:01 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:33.806 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-test-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:33.824 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:33.824 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:33.824 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:33.846 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph 2026-02-20T11:24:34.159 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:34.224 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:34.242 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:34.243 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:34.244 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:34.265 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-base 2026-02-20T11:24:34.577 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:34.641 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-base-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:34.660 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:34.661 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:34.661 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:34.682 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install cephadm 2026-02-20T11:24:34.990 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:02 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:35.054 INFO:teuthology.orchestra.run.trial038.stdout:Package cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:24:35.073 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:35.073 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:35.074 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:35.094 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-immutable-object-cache 2026-02-20T11:24:35.406 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:03 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:35.471 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-immutable-object-cache-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:35.490 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:35.491 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:35.491 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:35.511 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr 2026-02-20T11:24:35.819 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:03 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:35.884 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:35.903 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:35.903 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:35.903 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:35.925 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-dashboard 2026-02-20T11:24:36.231 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:04 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:36.297 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-dashboard-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:24:36.315 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:36.316 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:36.316 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:36.338 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-20T11:24:36.656 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:04 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:36.722 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:24:36.740 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:36.740 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:36.740 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:36.762 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-rook 2026-02-20T11:24:37.073 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:05 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:37.138 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-rook-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:24:37.156 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:37.157 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:37.157 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:37.180 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-mgr-cephadm 2026-02-20T11:24:37.493 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:05 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:37.560 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-mgr-cephadm-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:24:37.578 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:37.579 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:37.579 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:37.600 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-osd 2026-02-20T11:24:37.910 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:05 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:37.975 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-osd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:37.995 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:37.995 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:37.996 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:38.017 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-osd-classic 2026-02-20T11:24:38.324 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:06 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:38.389 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-osd-classic-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:38.407 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:38.408 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:38.408 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:38.428 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-fuse 2026-02-20T11:24:38.742 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:06 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:38.806 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:38.825 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:38.825 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:38.826 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:38.847 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install ceph-volume 2026-02-20T11:24:39.156 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:07 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:39.223 INFO:teuthology.orchestra.run.trial038.stdout:Package ceph-volume-2:20.3.0-5357.g0c3148ca.el9.noarch is already installed. 2026-02-20T11:24:39.241 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:39.242 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:39.242 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:39.264 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librados-devel 2026-02-20T11:24:39.577 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:07 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:39.644 INFO:teuthology.orchestra.run.trial038.stdout:Package librados-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:39.663 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:39.663 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:39.663 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:39.685 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install libcephfs2 2026-02-20T11:24:39.996 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:07 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:40.062 INFO:teuthology.orchestra.run.trial038.stdout:Package libcephfs2-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:40.080 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:40.081 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:40.081 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:40.102 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install libcephfs-devel 2026-02-20T11:24:40.419 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:08 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:40.487 INFO:teuthology.orchestra.run.trial038.stdout:Package libcephfs-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:40.506 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:40.506 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:40.507 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:40.528 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librados2 2026-02-20T11:24:40.834 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:08 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:40.899 INFO:teuthology.orchestra.run.trial038.stdout:Package librados2-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:40.917 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:40.918 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:40.918 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:40.939 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librbd1 2026-02-20T11:24:41.245 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:09 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:41.311 INFO:teuthology.orchestra.run.trial038.stdout:Package librbd1-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:41.330 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:41.331 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:41.331 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:41.353 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rados 2026-02-20T11:24:41.669 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:09 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:41.736 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-rados-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:41.755 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:41.755 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:41.755 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:41.777 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rgw 2026-02-20T11:24:42.090 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:10 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:42.155 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-rgw-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:42.174 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:42.175 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:42.175 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:42.197 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-cephfs 2026-02-20T11:24:42.508 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:10 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:42.574 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-cephfs-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:42.592 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:42.593 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:42.593 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:42.615 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install python3-rbd 2026-02-20T11:24:42.930 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:10 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:42.997 INFO:teuthology.orchestra.run.trial038.stdout:Package python3-rbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:43.016 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:43.017 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:43.017 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:43.038 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-fuse 2026-02-20T11:24:43.353 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:11 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:43.420 INFO:teuthology.orchestra.run.trial038.stdout:Package rbd-fuse-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:43.438 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:43.439 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:43.439 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:43.460 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-mirror 2026-02-20T11:24:43.778 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:11 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:43.843 INFO:teuthology.orchestra.run.trial038.stdout:Package rbd-mirror-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:43.862 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:43.862 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:43.863 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:43.885 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install rbd-nbd 2026-02-20T11:24:44.199 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:12 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:24:44.264 INFO:teuthology.orchestra.run.trial038.stdout:Package rbd-nbd-2:20.3.0-5357.g0c3148ca.el9.x86_64 is already installed. 2026-02-20T11:24:44.282 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:24:44.283 INFO:teuthology.orchestra.run.trial038.stdout:Nothing to do. 2026-02-20T11:24:44.283 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:24:44.307 DEBUG:teuthology.parallel:result is None 2026-02-20T11:24:44.308 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=0c3148ca19c827ca2a7a9b92c2554d99507fd1f4 2026-02-20T11:24:44.559 DEBUG:teuthology.orchestra.run.trial038:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-20T11:24:44.581 INFO:teuthology.orchestra.run.trial038.stdout:20.3.0-5357.g0c3148ca.el9 2026-02-20T11:24:44.582 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5357.g0c3148ca.el9 2026-02-20T11:24:44.582 INFO:teuthology.task.install:The correct ceph version 20.3.0-5357.g0c3148ca is installed. 2026-02-20T11:24:44.584 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-20T11:24:44.584 INFO:teuthology.run_tasks:Running task ceph... 2026-02-20T11:24:44.684 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-20T11:24:44.684 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod 777 /var/log/ceph 2026-02-20T11:24:44.709 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-20T11:24:44.709 DEBUG:teuthology.orchestra.run.trial038:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-20T11:24:44.774 INFO:tasks.ceph:Creating extra log directories... 2026-02-20T11:24:44.774 DEBUG:teuthology.orchestra.run.trial038:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-20T11:24:44.852 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-20T11:24:44.852 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-20T11:24:44.852 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial/60503', 'branch': 'wip-perezjos-crimson-only-19-02-2026-PR64439', 'description': 'crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} perf_ignorelist settings/optimized workloads/radosbench_4M_seq_read}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '60503', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'jjperez-2026-02-20_10:21:31-crimson-rados-wip-perezjos-crimson-only-19-02-2026-PR64439-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-perezjos-crimson-only-19-02-2026-PR64439'}, 'ceph': {'conf': {'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': '445d5c9ba18a4040e70e25303230c36c77773acb'}}, 'owner': 'scheduled_jjperez@soko04.front.sepia.ceph.com', 'priority': 101, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 1736, 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_445d5c9ba18a4040e70e25303230c36c77773acb/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': '445d5c9ba18a4040e70e25303230c36c77773acb', 'targets': {'trial038.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBI9Tb0uNtlqm3jR0v/dEugkpUoH0vqC8/9JBwkiq7f0M1eYWLFMugxzKmPHpwuYg1K5iUFFf69EvmZ2s846qAYo='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'wait-for-scrub': False, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': '0c3148ca19c827ca2a7a9b92c2554d99507fd1f4', 'cluster': 'ceph'}}, {'ssh_keys': None}, {'cbt': {'benchmarks': {'radosbench': {'concurrent_ops': 4, 'concurrent_procs': 2, 'op_size': [4194304], 'pool_monitoring_list': ['collectl'], 'pool_profile': 'replicated', 'run_monitoring_list': ['collectl'], 'time': 60, 'write_only': False}}, 'cluster': {'iterations': 1, 'osds_per_node': 3, 'pool_profiles': {'replicated': {'pg_size': 256, 'pgp_size': 256, 'replication': 'replicated'}}, 'user': 'ubuntu'}, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'reimage-ssh-down', 'teuthology_sha1': 'e6d9099ce03dc21193fda653159ec82465c3a94a', 'timestamp': '2026-02-20_10:21:31', 'tube': 'trial', 'user': 'jjperez', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.1540699'} 2026-02-20T11:24:44.853 DEBUG:teuthology.orchestra.run.trial038:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-20T11:24:44.895 DEBUG:teuthology.orchestra.run.trial038:> sudo install -d -m0777 -- /var/run/ceph 2026-02-20T11:24:44.960 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:24:44.960 DEBUG:teuthology.orchestra.run.trial038:> dd if=/scratch_devs of=/dev/stdout 2026-02-20T11:24:45.016 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-20T11:24:45.016 DEBUG:teuthology.orchestra.run.trial038:> stat /dev/vg_nvme/lv_1 2026-02-20T11:24:45.073 INFO:teuthology.orchestra.run.trial038.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-20T11:24:45.074 INFO:teuthology.orchestra.run.trial038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T11:24:45.074 INFO:teuthology.orchestra.run.trial038.stdout:Device: 6h/6d Inode: 1302 Links: 1 2026-02-20T11:24:45.074 INFO:teuthology.orchestra.run.trial038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T11:24:45.074 INFO:teuthology.orchestra.run.trial038.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T11:24:45.074 INFO:teuthology.orchestra.run.trial038.stdout:Access: 2026-02-20 11:22:07.550671036 +0000 2026-02-20T11:24:45.074 INFO:teuthology.orchestra.run.trial038.stdout:Modify: 2026-02-20 11:20:13.942880573 +0000 2026-02-20T11:24:45.074 INFO:teuthology.orchestra.run.trial038.stdout:Change: 2026-02-20 11:20:13.942880573 +0000 2026-02-20T11:24:45.075 INFO:teuthology.orchestra.run.trial038.stdout: Birth: 2026-02-20 11:20:13.942880573 +0000 2026-02-20T11:24:45.075 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-20T11:24:45.136 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records in 2026-02-20T11:24:45.136 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records out 2026-02-20T11:24:45.136 INFO:teuthology.orchestra.run.trial038.stderr:512 bytes copied, 0.000159947 s, 3.2 MB/s 2026-02-20T11:24:45.138 DEBUG:teuthology.orchestra.run.trial038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-20T11:24:45.194 DEBUG:teuthology.orchestra.run.trial038:> stat /dev/vg_nvme/lv_2 2026-02-20T11:24:45.253 INFO:teuthology.orchestra.run.trial038.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-20T11:24:45.253 INFO:teuthology.orchestra.run.trial038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T11:24:45.253 INFO:teuthology.orchestra.run.trial038.stdout:Device: 6h/6d Inode: 1287 Links: 1 2026-02-20T11:24:45.253 INFO:teuthology.orchestra.run.trial038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T11:24:45.253 INFO:teuthology.orchestra.run.trial038.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T11:24:45.253 INFO:teuthology.orchestra.run.trial038.stdout:Access: 2026-02-20 11:22:07.551671008 +0000 2026-02-20T11:24:45.253 INFO:teuthology.orchestra.run.trial038.stdout:Modify: 2026-02-20 11:20:13.942880573 +0000 2026-02-20T11:24:45.253 INFO:teuthology.orchestra.run.trial038.stdout:Change: 2026-02-20 11:20:13.942880573 +0000 2026-02-20T11:24:45.253 INFO:teuthology.orchestra.run.trial038.stdout: Birth: 2026-02-20 11:20:13.942880573 +0000 2026-02-20T11:24:45.253 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-20T11:24:45.315 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records in 2026-02-20T11:24:45.315 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records out 2026-02-20T11:24:45.315 INFO:teuthology.orchestra.run.trial038.stderr:512 bytes copied, 0.000141322 s, 3.6 MB/s 2026-02-20T11:24:45.316 DEBUG:teuthology.orchestra.run.trial038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-20T11:24:45.375 DEBUG:teuthology.orchestra.run.trial038:> stat /dev/vg_nvme/lv_3 2026-02-20T11:24:45.432 INFO:teuthology.orchestra.run.trial038.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-20T11:24:45.432 INFO:teuthology.orchestra.run.trial038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T11:24:45.432 INFO:teuthology.orchestra.run.trial038.stdout:Device: 6h/6d Inode: 1308 Links: 1 2026-02-20T11:24:45.432 INFO:teuthology.orchestra.run.trial038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T11:24:45.432 INFO:teuthology.orchestra.run.trial038.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T11:24:45.432 INFO:teuthology.orchestra.run.trial038.stdout:Access: 2026-02-20 11:22:07.551671008 +0000 2026-02-20T11:24:45.432 INFO:teuthology.orchestra.run.trial038.stdout:Modify: 2026-02-20 11:20:13.942880573 +0000 2026-02-20T11:24:45.432 INFO:teuthology.orchestra.run.trial038.stdout:Change: 2026-02-20 11:20:13.942880573 +0000 2026-02-20T11:24:45.432 INFO:teuthology.orchestra.run.trial038.stdout: Birth: 2026-02-20 11:20:13.942880573 +0000 2026-02-20T11:24:45.433 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-20T11:24:45.496 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records in 2026-02-20T11:24:45.496 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records out 2026-02-20T11:24:45.497 INFO:teuthology.orchestra.run.trial038.stderr:512 bytes copied, 0.000167631 s, 3.1 MB/s 2026-02-20T11:24:45.498 DEBUG:teuthology.orchestra.run.trial038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-20T11:24:45.556 DEBUG:teuthology.orchestra.run.trial038:> stat /dev/vg_nvme/lv_4 2026-02-20T11:24:45.613 INFO:teuthology.orchestra.run.trial038.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-20T11:24:45.613 INFO:teuthology.orchestra.run.trial038.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-20T11:24:45.613 INFO:teuthology.orchestra.run.trial038.stdout:Device: 6h/6d Inode: 1295 Links: 1 2026-02-20T11:24:45.613 INFO:teuthology.orchestra.run.trial038.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-20T11:24:45.613 INFO:teuthology.orchestra.run.trial038.stdout:Context: system_u:object_r:device_t:s0 2026-02-20T11:24:45.613 INFO:teuthology.orchestra.run.trial038.stdout:Access: 2026-02-20 11:22:07.551671008 +0000 2026-02-20T11:24:45.613 INFO:teuthology.orchestra.run.trial038.stdout:Modify: 2026-02-20 11:20:13.942880573 +0000 2026-02-20T11:24:45.613 INFO:teuthology.orchestra.run.trial038.stdout:Change: 2026-02-20 11:20:13.942880573 +0000 2026-02-20T11:24:45.613 INFO:teuthology.orchestra.run.trial038.stdout: Birth: 2026-02-20 11:20:13.942880573 +0000 2026-02-20T11:24:45.614 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-20T11:24:45.676 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records in 2026-02-20T11:24:45.676 INFO:teuthology.orchestra.run.trial038.stderr:1+0 records out 2026-02-20T11:24:45.676 INFO:teuthology.orchestra.run.trial038.stderr:512 bytes copied, 0.000142695 s, 3.6 MB/s 2026-02-20T11:24:45.677 DEBUG:teuthology.orchestra.run.trial038:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-20T11:24:45.734 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T11:24:45.735 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial038.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-20T11:24:45.735 INFO:tasks.ceph:Generating config... 2026-02-20T11:24:45.737 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-20T11:24:45.737 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-20T11:24:45.738 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-20T11:24:45.738 INFO:tasks.ceph:[osd] debug filestore = 0/0 2026-02-20T11:24:45.738 INFO:tasks.ceph:[osd] debug journal = 0/0 2026-02-20T11:24:45.738 INFO:tasks.ceph:[osd] debug ms = 0/0 2026-02-20T11:24:45.738 INFO:tasks.ceph:[osd] debug osd = 0/0 2026-02-20T11:24:45.738 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-20T11:24:45.738 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-20T11:24:45.738 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-20T11:24:45.738 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-20T11:24:45.739 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-20T11:24:45.739 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-20T11:24:45.739 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-20T11:24:45.739 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-20T11:24:45.739 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-20T11:24:45.739 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-20T11:24:45.739 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-20T11:24:45.739 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-20T11:24:45.739 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-20T11:24:45.739 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-20T11:24:45.739 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-20T11:24:45.739 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-20T11:24:45.740 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-20T11:24:45.740 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-20T11:24:45.740 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-20T11:24:45.740 INFO:tasks.ceph:[global] auth client required = none 2026-02-20T11:24:45.740 INFO:tasks.ceph:[global] auth cluster required = none 2026-02-20T11:24:45.740 INFO:tasks.ceph:[global] auth service required = none 2026-02-20T11:24:45.740 INFO:tasks.ceph:[global] auth supported = none 2026-02-20T11:24:45.740 INFO:tasks.ceph:[global] debug asok = 0/0 2026-02-20T11:24:45.740 INFO:tasks.ceph:[global] debug auth = 0/0 2026-02-20T11:24:45.740 INFO:tasks.ceph:[global] debug bdev = 0/0 2026-02-20T11:24:45.740 INFO:tasks.ceph:[global] debug bluefs = 0/0 2026-02-20T11:24:45.741 INFO:tasks.ceph:[global] debug bluestore = 0/0 2026-02-20T11:24:45.741 INFO:tasks.ceph:[global] debug buffer = 0/0 2026-02-20T11:24:45.741 INFO:tasks.ceph:[global] debug civetweb = 0/0 2026-02-20T11:24:45.741 INFO:tasks.ceph:[global] debug client = 0/0 2026-02-20T11:24:45.741 INFO:tasks.ceph:[global] debug compressor = 0/0 2026-02-20T11:24:45.741 INFO:tasks.ceph:[global] debug context = 0/0 2026-02-20T11:24:45.741 INFO:tasks.ceph:[global] debug crush = 0/0 2026-02-20T11:24:45.741 INFO:tasks.ceph:[global] debug crypto = 0/0 2026-02-20T11:24:45.741 INFO:tasks.ceph:[global] debug dpdk = 0/0 2026-02-20T11:24:45.741 INFO:tasks.ceph:[global] debug eventtrace = 0/0 2026-02-20T11:24:45.741 INFO:tasks.ceph:[global] debug filer = 0/0 2026-02-20T11:24:45.741 INFO:tasks.ceph:[global] debug filestore = 0/0 2026-02-20T11:24:45.742 INFO:tasks.ceph:[global] debug finisher = 0/0 2026-02-20T11:24:45.742 INFO:tasks.ceph:[global] debug fuse = 0/0 2026-02-20T11:24:45.742 INFO:tasks.ceph:[global] debug heartbeatmap = 0/0 2026-02-20T11:24:45.742 INFO:tasks.ceph:[global] debug javaclient = 0/0 2026-02-20T11:24:45.742 INFO:tasks.ceph:[global] debug journal = 0/0 2026-02-20T11:24:45.742 INFO:tasks.ceph:[global] debug journaler = 0/0 2026-02-20T11:24:45.742 INFO:tasks.ceph:[global] debug kstore = 0/0 2026-02-20T11:24:45.742 INFO:tasks.ceph:[global] debug leveldb = 0/0 2026-02-20T11:24:45.742 INFO:tasks.ceph:[global] debug lockdep = 0/0 2026-02-20T11:24:45.742 INFO:tasks.ceph:[global] debug mds = 0/0 2026-02-20T11:24:45.743 INFO:tasks.ceph:[global] debug mds balancer = 0/0 2026-02-20T11:24:45.743 INFO:tasks.ceph:[global] debug mds locker = 0/0 2026-02-20T11:24:45.743 INFO:tasks.ceph:[global] debug mds log = 0/0 2026-02-20T11:24:45.743 INFO:tasks.ceph:[global] debug mds log expire = 0/0 2026-02-20T11:24:45.743 INFO:tasks.ceph:[global] debug mds migrator = 0/0 2026-02-20T11:24:45.743 INFO:tasks.ceph:[global] debug memdb = 0/0 2026-02-20T11:24:45.743 INFO:tasks.ceph:[global] debug mgr = 0/0 2026-02-20T11:24:45.743 INFO:tasks.ceph:[global] debug mgrc = 0/0 2026-02-20T11:24:45.743 INFO:tasks.ceph:[global] debug mon = 0/0 2026-02-20T11:24:45.743 INFO:tasks.ceph:[global] debug monc = 0/0 2026-02-20T11:24:45.743 INFO:tasks.ceph:[global] debug ms = 0/0 2026-02-20T11:24:45.743 INFO:tasks.ceph:[global] debug objclass = 0/0 2026-02-20T11:24:45.744 INFO:tasks.ceph:[global] debug objectcacher = 0/0 2026-02-20T11:24:45.744 INFO:tasks.ceph:[global] debug objecter = 0/0 2026-02-20T11:24:45.744 INFO:tasks.ceph:[global] debug optracker = 0/0 2026-02-20T11:24:45.744 INFO:tasks.ceph:[global] debug osd = 0/0 2026-02-20T11:24:45.744 INFO:tasks.ceph:[global] debug paxos = 0/0 2026-02-20T11:24:45.744 INFO:tasks.ceph:[global] debug perfcounter = 0/0 2026-02-20T11:24:45.744 INFO:tasks.ceph:[global] debug rados = 0/0 2026-02-20T11:24:45.744 INFO:tasks.ceph:[global] debug rbd = 0/0 2026-02-20T11:24:45.744 INFO:tasks.ceph:[global] debug rbd mirror = 0/0 2026-02-20T11:24:45.744 INFO:tasks.ceph:[global] debug rbd replay = 0/0 2026-02-20T11:24:45.744 INFO:tasks.ceph:[global] debug refs = 0/0 2026-02-20T11:24:45.744 INFO:tasks.ceph:[global] debug rgw = 0/0 2026-02-20T11:24:45.745 INFO:tasks.ceph:[global] debug rgw lifecycle = 0/0 2026-02-20T11:24:45.745 INFO:tasks.ceph:[global] debug rgw notification = 0/0 2026-02-20T11:24:45.745 INFO:tasks.ceph:[global] debug rgw sync = 0/0 2026-02-20T11:24:45.745 INFO:tasks.ceph:[global] debug rocksdb = 0/0 2026-02-20T11:24:45.745 INFO:tasks.ceph:[global] debug striper = 0/0 2026-02-20T11:24:45.745 INFO:tasks.ceph:[global] debug throttle = 0/0 2026-02-20T11:24:45.745 INFO:tasks.ceph:[global] debug timer = 0/0 2026-02-20T11:24:45.745 INFO:tasks.ceph:[global] debug tp = 0/0 2026-02-20T11:24:45.745 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-20T11:24:45.745 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-20T11:24:45.746 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-20T11:24:45.746 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-20T11:24:45.746 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-20T11:24:45.746 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-20T11:24:45.746 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-20T11:24:45.746 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-20T11:24:45.746 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-20T11:24:45.746 INFO:tasks.ceph:[mon] debug mon = 0/0 2026-02-20T11:24:45.746 INFO:tasks.ceph:[mon] debug ms = 0/0 2026-02-20T11:24:45.746 INFO:tasks.ceph:[mon] debug paxos = 0/0 2026-02-20T11:24:45.747 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-20T11:24:45.747 INFO:tasks.ceph:Setting up mon.a... 2026-02-20T11:24:45.747 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-20T11:24:45.815 INFO:teuthology.orchestra.run.trial038.stdout:creating /etc/ceph/ceph.keyring 2026-02-20T11:24:45.817 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-20T11:24:45.903 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-20T11:24:45.927 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.38')] 2026-02-20T11:24:45.928 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'none', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.38', 'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': '0/0', 'debug mon': '0/0', 'debug paxos': '0/0', 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-20T11:24:45.929 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:24:45.929 DEBUG:teuthology.orchestra.run.trial038:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-20T11:24:45.986 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.38 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:24:46.067 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:24:46.066+0000 7fc4324caf40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:24:46.068 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:24:46.066+0000 7fc4324caf40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:24:46.068 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:24:46.066+0000 7fc4324caf40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:24:46.069 INFO:teuthology.orchestra.run.trial038.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:24:46.069 INFO:teuthology.orchestra.run.trial038.stdout:monmaptool: generated fsid 97ccc1d9-5f9e-4a23-a83d-fcf45d35bda8 2026-02-20T11:24:46.069 INFO:teuthology.orchestra.run.trial038.stdout:setting min_mon_release = tentacle 2026-02-20T11:24:46.069 INFO:teuthology.orchestra.run.trial038.stdout:epoch 0 2026-02-20T11:24:46.070 INFO:teuthology.orchestra.run.trial038.stdout:fsid 97ccc1d9-5f9e-4a23-a83d-fcf45d35bda8 2026-02-20T11:24:46.070 INFO:teuthology.orchestra.run.trial038.stdout:last_changed 2026-02-20T11:24:46.066649+0000 2026-02-20T11:24:46.070 INFO:teuthology.orchestra.run.trial038.stdout:created 2026-02-20T11:24:46.066649+0000 2026-02-20T11:24:46.070 INFO:teuthology.orchestra.run.trial038.stdout:min_mon_release 20 (tentacle) 2026-02-20T11:24:46.070 INFO:teuthology.orchestra.run.trial038.stdout:election_strategy: 1 2026-02-20T11:24:46.070 INFO:teuthology.orchestra.run.trial038.stdout:0: [v2:10.20.193.38:3300/0,v1:10.20.193.38:6789/0] mon.a 2026-02-20T11:24:46.070 INFO:teuthology.orchestra.run.trial038.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-20T11:24:46.071 DEBUG:teuthology.orchestra.run.trial038:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-20T11:24:46.125 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 97ccc1d9-5f9e-4a23-a83d-fcf45d35bda8... 2026-02-20T11:24:46.127 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-20T11:24:46.207 INFO:teuthology.orchestra.run.trial038.stdout:[global] 2026-02-20T11:24:46.207 INFO:teuthology.orchestra.run.trial038.stdout: chdir = "" 2026-02-20T11:24:46.207 INFO:teuthology.orchestra.run.trial038.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-20T11:24:46.207 INFO:teuthology.orchestra.run.trial038.stdout: auth supported = none 2026-02-20T11:24:46.207 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.208 INFO:teuthology.orchestra.run.trial038.stdout: filestore xattr use omap = true 2026-02-20T11:24:46.208 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.208 INFO:teuthology.orchestra.run.trial038.stdout: mon clock drift allowed = 1.000 2026-02-20T11:24:46.208 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.208 INFO:teuthology.orchestra.run.trial038.stdout: osd crush chooseleaf type = 0 2026-02-20T11:24:46.208 INFO:teuthology.orchestra.run.trial038.stdout: auth debug = true 2026-02-20T11:24:46.208 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.208 INFO:teuthology.orchestra.run.trial038.stdout: ms die on old message = true 2026-02-20T11:24:46.208 INFO:teuthology.orchestra.run.trial038.stdout: ms die on bug = true 2026-02-20T11:24:46.208 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.208 INFO:teuthology.orchestra.run.trial038.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-20T11:24:46.209 INFO:teuthology.orchestra.run.trial038.stdout: mon pg warn max object skew = 0 2026-02-20T11:24:46.209 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.209 INFO:teuthology.orchestra.run.trial038.stdout: # disable pg_autoscaler by default for new pools 2026-02-20T11:24:46.209 INFO:teuthology.orchestra.run.trial038.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-20T11:24:46.209 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.209 INFO:teuthology.orchestra.run.trial038.stdout: osd pool default size = 2 2026-02-20T11:24:46.209 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.209 INFO:teuthology.orchestra.run.trial038.stdout: mon osd allow primary affinity = true 2026-02-20T11:24:46.209 INFO:teuthology.orchestra.run.trial038.stdout: mon osd allow pg remap = true 2026-02-20T11:24:46.209 INFO:teuthology.orchestra.run.trial038.stdout: mon warn on legacy crush tunables = false 2026-02-20T11:24:46.209 INFO:teuthology.orchestra.run.trial038.stdout: mon warn on crush straw calc version zero = false 2026-02-20T11:24:46.210 INFO:teuthology.orchestra.run.trial038.stdout: mon warn on no sortbitwise = false 2026-02-20T11:24:46.210 INFO:teuthology.orchestra.run.trial038.stdout: mon warn on osd down out interval zero = false 2026-02-20T11:24:46.210 INFO:teuthology.orchestra.run.trial038.stdout: mon warn on too few osds = false 2026-02-20T11:24:46.210 INFO:teuthology.orchestra.run.trial038.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-20T11:24:46.210 INFO:teuthology.orchestra.run.trial038.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-20T11:24:46.210 INFO:teuthology.orchestra.run.trial038.stdout: mon_allow_pool_size_one = true 2026-02-20T11:24:46.210 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.210 INFO:teuthology.orchestra.run.trial038.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-20T11:24:46.210 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.210 INFO:teuthology.orchestra.run.trial038.stdout: osd default data pool replay window = 5 2026-02-20T11:24:46.210 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.210 INFO:teuthology.orchestra.run.trial038.stdout: mon allow pool delete = true 2026-02-20T11:24:46.211 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.211 INFO:teuthology.orchestra.run.trial038.stdout: mon cluster log file level = debug 2026-02-20T11:24:46.211 INFO:teuthology.orchestra.run.trial038.stdout: debug asserts on shutdown = true 2026-02-20T11:24:46.211 INFO:teuthology.orchestra.run.trial038.stdout: mon health detail to clog = false 2026-02-20T11:24:46.211 INFO:teuthology.orchestra.run.trial038.stdout: mon host = 10.20.193.38 2026-02-20T11:24:46.211 INFO:teuthology.orchestra.run.trial038.stdout: auth client required = none 2026-02-20T11:24:46.211 INFO:teuthology.orchestra.run.trial038.stdout: auth cluster required = none 2026-02-20T11:24:46.211 INFO:teuthology.orchestra.run.trial038.stdout: auth service required = none 2026-02-20T11:24:46.211 INFO:teuthology.orchestra.run.trial038.stdout: debug asok = 0/0 2026-02-20T11:24:46.211 INFO:teuthology.orchestra.run.trial038.stdout: debug auth = 0/0 2026-02-20T11:24:46.211 INFO:teuthology.orchestra.run.trial038.stdout: debug bdev = 0/0 2026-02-20T11:24:46.212 INFO:teuthology.orchestra.run.trial038.stdout: debug bluefs = 0/0 2026-02-20T11:24:46.212 INFO:teuthology.orchestra.run.trial038.stdout: debug bluestore = 0/0 2026-02-20T11:24:46.212 INFO:teuthology.orchestra.run.trial038.stdout: debug buffer = 0/0 2026-02-20T11:24:46.212 INFO:teuthology.orchestra.run.trial038.stdout: debug civetweb = 0/0 2026-02-20T11:24:46.212 INFO:teuthology.orchestra.run.trial038.stdout: debug client = 0/0 2026-02-20T11:24:46.212 INFO:teuthology.orchestra.run.trial038.stdout: debug compressor = 0/0 2026-02-20T11:24:46.212 INFO:teuthology.orchestra.run.trial038.stdout: debug context = 0/0 2026-02-20T11:24:46.212 INFO:teuthology.orchestra.run.trial038.stdout: debug crush = 0/0 2026-02-20T11:24:46.212 INFO:teuthology.orchestra.run.trial038.stdout: debug crypto = 0/0 2026-02-20T11:24:46.212 INFO:teuthology.orchestra.run.trial038.stdout: debug dpdk = 0/0 2026-02-20T11:24:46.212 INFO:teuthology.orchestra.run.trial038.stdout: debug eventtrace = 0/0 2026-02-20T11:24:46.212 INFO:teuthology.orchestra.run.trial038.stdout: debug filer = 0/0 2026-02-20T11:24:46.212 INFO:teuthology.orchestra.run.trial038.stdout: debug filestore = 0/0 2026-02-20T11:24:46.213 INFO:teuthology.orchestra.run.trial038.stdout: debug finisher = 0/0 2026-02-20T11:24:46.213 INFO:teuthology.orchestra.run.trial038.stdout: debug fuse = 0/0 2026-02-20T11:24:46.213 INFO:teuthology.orchestra.run.trial038.stdout: debug heartbeatmap = 0/0 2026-02-20T11:24:46.213 INFO:teuthology.orchestra.run.trial038.stdout: debug javaclient = 0/0 2026-02-20T11:24:46.213 INFO:teuthology.orchestra.run.trial038.stdout: debug journal = 0/0 2026-02-20T11:24:46.213 INFO:teuthology.orchestra.run.trial038.stdout: debug journaler = 0/0 2026-02-20T11:24:46.213 INFO:teuthology.orchestra.run.trial038.stdout: debug kstore = 0/0 2026-02-20T11:24:46.213 INFO:teuthology.orchestra.run.trial038.stdout: debug leveldb = 0/0 2026-02-20T11:24:46.213 INFO:teuthology.orchestra.run.trial038.stdout: debug lockdep = 0/0 2026-02-20T11:24:46.213 INFO:teuthology.orchestra.run.trial038.stdout: debug mds = 0/0 2026-02-20T11:24:46.213 INFO:teuthology.orchestra.run.trial038.stdout: debug mds balancer = 0/0 2026-02-20T11:24:46.213 INFO:teuthology.orchestra.run.trial038.stdout: debug mds locker = 0/0 2026-02-20T11:24:46.214 INFO:teuthology.orchestra.run.trial038.stdout: debug mds log = 0/0 2026-02-20T11:24:46.214 INFO:teuthology.orchestra.run.trial038.stdout: debug mds log expire = 0/0 2026-02-20T11:24:46.214 INFO:teuthology.orchestra.run.trial038.stdout: debug mds migrator = 0/0 2026-02-20T11:24:46.214 INFO:teuthology.orchestra.run.trial038.stdout: debug memdb = 0/0 2026-02-20T11:24:46.214 INFO:teuthology.orchestra.run.trial038.stdout: debug mgr = 0/0 2026-02-20T11:24:46.214 INFO:teuthology.orchestra.run.trial038.stdout: debug mgrc = 0/0 2026-02-20T11:24:46.214 INFO:teuthology.orchestra.run.trial038.stdout: debug mon = 0/0 2026-02-20T11:24:46.214 INFO:teuthology.orchestra.run.trial038.stdout: debug monc = 0/0 2026-02-20T11:24:46.214 INFO:teuthology.orchestra.run.trial038.stdout: debug ms = 0/0 2026-02-20T11:24:46.214 INFO:teuthology.orchestra.run.trial038.stdout: debug objclass = 0/0 2026-02-20T11:24:46.214 INFO:teuthology.orchestra.run.trial038.stdout: debug objectcacher = 0/0 2026-02-20T11:24:46.214 INFO:teuthology.orchestra.run.trial038.stdout: debug objecter = 0/0 2026-02-20T11:24:46.214 INFO:teuthology.orchestra.run.trial038.stdout: debug optracker = 0/0 2026-02-20T11:24:46.215 INFO:teuthology.orchestra.run.trial038.stdout: debug osd = 0/0 2026-02-20T11:24:46.215 INFO:teuthology.orchestra.run.trial038.stdout: debug paxos = 0/0 2026-02-20T11:24:46.215 INFO:teuthology.orchestra.run.trial038.stdout: debug perfcounter = 0/0 2026-02-20T11:24:46.215 INFO:teuthology.orchestra.run.trial038.stdout: debug rados = 0/0 2026-02-20T11:24:46.215 INFO:teuthology.orchestra.run.trial038.stdout: debug rbd = 0/0 2026-02-20T11:24:46.215 INFO:teuthology.orchestra.run.trial038.stdout: debug rbd mirror = 0/0 2026-02-20T11:24:46.215 INFO:teuthology.orchestra.run.trial038.stdout: debug rbd replay = 0/0 2026-02-20T11:24:46.215 INFO:teuthology.orchestra.run.trial038.stdout: debug refs = 0/0 2026-02-20T11:24:46.215 INFO:teuthology.orchestra.run.trial038.stdout: debug rgw = 0/0 2026-02-20T11:24:46.215 INFO:teuthology.orchestra.run.trial038.stdout: debug rgw lifecycle = 0/0 2026-02-20T11:24:46.215 INFO:teuthology.orchestra.run.trial038.stdout: debug rgw notification = 0/0 2026-02-20T11:24:46.215 INFO:teuthology.orchestra.run.trial038.stdout: debug rgw sync = 0/0 2026-02-20T11:24:46.215 INFO:teuthology.orchestra.run.trial038.stdout: debug rocksdb = 0/0 2026-02-20T11:24:46.216 INFO:teuthology.orchestra.run.trial038.stdout: debug striper = 0/0 2026-02-20T11:24:46.216 INFO:teuthology.orchestra.run.trial038.stdout: debug throttle = 0/0 2026-02-20T11:24:46.216 INFO:teuthology.orchestra.run.trial038.stdout: debug timer = 0/0 2026-02-20T11:24:46.216 INFO:teuthology.orchestra.run.trial038.stdout: debug tp = 0/0 2026-02-20T11:24:46.216 INFO:teuthology.orchestra.run.trial038.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-20T11:24:46.216 INFO:teuthology.orchestra.run.trial038.stdout: ms client mode = crc 2026-02-20T11:24:46.216 INFO:teuthology.orchestra.run.trial038.stdout: ms cluster mode = crc 2026-02-20T11:24:46.216 INFO:teuthology.orchestra.run.trial038.stdout: ms mon client mode = crc 2026-02-20T11:24:46.216 INFO:teuthology.orchestra.run.trial038.stdout: ms mon cluster mode = crc 2026-02-20T11:24:46.216 INFO:teuthology.orchestra.run.trial038.stdout: ms mon service mode = crc 2026-02-20T11:24:46.216 INFO:teuthology.orchestra.run.trial038.stdout: ms service mode = crc 2026-02-20T11:24:46.216 INFO:teuthology.orchestra.run.trial038.stdout: fsid = 97ccc1d9-5f9e-4a23-a83d-fcf45d35bda8 2026-02-20T11:24:46.217 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.217 INFO:teuthology.orchestra.run.trial038.stdout:[osd] 2026-02-20T11:24:46.217 INFO:teuthology.orchestra.run.trial038.stdout: osd journal size = 100 2026-02-20T11:24:46.217 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.217 INFO:teuthology.orchestra.run.trial038.stdout: osd scrub load threshold = 5.0 2026-02-20T11:24:46.217 INFO:teuthology.orchestra.run.trial038.stdout: osd scrub max interval = 600 2026-02-20T11:24:46.217 INFO:teuthology.orchestra.run.trial038.stdout: osd mclock profile = high_recovery_ops 2026-02-20T11:24:46.217 INFO:teuthology.orchestra.run.trial038.stdout: osd mclock skip benchmark = true 2026-02-20T11:24:46.217 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.217 INFO:teuthology.orchestra.run.trial038.stdout: osd recover clone overlap = true 2026-02-20T11:24:46.217 INFO:teuthology.orchestra.run.trial038.stdout: osd recovery max chunk = 1048576 2026-02-20T11:24:46.217 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.218 INFO:teuthology.orchestra.run.trial038.stdout: osd debug shutdown = true 2026-02-20T11:24:46.218 INFO:teuthology.orchestra.run.trial038.stdout: osd debug op order = true 2026-02-20T11:24:46.218 INFO:teuthology.orchestra.run.trial038.stdout: osd debug verify stray on activate = true 2026-02-20T11:24:46.218 INFO:teuthology.orchestra.run.trial038.stdout: osd debug trim objects = true 2026-02-20T11:24:46.218 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.218 INFO:teuthology.orchestra.run.trial038.stdout: osd open classes on start = true 2026-02-20T11:24:46.218 INFO:teuthology.orchestra.run.trial038.stdout: osd debug pg log writeout = true 2026-02-20T11:24:46.218 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.218 INFO:teuthology.orchestra.run.trial038.stdout: osd deep scrub update digest min age = 30 2026-02-20T11:24:46.218 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.218 INFO:teuthology.orchestra.run.trial038.stdout: osd map max advance = 10 2026-02-20T11:24:46.218 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.219 INFO:teuthology.orchestra.run.trial038.stdout: journal zero on create = true 2026-02-20T11:24:46.219 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.219 INFO:teuthology.orchestra.run.trial038.stdout: filestore ondisk finisher threads = 3 2026-02-20T11:24:46.219 INFO:teuthology.orchestra.run.trial038.stdout: filestore apply finisher threads = 3 2026-02-20T11:24:46.219 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.219 INFO:teuthology.orchestra.run.trial038.stdout: bdev debug aio = true 2026-02-20T11:24:46.219 INFO:teuthology.orchestra.run.trial038.stdout: osd debug misdirected ops = true 2026-02-20T11:24:46.219 INFO:teuthology.orchestra.run.trial038.stdout: debug monc = 20 2026-02-20T11:24:46.219 INFO:teuthology.orchestra.run.trial038.stdout: crimson cpu num = 4 2026-02-20T11:24:46.219 INFO:teuthology.orchestra.run.trial038.stdout: crimson osd obc lru size = 10 2026-02-20T11:24:46.219 INFO:teuthology.orchestra.run.trial038.stdout: debug filestore = 0/0 2026-02-20T11:24:46.219 INFO:teuthology.orchestra.run.trial038.stdout: debug journal = 0/0 2026-02-20T11:24:46.220 INFO:teuthology.orchestra.run.trial038.stdout: debug ms = 0/0 2026-02-20T11:24:46.220 INFO:teuthology.orchestra.run.trial038.stdout: debug osd = 0/0 2026-02-20T11:24:46.220 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore = 20 2026-02-20T11:24:46.220 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore backref = 20 2026-02-20T11:24:46.220 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore cache = 20 2026-02-20T11:24:46.220 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore cleaner = 20 2026-02-20T11:24:46.220 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore device = 20 2026-02-20T11:24:46.220 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore epm = 20 2026-02-20T11:24:46.220 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore fixedkv tree = 20 2026-02-20T11:24:46.220 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore journal = 20 2026-02-20T11:24:46.220 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore lba = 20 2026-02-20T11:24:46.221 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore odata = 20 2026-02-20T11:24:46.221 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore omap = 20 2026-02-20T11:24:46.221 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore onode = 20 2026-02-20T11:24:46.221 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore t = 20 2026-02-20T11:24:46.221 INFO:teuthology.orchestra.run.trial038.stdout: debug seastore tm = 20 2026-02-20T11:24:46.221 INFO:teuthology.orchestra.run.trial038.stdout: osd objectstore = seastore 2026-02-20T11:24:46.221 INFO:teuthology.orchestra.run.trial038.stdout: osd shutdown pgref assert = True 2026-02-20T11:24:46.221 INFO:teuthology.orchestra.run.trial038.stdout: seastore cachepin size pershard = 64M 2026-02-20T11:24:46.221 INFO:teuthology.orchestra.run.trial038.stdout: seastore cachepin type = 2Q 2026-02-20T11:24:46.221 INFO:teuthology.orchestra.run.trial038.stdout: seastore max concurrent transactions = 8 2026-02-20T11:24:46.221 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.221 INFO:teuthology.orchestra.run.trial038.stdout:[mgr] 2026-02-20T11:24:46.222 INFO:teuthology.orchestra.run.trial038.stdout: debug ms = 1 2026-02-20T11:24:46.222 INFO:teuthology.orchestra.run.trial038.stdout: debug mgr = 20 2026-02-20T11:24:46.222 INFO:teuthology.orchestra.run.trial038.stdout: debug mon = 20 2026-02-20T11:24:46.222 INFO:teuthology.orchestra.run.trial038.stdout: debug auth = 20 2026-02-20T11:24:46.222 INFO:teuthology.orchestra.run.trial038.stdout: mon reweight min pgs per osd = 4 2026-02-20T11:24:46.222 INFO:teuthology.orchestra.run.trial038.stdout: mon reweight min bytes per osd = 10 2026-02-20T11:24:46.222 INFO:teuthology.orchestra.run.trial038.stdout: mgr/telemetry/nag = false 2026-02-20T11:24:46.222 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.222 INFO:teuthology.orchestra.run.trial038.stdout:[mon] 2026-02-20T11:24:46.222 INFO:teuthology.orchestra.run.trial038.stdout: debug ms = 0/0 2026-02-20T11:24:46.222 INFO:teuthology.orchestra.run.trial038.stdout: debug mon = 0/0 2026-02-20T11:24:46.222 INFO:teuthology.orchestra.run.trial038.stdout: debug paxos = 0/0 2026-02-20T11:24:46.223 INFO:teuthology.orchestra.run.trial038.stdout: debug auth = 20 2026-02-20T11:24:46.223 INFO:teuthology.orchestra.run.trial038.stdout: mon data avail warn = 5 2026-02-20T11:24:46.223 INFO:teuthology.orchestra.run.trial038.stdout: mon mgr mkfs grace = 240 2026-02-20T11:24:46.223 INFO:teuthology.orchestra.run.trial038.stdout: mon reweight min pgs per osd = 4 2026-02-20T11:24:46.223 INFO:teuthology.orchestra.run.trial038.stdout: mon osd reporter subtree level = osd 2026-02-20T11:24:46.223 INFO:teuthology.orchestra.run.trial038.stdout: mon osd prime pg temp = true 2026-02-20T11:24:46.223 INFO:teuthology.orchestra.run.trial038.stdout: mon reweight min bytes per osd = 10 2026-02-20T11:24:46.223 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.223 INFO:teuthology.orchestra.run.trial038.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-20T11:24:46.223 INFO:teuthology.orchestra.run.trial038.stdout: auth mon ticket ttl = 660 # 11m 2026-02-20T11:24:46.223 INFO:teuthology.orchestra.run.trial038.stdout: auth service ticket ttl = 240 # 4m 2026-02-20T11:24:46.223 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.224 INFO:teuthology.orchestra.run.trial038.stdout: # don't complain about insecure global_id in the test suite 2026-02-20T11:24:46.224 INFO:teuthology.orchestra.run.trial038.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-20T11:24:46.224 INFO:teuthology.orchestra.run.trial038.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-20T11:24:46.224 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.224 INFO:teuthology.orchestra.run.trial038.stdout: # 1m isn't quite enough 2026-02-20T11:24:46.224 INFO:teuthology.orchestra.run.trial038.stdout: mon_down_mkfs_grace = 2m 2026-02-20T11:24:46.224 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.224 INFO:teuthology.orchestra.run.trial038.stdout: mon_warn_on_filestore_osds = false 2026-02-20T11:24:46.224 INFO:teuthology.orchestra.run.trial038.stdout: osd pool default crimson = True 2026-02-20T11:24:46.224 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:24:46.224 INFO:teuthology.orchestra.run.trial038.stdout:[client] 2026-02-20T11:24:46.224 INFO:teuthology.orchestra.run.trial038.stdout: rgw cache enabled = true 2026-02-20T11:24:46.225 INFO:teuthology.orchestra.run.trial038.stdout: rgw enable ops log = true 2026-02-20T11:24:46.225 INFO:teuthology.orchestra.run.trial038.stdout: rgw enable usage log = true 2026-02-20T11:24:46.225 INFO:teuthology.orchestra.run.trial038.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-20T11:24:46.225 INFO:teuthology.orchestra.run.trial038.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-20T11:24:46.225 INFO:teuthology.orchestra.run.trial038.stdout:[mon.a] 2026-02-20T11:24:46.226 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-20T11:24:46.227 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-20T11:24:46.318 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-20T11:24:46.318 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:24:46.318 DEBUG:teuthology.orchestra.run.trial038:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-20T11:24:46.335 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:24:46.335 DEBUG:teuthology.orchestra.run.trial038:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-20T11:24:46.391 INFO:tasks.ceph:Sending monmap to node ubuntu@trial038.front.sepia.ceph.com 2026-02-20T11:24:46.391 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:24:46.391 DEBUG:teuthology.orchestra.run.trial038:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-20T11:24:46.391 DEBUG:teuthology.orchestra.run.trial038:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-20T11:24:46.471 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:24:46.472 DEBUG:teuthology.orchestra.run.trial038:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:24:46.529 INFO:tasks.ceph:Setting up mon nodes... 2026-02-20T11:24:46.529 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-20T11:24:46.530 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-20T11:24:46.620 INFO:teuthology.orchestra.run.trial038.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-20T11:24:46.622 INFO:tasks.ceph:Setting up mds nodes... 2026-02-20T11:24:46.622 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-20T11:24:46.623 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-20T11:24:46.664 INFO:teuthology.orchestra.run.trial038.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-20T11:24:46.676 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-20T11:24:46.676 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial038.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-20T11:24:46.676 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-20T11:24:46.742 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T11:24:46.742 INFO:tasks.ceph:role: osd.0 2026-02-20T11:24:46.743 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial038.front.sepia.ceph.com 2026-02-20T11:24:46.743 DEBUG:teuthology.orchestra.run.trial038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-20T11:24:46.806 INFO:teuthology.orchestra.run.trial038.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T11:24:46.806 INFO:teuthology.orchestra.run.trial038.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T11:24:46.806 INFO:teuthology.orchestra.run.trial038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T11:24:46.806 INFO:teuthology.orchestra.run.trial038.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T11:24:46.806 INFO:teuthology.orchestra.run.trial038.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T11:24:46.806 INFO:teuthology.orchestra.run.trial038.stdout: = sunit=0 swidth=0 blks 2026-02-20T11:24:46.806 INFO:teuthology.orchestra.run.trial038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T11:24:46.806 INFO:teuthology.orchestra.run.trial038.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T11:24:46.806 INFO:teuthology.orchestra.run.trial038.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T11:24:46.806 INFO:teuthology.orchestra.run.trial038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T11:24:47.406 INFO:teuthology.orchestra.run.trial038.stdout:Discarding blocks...Done. 2026-02-20T11:24:47.408 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial038.front.sepia.ceph.com -o noatime 2026-02-20T11:24:47.408 DEBUG:teuthology.orchestra.run.trial038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-20T11:24:47.477 DEBUG:teuthology.orchestra.run.trial038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-20T11:24:47.543 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-20T11:24:47.608 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T11:24:47.608 INFO:tasks.ceph:role: osd.1 2026-02-20T11:24:47.608 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial038.front.sepia.ceph.com 2026-02-20T11:24:47.608 DEBUG:teuthology.orchestra.run.trial038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-20T11:24:47.673 INFO:teuthology.orchestra.run.trial038.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T11:24:47.673 INFO:teuthology.orchestra.run.trial038.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T11:24:47.673 INFO:teuthology.orchestra.run.trial038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T11:24:47.673 INFO:teuthology.orchestra.run.trial038.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T11:24:47.673 INFO:teuthology.orchestra.run.trial038.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T11:24:47.673 INFO:teuthology.orchestra.run.trial038.stdout: = sunit=0 swidth=0 blks 2026-02-20T11:24:47.673 INFO:teuthology.orchestra.run.trial038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T11:24:47.673 INFO:teuthology.orchestra.run.trial038.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T11:24:47.673 INFO:teuthology.orchestra.run.trial038.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T11:24:47.674 INFO:teuthology.orchestra.run.trial038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T11:24:48.230 INFO:teuthology.orchestra.run.trial038.stdout:Discarding blocks...Done. 2026-02-20T11:24:48.231 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial038.front.sepia.ceph.com -o noatime 2026-02-20T11:24:48.231 DEBUG:teuthology.orchestra.run.trial038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-20T11:24:48.299 DEBUG:teuthology.orchestra.run.trial038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-20T11:24:48.365 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-20T11:24:48.428 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-20T11:24:48.428 INFO:tasks.ceph:role: osd.2 2026-02-20T11:24:48.428 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial038.front.sepia.ceph.com 2026-02-20T11:24:48.428 DEBUG:teuthology.orchestra.run.trial038:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-20T11:24:48.493 INFO:teuthology.orchestra.run.trial038.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-20T11:24:48.493 INFO:teuthology.orchestra.run.trial038.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-20T11:24:48.493 INFO:teuthology.orchestra.run.trial038.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-20T11:24:48.493 INFO:teuthology.orchestra.run.trial038.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-20T11:24:48.493 INFO:teuthology.orchestra.run.trial038.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-20T11:24:48.494 INFO:teuthology.orchestra.run.trial038.stdout: = sunit=0 swidth=0 blks 2026-02-20T11:24:48.494 INFO:teuthology.orchestra.run.trial038.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-20T11:24:48.494 INFO:teuthology.orchestra.run.trial038.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-20T11:24:48.494 INFO:teuthology.orchestra.run.trial038.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-20T11:24:48.494 INFO:teuthology.orchestra.run.trial038.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-20T11:24:49.093 INFO:teuthology.orchestra.run.trial038.stdout:Discarding blocks...Done. 2026-02-20T11:24:49.095 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial038.front.sepia.ceph.com -o noatime 2026-02-20T11:24:49.095 DEBUG:teuthology.orchestra.run.trial038:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-20T11:24:49.163 DEBUG:teuthology.orchestra.run.trial038:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-20T11:24:49.230 DEBUG:teuthology.orchestra.run.trial038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:24:49.381 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T11:24:49.435 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:49,434 seastar - Reactor backend: linux-aio 2026-02-20T11:24:49.445 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:49,444 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:24:49.445 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:49,444 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:24:49.448 INFO:teuthology.orchestra.run.trial038.stderr:==39294==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:24:49.507 INFO:teuthology.orchestra.run.trial038.stderr:DEBUG 2026-02-20 11:24:49,506 [shard 0:main] seastar - reactor::drain 2026-02-20T11:24:51.560 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-20T11:24:51.560 INFO:teuthology.orchestra.run.trial038.stderr:Suppressions used: 2026-02-20T11:24:51.560 INFO:teuthology.orchestra.run.trial038.stderr: count bytes template 2026-02-20T11:24:51.560 INFO:teuthology.orchestra.run.trial038.stderr: 1 8 MallocExtension::Register 2026-02-20T11:24:51.560 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-20T11:24:51.560 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-20T11:24:51.580 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T11:24:51.581 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T11:24:51.625 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T11:24:51.625 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T11:24:51.632 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:51,631 seastar - Reactor backend: linux-aio 2026-02-20T11:24:51.644 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:51,643 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:24:51.644 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:51,643 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:24:51.647 INFO:teuthology.orchestra.run.trial038.stderr:==39292==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:24:51.647 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:51,646 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T11:24:51.790 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1586f 0x10a3723 0xf4865a 0x9e1e2c5 0x9e1e78f 0x9e1ea02 0x9e1ecb8 0x9e1f27a 0x9e27721 0x9e27e01 0x9f17e54 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:51.790 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x9d70660 0x9d70939 0x9d70a2c 0x9d70b08 0x9d70d84 0x9d7132b 0x9d7172f 0x9d719ec 0x9f0638f 0x9f0656d 0x9f067c9 0x9f068ba 0x9f0745a 0x9f07a5e 0x9f07df1 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:51.791 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9ef014c 0x9ef025d 0x7b9c6c3 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:51.791 INFO:teuthology.orchestra.run.trial038.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff8ad27ef7 0xffffffff8a9f7f16 0xffffffff8a9fb8d0 0xffffffff8a9fcc31 0xffffffff8aa24893 0xffffffff8aa24a55 0xffffffff8a9c97c8 0xffffffff8a9d3aeb 0xffffffff8a9d3f4b 0xffffffff8a6910d2 0xffffffff8b34d451 0xffffffff8b400ba2 2026-02-20T11:24:51.823 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x9e1e634 0x9e1ea02 0x9e1ecb8 0x9e1f767 0x9e1fc65 0x9e20295 0x9e203ff 0x9f07c75 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7b8bc1a 0x7ba0473 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:24:51.866 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0xa2ebdb4 0xa2ed8df 0x9d7a2c4 0x9d7a7cd 0x9d7a8f1 0x9d7a99f 0x9d7aa4d 0x9d91b74 0x9d91c56 0x9da7c9e 0x9da7e71 0xcf9dd4 0x9d557fb 0x9d7d1a3 0x9d7e33d 0x9d93d1b 0x9d949f2 0x9d9199d 0x783a75a 0x78e5228 0x78ea3aa 0x95fdc26 0x9643312 0x7fb16bf 0x7fb1c5d 0x7fb1fb5 0x7fb25de 0x7fb4951 0x911dcdf 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:51.892 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:24:51.928 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:51.931 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x297f494 0x29d89ca 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:51.931 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x297f494 0x29d89ca 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:52.054 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x9f178f0 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:52.054 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x157c7 0x10a3723 0xf4865a 0x9e1e2c5 0x9e1e78f 0x9e1ea02 0x9e1ecb8 0x9e1f767 0x9e1fc65 0x9e20295 0x9e203ff 0x9f07c75 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:52.054 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc230 /lib64/libasan.so.8+0xc553 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe419a1 0x9d704b4 0x9d705f8 0x9d706dd 0x9d70939 0x9d70a2c 0x9d70b08 0x9d70d84 0x9d7132b 0x9d7172f 0x9d719ec 0x9d72834 0x9ef0163 0x9ef025d 0x7b9c0be 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:52.120 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1294d2 /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:52.136 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x157b4 0x29e23b0 0x29e25fc 0x7f05599 0x7f05842 0x7f0596f 0x7f05a3a 0x7f05bf3 0x7f05d34 0x7e70a53 0x7ee5e88 0x7e59ad6 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:24:52.152 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d7d20 0x7d69b1d 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:52.152 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d885f 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:54.268 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-20T11:24:54.268 INFO:teuthology.orchestra.run.trial038.stderr:Suppressions used: 2026-02-20T11:24:54.268 INFO:teuthology.orchestra.run.trial038.stderr: count bytes template 2026-02-20T11:24:54.268 INFO:teuthology.orchestra.run.trial038.stderr: 1 8 MallocExtension::Register 2026-02-20T11:24:54.268 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-20T11:24:54.268 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-20T11:24:54.303 DEBUG:teuthology.orchestra.run.trial038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-20T11:24:54.328 DEBUG:teuthology.orchestra.run.trial038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:24:54.479 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T11:24:54.535 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:54,534 seastar - Reactor backend: linux-aio 2026-02-20T11:24:54.547 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:54,546 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:24:54.547 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:54,546 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:24:54.549 INFO:teuthology.orchestra.run.trial038.stderr:==39362==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:24:54.609 INFO:teuthology.orchestra.run.trial038.stderr:DEBUG 2026-02-20 11:24:54,608 [shard 0:main] seastar - reactor::drain 2026-02-20T11:24:56.666 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-20T11:24:56.666 INFO:teuthology.orchestra.run.trial038.stderr:Suppressions used: 2026-02-20T11:24:56.666 INFO:teuthology.orchestra.run.trial038.stderr: count bytes template 2026-02-20T11:24:56.666 INFO:teuthology.orchestra.run.trial038.stderr: 1 8 MallocExtension::Register 2026-02-20T11:24:56.666 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-20T11:24:56.666 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-20T11:24:56.687 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T11:24:56.688 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T11:24:56.729 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T11:24:56.729 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T11:24:56.736 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:56,734 seastar - Reactor backend: linux-aio 2026-02-20T11:24:56.750 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:56,748 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:24:56.750 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:56,749 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:24:56.754 INFO:teuthology.orchestra.run.trial038.stderr:==39360==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:24:56.754 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:56,753 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T11:24:56.901 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xfcaf3 /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x157c7 0x10a3723 0xf4865a 0x9e1e492 0x9e1e78f 0x9e1ea02 0x9e1fa02 0x9e27a85 0x9e27e01 0x9f17e54 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:56.901 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x31f9b8a 0x9e1e750 0x9e1ea02 0x9e1ecb8 0x9e1f27a 0x9e27721 0x9e27e01 0x9f17e54 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:56.901 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x16cc6 0x9f0e754 0x9f13918 0x9f13f56 0x9f1e423 0x78bcf19 0x7b9c686 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:56.933 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x129793 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15e37 0x9e1e650 0x9e1ea02 0x9e1ecb8 0x9e1f767 0x9e1fc65 0x9e20295 0x9e203ff 0x9f07c75 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7f67185 0x7fb45b4 0x911dcdf 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:24:56.977 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 66 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x7c8f295 0x7c8f3c0 0x7cac097 0x7b9c32b 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:56.999 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d79b2 0x7d69b08 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:24:57.040 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x297f494 0x29d89ca 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:57.041 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0f /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:57.048 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x3423 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:57.171 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65b1 /lib64/libasan.so.8+0x157c7 0x10a3723 0xf4865a 0x9e22cec 0x9e22f99 0x9f17991 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:57.171 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1294d2 /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x157c7 0x3549a64 0x3549be4 0x359d2e4 0x9e30705 0x9e307e8 0x9ef7d44 0x9ef7ed9 0x9ef826b 0x9ef9851 0x9efab6c 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:57.171 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15835 0x3549a64 0x3549be4 0x359d2e4 0x9e30705 0x9e307e8 0x9ef7d44 0x9ef7ed9 0x9ef826b 0x9ef9851 0x9efab6c 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:57.254 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:24:57.269 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d7d7b 0x7d69b1d 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:57.270 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d8821 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:57.270 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cbf3 /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:24:59.455 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-20T11:24:59.455 INFO:teuthology.orchestra.run.trial038.stderr:Suppressions used: 2026-02-20T11:24:59.455 INFO:teuthology.orchestra.run.trial038.stderr: count bytes template 2026-02-20T11:24:59.455 INFO:teuthology.orchestra.run.trial038.stderr: 1 8 MallocExtension::Register 2026-02-20T11:24:59.455 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-20T11:24:59.456 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-20T11:24:59.492 DEBUG:teuthology.orchestra.run.trial038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-20T11:24:59.516 DEBUG:teuthology.orchestra.run.trial038:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:24:59.670 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: parsing early config 2026-02-20T11:24:59.724 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:59,723 seastar - Reactor backend: linux-aio 2026-02-20T11:24:59.734 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:59,732 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:24:59.734 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:24:59,732 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:24:59.736 INFO:teuthology.orchestra.run.trial038.stderr:==39426==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:24:59.796 INFO:teuthology.orchestra.run.trial038.stderr:DEBUG 2026-02-20 11:24:59,795 [shard 0:main] seastar - reactor::drain 2026-02-20T11:25:01.852 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-20T11:25:01.853 INFO:teuthology.orchestra.run.trial038.stderr:Suppressions used: 2026-02-20T11:25:01.853 INFO:teuthology.orchestra.run.trial038.stderr: count bytes template 2026-02-20T11:25:01.853 INFO:teuthology.orchestra.run.trial038.stderr: 1 8 MallocExtension::Register 2026-02-20T11:25:01.853 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-20T11:25:01.853 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-20T11:25:01.873 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-20T11:25:01.873 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-20T11:25:01.916 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-20T11:25:01.916 INFO:teuthology.orchestra.run.trial038.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-20T11:25:01.923 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:25:01,922 seastar - Reactor backend: linux-aio 2026-02-20T11:25:01.935 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:25:01,934 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:25:01.935 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:25:01,934 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-20T11:25:01.939 INFO:teuthology.orchestra.run.trial038.stderr:==39424==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-20T11:25:01.939 INFO:teuthology.orchestra.run.trial038.stderr:INFO 2026-02-20 11:25:01,938 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-20T11:25:02.083 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x9dde3d9 0x9f0a2da 0x9f0e9f4 0x9f13918 0x9f13f56 0x9f1e423 0x78bcf19 0x7b9c686 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:25:02.084 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x16b36a /lib64/libasan.so.8+0xb0541 /lib64/libasan.so.8+0xb1437 0xb7b689 0x10a36ac 0xf4865a 0x9e1e492 0x9e1e78f 0x9e1ea02 0x9e20729 0x9e20c76 0x9e20e75 0x9f0d69d 0x9f0ddf8 0x9f18086 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:25:02.084 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x129747 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x157c7 0x10a3723 0xf4865a 0x9e1e2c5 0x9e1e78f 0x9e1ea02 0x9e20729 0x9f06dfa 0x9f07573 0x9f07a5e 0x9f07df1 0x9f1aa78 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f75f2c 0x6f79bf7 0x6f7f709 0x73d6e81 0x73d6f06 0x73d7008 0x73d7100 0x74a009e 0x7246aa6 0x727c254 0x73c1550 0x7413540 0x7246b2d 0x727c45b 0x756db47 0x73d7185 0x73d7263 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:25:02.116 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc962 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0xbcac15 0xbcae1e 0xe76fa5 /lib64/libstdc++.so.6+0x140b82 /lib64/libstdc++.so.6+0x149689 /lib64/libstdc++.so.6+0x13b003 /lib64/libstdc++.so.6+0x13b3cb 0x76f1f6e 0x7ba081f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:25:02.158 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1006a8 /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x30f33e2 0x4cb11cb 0x9efa8f3 0x9efaa62 0x9efaf8c 0x9f17817 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7f67185 0x7fb45b4 0x911dcdf 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:25:02.159 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x10a35bf 0xf4865a 0x9e22cec 0x9e2362b 0x9f16b01 0x9f19afa 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7f67185 0x7fb45b4 0x911dcdf 0x912064d 0x7862725 0x7a52d48 0x6f770d1 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:25:02.159 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xf0f3e /lib64/libasan.so.8+0xe908 /lib64/libasan.so.8+0xec776 0xcc3105 0x78bcf7b 0x7b9c686 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f76cf2 0x7518264 0x75183b8 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:25:02.183 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:25:02.224 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x129764 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:25:02.225 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297db /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:25:02.230 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x29d885f 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:25:02.351 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1580a 0x9e22a50 0x9e22f99 0x9f17991 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:25:02.351 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0xcc321b 0x10a3600 0xf4865a 0x9e1e2c5 0x9e1e78f 0x9e1ea02 0x9e20729 0x9e20c76 0x9e20e75 0x9f0d69d 0x9f0ddf8 0x9f18086 0x9f1c4ff 0x9edc6d1 0x9edb6d1 0x7b9c74f 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:25:02.351 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f 0x10a35bf 0xf4865a 0x9e21fb0 0x9e228ef 0x9f13d3d 0x9f1e423 0x78bcf19 0x7b9c686 0x7ba6272 0x79ff8c1 0x7a3f1fd 0x718b8df 0x6f2e256 0x6f30572 0x73d4684 0x73d4709 0x73d480b 0x73d4903 0x749ff22 0x7246045 0x727559c 0x73bfa6b 0x740df81 0x72460cc 0x72757a3 0x756b847 0x73d4988 0x73d4a66 0x73bfbe6 0x74a22b8 0x754287a 0x75428f5 0x756b8c8 0x74a0618 0x75d60f4 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:25:02.433 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cbf3 /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa145278 0x9dc2113 0x9dc294e 0xefe35b /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb58eb4 2026-02-20T11:25:02.449 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1517f 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:25:02.449 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:25:02.450 INFO:teuthology.orchestra.run.trial038.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0xa009912 0xa009bfb 0xa053fd5 0xa0547c8 0xa054a36 0xa054b7e 0xa05503d /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc657f /lib64/libasan.so.8+0x15197 0x29d8a29 0x7d69aef 0x7e5b15a 0x7e5b9c1 0x7e59663 0x78572bb 0x792ffbe 0x794043b 0x794059a 0x7973fa7 0x797959c 0x7979812 0x7979a12 0x7979a8d 0x7979b08 0x7979d54 0x7acf39f 0x7965b04 0x7aa3401 0x73acf1e 0x73b7ed5 0x747550c 0x760f6dc 0xa0a9e81 0xa006b00 0xa0dacb3 0xa143a5a 0xa15e4b8 0xa15eb34 0x277dec1 0xa780984 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-20T11:25:04.633 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-20T11:25:04.633 INFO:teuthology.orchestra.run.trial038.stderr:Suppressions used: 2026-02-20T11:25:04.633 INFO:teuthology.orchestra.run.trial038.stderr: count bytes template 2026-02-20T11:25:04.633 INFO:teuthology.orchestra.run.trial038.stderr: 1 8 MallocExtension::Register 2026-02-20T11:25:04.634 INFO:teuthology.orchestra.run.trial038.stderr:----------------------------------------------------- 2026-02-20T11:25:04.634 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-20T11:25:04.668 DEBUG:teuthology.orchestra.run.trial038:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-20T11:25:04.693 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-20T11:25:04.693 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:25:04.693 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-20T11:25:04.758 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:25:04.758 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-20T11:25:04.823 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:25:04.823 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-20T11:25:04.888 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:25:04.888 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-20T11:25:04.953 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:25:04.953 DEBUG:teuthology.orchestra.run.trial038:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-20T11:25:05.008 INFO:tasks.ceph:Adding keys to all mons... 2026-02-20T11:25:05.008 DEBUG:teuthology.orchestra.run.trial038:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-20T11:25:05.070 INFO:teuthology.orchestra.run.trial038.stdout:[mgr.x] 2026-02-20T11:25:05.070 INFO:teuthology.orchestra.run.trial038.stdout: key = AQB+RJhp0kThJBAADS9Xd7iFzcCV9f2l4Ki4vg== 2026-02-20T11:25:05.070 INFO:teuthology.orchestra.run.trial038.stdout:[osd.0] 2026-02-20T11:25:05.070 INFO:teuthology.orchestra.run.trial038.stdout: key = AQCDRJhpNvdiKhAAFgBiwSPBrlOFIqKzY6iLfA== 2026-02-20T11:25:05.071 INFO:teuthology.orchestra.run.trial038.stdout:[osd.1] 2026-02-20T11:25:05.071 INFO:teuthology.orchestra.run.trial038.stdout: key = AQCIRJhpErgBMRAA6Flef5nzoUBbjHLGKlQQcw== 2026-02-20T11:25:05.071 INFO:teuthology.orchestra.run.trial038.stdout:[osd.2] 2026-02-20T11:25:05.071 INFO:teuthology.orchestra.run.trial038.stdout: key = AQCORJhpSSNRABAAQUU8BYCT3/8V6o1FhsCzRQ== 2026-02-20T11:25:05.071 INFO:teuthology.orchestra.run.trial038.stdout:[client.0] 2026-02-20T11:25:05.071 INFO:teuthology.orchestra.run.trial038.stdout: key = AQB+RJhpVGyEJxAAoDd2odxJaV2rKK7TtvaS2A== 2026-02-20T11:25:05.071 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-20T11:25:05.159 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T11:25:05.206 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T11:25:05.253 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-20T11:25:05.301 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-20T11:25:05.351 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-20T11:25:05.351 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-20T11:25:05.376 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-20T11:25:05.462 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:05.460+0000 7fe451700c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:05.463 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:05.460+0000 7fe451700c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:05.474 DEBUG:teuthology.orchestra.run.trial038:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-20T11:25:05.499 DEBUG:teuthology.orchestra.run.trial038:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-20T11:25:05.555 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-20T11:25:05.555 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-20T11:25:05.556 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-20T11:25:05.597 INFO:tasks.ceph.mon.a:Started 2026-02-20T11:25:05.598 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-20T11:25:05.598 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-20T11:25:05.598 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-20T11:25:05.600 INFO:tasks.ceph.mgr.x:Started 2026-02-20T11:25:05.601 DEBUG:tasks.ceph:set 0 configs 2026-02-20T11:25:05.601 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph config dump 2026-02-20T11:25:05.661 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:25:05.659+0000 7f8482a8ec00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:05.662 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:25:05.659+0000 7f8482a8ec00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:05.662 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:25:05.659+0000 7f8482a8ec00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:05.663 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-20T11:25:05.661+0000 7f4cfc487fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:05.675 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:05.671+0000 7fefeb65a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:05.676 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:05.674+0000 7fefeb65a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:05.707 INFO:teuthology.orchestra.run.trial038.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-20T11:25:05.715 INFO:tasks.ceph:Setting crush tunables to default 2026-02-20T11:25:05.716 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph osd crush tunables default 2026-02-20T11:25:05.787 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:05.782+0000 7f214ab0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:05.788 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:05.786+0000 7f214ab0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:05.821 INFO:teuthology.orchestra.run.trial038.stderr:adjusted tunables profile to default 2026-02-20T11:25:05.829 INFO:tasks.ceph:check_enable_crimson: True 2026-02-20T11:25:05.829 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-20T11:25:05.829 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-20T11:25:05.867 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-20T11:25:05.865+0000 7f4cfc487fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:05.867 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-20T11:25:05.865+0000 7f4cfc487fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:05.903 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:05.899+0000 7f441c55b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:05.904 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:05.902+0000 7f441c55b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:05.943 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:25:05.940+0000 7f847a1f1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-20T11:25:05.943 INFO:tasks.ceph.mon.a.trial038.stderr:Please be aware that this feature is experimental, untested, 2026-02-20T11:25:05.943 INFO:tasks.ceph.mon.a.trial038.stderr:unsupported, and may result in data corruption, data loss, 2026-02-20T11:25:05.943 INFO:tasks.ceph.mon.a.trial038.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-20T11:25:05.943 INFO:tasks.ceph.mon.a.trial038.stderr:feature with important data. 2026-02-20T11:25:05.944 INFO:tasks.ceph.mon.a.trial038.stderr: 2026-02-20T11:25:06.704 INFO:tasks.ceph.mgr.x.trial038.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-20T11:25:06.704 INFO:tasks.ceph.mgr.x.trial038.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-20T11:25:06.704 INFO:tasks.ceph.mgr.x.trial038.stderr: from numpy import show_config as show_numpy_config 2026-02-20T11:25:06.830 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-20T11:25:06.830 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:25:06.831 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-20T11:25:06.896 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:25:06.896 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-20T11:25:06.960 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:25:06.960 DEBUG:teuthology.orchestra.run.trial038:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-20T11:25:07.024 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph osd new 799f2bec-9817-4b9b-bf35-54f155251e89 0 2026-02-20T11:25:07.139 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:07.134+0000 7fc8f3651640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:07.140 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:07.138+0000 7fc8f3651640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:07.177 INFO:teuthology.orchestra.run.trial038.stdout:0 2026-02-20T11:25:07.185 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph osd new c193b783-ad52-4df0-80c8-b5df95abd182 1 2026-02-20T11:25:07.260 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:07.255+0000 7f394d61a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:07.261 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:07.259+0000 7f394d61a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:07.298 INFO:teuthology.orchestra.run.trial038.stdout:1 2026-02-20T11:25:07.307 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph osd new 2dcf13a8-cd3d-4dc4-a01c-ef75162c3483 2 2026-02-20T11:25:07.383 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:07.377+0000 7f2c34374640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:07.384 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:07.382+0000 7f2c34374640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:07.420 INFO:teuthology.orchestra.run.trial038.stdout:2 2026-02-20T11:25:07.429 DEBUG:teuthology.orchestra.run.trial038:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-20T11:25:07.458 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-20T11:25:07.458 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-20T11:25:07.500 INFO:tasks.ceph.osd.0:Started 2026-02-20T11:25:07.500 DEBUG:teuthology.orchestra.run.trial038:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-20T11:25:07.528 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-20T11:25:07.528 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-20T11:25:07.571 INFO:tasks.ceph.osd.1:Started 2026-02-20T11:25:07.571 DEBUG:teuthology.orchestra.run.trial038:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-20T11:25:07.599 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-20T11:25:07.599 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-20T11:25:07.641 INFO:tasks.ceph.osd.2:Started 2026-02-20T11:25:07.641 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T11:25:07.716 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:07.713+0000 7f4e1a85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:07.718 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:07.716+0000 7f4e1a85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:07.750 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:25:07.751 INFO:teuthology.orchestra.run.trial038.stdout:{"epoch":6,"fsid":"97ccc1d9-5f9e-4a23-a83d-fcf45d35bda8","created":"2026-02-20T11:25:05.670427+0000","modified":"2026-02-20T11:25:07.418999+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T11:25:07.418999+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"799f2bec-9817-4b9b-bf35-54f155251e89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c193b783-ad52-4df0-80c8-b5df95abd182","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2dcf13a8-cd3d-4dc4-a01c-ef75162c3483","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:25:07.761 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-20T11:25:07.761 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-20T11:25:08.063 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T11:25:08.131 INFO:teuthology.misc.health.trial038.stderr:2026-02-20T11:25:08.127+0000 7ff743ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:08.132 INFO:teuthology.misc.health.trial038.stderr:2026-02-20T11:25:08.130+0000 7ff743ecb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:08.163 INFO:teuthology.misc.health.trial038.stdout: 2026-02-20T11:25:08.163 INFO:teuthology.misc.health.trial038.stdout:{"epoch":6,"fsid":"97ccc1d9-5f9e-4a23-a83d-fcf45d35bda8","created":"2026-02-20T11:25:05.670427+0000","modified":"2026-02-20T11:25:07.418999+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T11:25:07.418999+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"799f2bec-9817-4b9b-bf35-54f155251e89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c193b783-ad52-4df0-80c8-b5df95abd182","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2dcf13a8-cd3d-4dc4-a01c-ef75162c3483","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:25:08.168 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-20T11:25:09.094 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-20T11:25:09.091+0000 7f4c1796a640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-20T11:25:14.470 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T11:25:14.543 INFO:teuthology.misc.health.trial038.stderr:2026-02-20T11:25:14.539+0000 7f6f7350b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:14.544 INFO:teuthology.misc.health.trial038.stderr:2026-02-20T11:25:14.542+0000 7f6f7350b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:14.673 INFO:teuthology.misc.health.trial038.stdout: 2026-02-20T11:25:14.673 INFO:teuthology.misc.health.trial038.stdout:{"epoch":8,"fsid":"97ccc1d9-5f9e-4a23-a83d-fcf45d35bda8","created":"2026-02-20T11:25:05.670427+0000","modified":"2026-02-20T11:25:13.858652+0000","last_up_change":"0.000000","last_in_change":"2026-02-20T11:25:07.418999+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":4,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"799f2bec-9817-4b9b-bf35-54f155251e89","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"c193b783-ad52-4df0-80c8-b5df95abd182","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"2dcf13a8-cd3d-4dc4-a01c-ef75162c3483","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:25:14.684 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-20T11:25:20.987 DEBUG:teuthology.orchestra.run.trial038:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-20T11:25:21.055 INFO:teuthology.misc.health.trial038.stderr:2026-02-20T11:25:21.052+0000 7fbb9545a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:21.056 INFO:teuthology.misc.health.trial038.stderr:2026-02-20T11:25:21.055+0000 7fbb9545a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:21.165 INFO:teuthology.misc.health.trial038.stdout: 2026-02-20T11:25:21.165 INFO:teuthology.misc.health.trial038.stdout:{"epoch":14,"fsid":"97ccc1d9-5f9e-4a23-a83d-fcf45d35bda8","created":"2026-02-20T11:25:05.670427+0000","modified":"2026-02-20T11:25:20.882968+0000","last_up_change":"2026-02-20T11:25:16.869806+0000","last_in_change":"2026-02-20T11:25:07.418999+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T11:25:18.104514+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"799f2bec-9817-4b9b-bf35-54f155251e89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6807","nonce":1023039484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6802","nonce":1023039484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6808","nonce":1023039484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6811","nonce":1023039484}]},"public_addr":"10.20.193.38:6807/1023039484","cluster_addr":"10.20.193.38:6802/1023039484","heartbeat_back_addr":"10.20.193.38:6808/1023039484","heartbeat_front_addr":"10.20.193.38:6811/1023039484","state":["exists","up"]},{"osd":1,"uuid":"c193b783-ad52-4df0-80c8-b5df95abd182","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6803","nonce":2864570391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6804","nonce":2864570391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6805","nonce":2864570391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6806","nonce":2864570391}]},"public_addr":"10.20.193.38:6803/2864570391","cluster_addr":"10.20.193.38:6804/2864570391","heartbeat_back_addr":"10.20.193.38:6805/2864570391","heartbeat_front_addr":"10.20.193.38:6806/2864570391","state":["exists","up"]},{"osd":2,"uuid":"2dcf13a8-cd3d-4dc4-a01c-ef75162c3483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6809","nonce":1213923394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6810","nonce":1213923394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6812","nonce":1213923394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6813","nonce":1213923394}]},"public_addr":"10.20.193.38:6809/1213923394","cluster_addr":"10.20.193.38:6810/1213923394","heartbeat_back_addr":"10.20.193.38:6812/1213923394","heartbeat_front_addr":"10.20.193.38:6813/1213923394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:25:21.173 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-20T11:25:21.173 INFO:tasks.ceph:Creating RBD pool 2026-02-20T11:25:21.173 DEBUG:teuthology.orchestra.run.trial038:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-20T11:25:21.248 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:21.243+0000 7f89396cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:21.249 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:21.247+0000 7f89396cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:21.892 INFO:teuthology.orchestra.run.trial038.stderr:pool 'rbd' created 2026-02-20T11:25:21.905 DEBUG:teuthology.orchestra.run.trial038:> rbd --cluster ceph pool init rbd 2026-02-20T11:25:21.947 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:21.944+0000 7f910d4e0240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:21.947 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:21.944+0000 7f910d4e0240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:21.948 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:21.946+0000 7f910d4e0240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:24.930 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-20T11:25:24.931 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-20T11:25:24.932 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-20T11:25:25.047 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.042+0000 7f0a4a85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:25.048 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.046+0000 7f0a4a85a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:25.157 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:25:25.166 INFO:teuthology.orchestra.run.trial038.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-20T11:25:25.166 INFO:tasks.ceph_manager:config epoch is 1 2026-02-20T11:25:25.167 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-20T11:25:25.167 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-20T11:25:25.167 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-20T11:25:25.243 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.239+0000 7fa40541a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:25.244 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.242+0000 7fa40541a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:25.384 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:25:25.392 INFO:teuthology.orchestra.run.trial038.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6800","nonce":4157060660},{"type":"v1","addr":"10.20.193.38:6801","nonce":4157060660}]},"active_addr":"10.20.193.38:6801/4157060660","active_change":"2026-02-20T11:25:08.082323+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.38:0","nonce":2918145045}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.38:0","nonce":2560221138}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.38:0","nonce":3102650073}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.38:0","nonce":4039636301}]}]} 2026-02-20T11:25:25.396 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-20T11:25:25.396 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-20T11:25:25.396 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T11:25:25.472 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.467+0000 7fa45e39c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:25.472 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.470+0000 7fa45e39c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:25.582 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:25:25.582 INFO:teuthology.orchestra.run.trial038.stdout:{"epoch":18,"fsid":"97ccc1d9-5f9e-4a23-a83d-fcf45d35bda8","created":"2026-02-20T11:25:05.670427+0000","modified":"2026-02-20T11:25:24.903913+0000","last_up_change":"2026-02-20T11:25:16.869806+0000","last_in_change":"2026-02-20T11:25:07.418999+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T11:25:18.104514+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-20T11:25:21.360480+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"799f2bec-9817-4b9b-bf35-54f155251e89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6807","nonce":1023039484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6802","nonce":1023039484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6808","nonce":1023039484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6811","nonce":1023039484}]},"public_addr":"10.20.193.38:6807/1023039484","cluster_addr":"10.20.193.38:6802/1023039484","heartbeat_back_addr":"10.20.193.38:6808/1023039484","heartbeat_front_addr":"10.20.193.38:6811/1023039484","state":["exists","up"]},{"osd":1,"uuid":"c193b783-ad52-4df0-80c8-b5df95abd182","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6803","nonce":2864570391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6804","nonce":2864570391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6805","nonce":2864570391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6806","nonce":2864570391}]},"public_addr":"10.20.193.38:6803/2864570391","cluster_addr":"10.20.193.38:6804/2864570391","heartbeat_back_addr":"10.20.193.38:6805/2864570391","heartbeat_front_addr":"10.20.193.38:6806/2864570391","state":["exists","up"]},{"osd":2,"uuid":"2dcf13a8-cd3d-4dc4-a01c-ef75162c3483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6809","nonce":1213923394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6810","nonce":1213923394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6812","nonce":1213923394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6813","nonce":1213923394}]},"public_addr":"10.20.193.38:6809/1213923394","cluster_addr":"10.20.193.38:6810/1213923394","heartbeat_back_addr":"10.20.193.38:6812/1213923394","heartbeat_front_addr":"10.20.193.38:6813/1213923394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:25:25.591 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-20T11:25:25.591 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-20T11:25:25.669 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.664+0000 7efee36cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:25.670 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.668+0000 7efee36cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:25.777 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:25:25.777 INFO:teuthology.orchestra.run.trial038.stdout:{"epoch":18,"fsid":"97ccc1d9-5f9e-4a23-a83d-fcf45d35bda8","created":"2026-02-20T11:25:05.670427+0000","modified":"2026-02-20T11:25:24.903913+0000","last_up_change":"2026-02-20T11:25:16.869806+0000","last_in_change":"2026-02-20T11:25:07.418999+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":6,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-20T11:25:18.104514+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-20T11:25:21.360480+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"18","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":18,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"799f2bec-9817-4b9b-bf35-54f155251e89","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6807","nonce":1023039484}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6802","nonce":1023039484}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6808","nonce":1023039484}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6811","nonce":1023039484}]},"public_addr":"10.20.193.38:6807/1023039484","cluster_addr":"10.20.193.38:6802/1023039484","heartbeat_back_addr":"10.20.193.38:6808/1023039484","heartbeat_front_addr":"10.20.193.38:6811/1023039484","state":["exists","up"]},{"osd":1,"uuid":"c193b783-ad52-4df0-80c8-b5df95abd182","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6803","nonce":2864570391}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6804","nonce":2864570391}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6805","nonce":2864570391}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6806","nonce":2864570391}]},"public_addr":"10.20.193.38:6803/2864570391","cluster_addr":"10.20.193.38:6804/2864570391","heartbeat_back_addr":"10.20.193.38:6805/2864570391","heartbeat_front_addr":"10.20.193.38:6806/2864570391","state":["exists","up"]},{"osd":2,"uuid":"2dcf13a8-cd3d-4dc4-a01c-ef75162c3483","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":15,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6809","nonce":1213923394}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6810","nonce":1213923394}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6812","nonce":1213923394}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.38:6813","nonce":1213923394}]},"public_addr":"10.20.193.38:6809/1213923394","cluster_addr":"10.20.193.38:6810/1213923394","heartbeat_back_addr":"10.20.193.38:6812/1213923394","heartbeat_front_addr":"10.20.193.38:6813/1213923394","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-20T11:25:25.786 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-20T11:25:25.786 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-20T11:25:25.786 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-20T11:25:25.867 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.862+0000 7f418441a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:25.868 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.863+0000 7f129195b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:25.868 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.862+0000 7fb6acf9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:25.869 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.866+0000 7f418441a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:25.869 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.866+0000 7f129195b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:25.869 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.866+0000 7fb6acf9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:25.882 INFO:teuthology.orchestra.run.trial038.stdout:47244640264 2026-02-20T11:25:25.883 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-20T11:25:25.883 INFO:teuthology.orchestra.run.trial038.stdout:38654705674 2026-02-20T11:25:25.883 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-20T11:25:25.884 INFO:teuthology.orchestra.run.trial038.stdout:47244640264 2026-02-20T11:25:25.884 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-20T11:25:26.003 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.999+0000 7f3932fdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:26.004 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.999+0000 7f9b7b2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:26.005 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:26.002+0000 7f3932fdc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:26.005 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:25.998+0000 7fe0a20cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:26.006 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:26.003+0000 7f9b7b2cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:26.006 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:26.004+0000 7fe0a20cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:26.117 INFO:teuthology.orchestra.run.trial038.stdout:47244640261 2026-02-20T11:25:26.123 INFO:teuthology.orchestra.run.trial038.stdout:47244640261 2026-02-20T11:25:26.123 INFO:teuthology.orchestra.run.trial038.stdout:38654705668 2026-02-20T11:25:26.127 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640264 got 47244640261 for osd.2 2026-02-20T11:25:26.133 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640264 got 47244640261 for osd.0 2026-02-20T11:25:26.135 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705668 for osd.1 2026-02-20T11:25:27.128 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-20T11:25:27.135 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-20T11:25:27.135 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-20T11:25:27.208 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:27.203+0000 7f19f790b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:27.209 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:27.207+0000 7f19f790b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:27.251 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:27.245+0000 7ff39883d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:27.252 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:27.246+0000 7fbbc810b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:27.252 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:27.249+0000 7ff39883d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:27.253 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:27.251+0000 7fbbc810b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:27.321 INFO:teuthology.orchestra.run.trial038.stdout:47244640264 2026-02-20T11:25:27.332 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640264 got 47244640264 for osd.2 2026-02-20T11:25:27.332 DEBUG:teuthology.parallel:result is None 2026-02-20T11:25:27.364 INFO:teuthology.orchestra.run.trial038.stdout:47244640264 2026-02-20T11:25:27.368 INFO:teuthology.orchestra.run.trial038.stdout:38654705674 2026-02-20T11:25:27.374 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640264 got 47244640264 for osd.0 2026-02-20T11:25:27.375 DEBUG:teuthology.parallel:result is None 2026-02-20T11:25:27.379 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705674 for osd.1 2026-02-20T11:25:27.379 DEBUG:teuthology.parallel:result is None 2026-02-20T11:25:27.379 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-20T11:25:27.380 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:25:27.495 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:27.490+0000 7f12f230b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:27.496 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:27.494+0000 7f12f230b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:27.605 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:25:27.606 INFO:teuthology.orchestra.run.trial038.stderr:dumped all 2026-02-20T11:25:27.614 INFO:teuthology.orchestra.run.trial038.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2026-02-20T11:25:26.090743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":2836,"kb_used_data":2836,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156497132,"statfs":{"total":160255967232,"available":160253063168,"internally_reserved":0,"allocated":2904064,"data_stored":2904064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.184279"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.915622+0000","last_change":"2026-02-20T11:25:24.918294+0000","last_active":"2026-02-20T11:25:24.915622+0000","last_peered":"2026-02-20T11:25:24.915622+0000","last_clean":"2026-02-20T11:25:24.915622+0000","last_became_active":"2026-02-20T11:25:22.931464+0000","last_became_peered":"2026-02-20T11:25:22.931464+0000","last_unstale":"2026-02-20T11:25:24.915622+0000","last_undegraded":"2026-02-20T11:25:24.915622+0000","last_fullsized":"2026-02-20T11:25:24.915622+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_clean_scrub_stamp":"2026-02-20T11:25:21.887889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.917638+0000","last_change":"2026-02-20T11:25:24.919809+0000","last_active":"2026-02-20T11:25:24.917638+0000","last_peered":"2026-02-20T11:25:24.917638+0000","last_clean":"2026-02-20T11:25:24.917638+0000","last_became_active":"2026-02-20T11:25:22.926970+0000","last_became_peered":"2026-02-20T11:25:22.926970+0000","last_unstale":"2026-02-20T11:25:24.917638+0000","last_undegraded":"2026-02-20T11:25:24.917638+0000","last_fullsized":"2026-02-20T11:25:24.917638+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_clean_scrub_stamp":"2026-02-20T11:25:21.887889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.917599+0000","last_change":"2026-02-20T11:25:24.920903+0000","last_active":"2026-02-20T11:25:24.917599+0000","last_peered":"2026-02-20T11:25:24.917599+0000","last_clean":"2026-02-20T11:25:24.917599+0000","last_became_active":"2026-02-20T11:25:22.930890+0000","last_became_peered":"2026-02-20T11:25:22.930890+0000","last_unstale":"2026-02-20T11:25:24.917599+0000","last_undegraded":"2026-02-20T11:25:24.917599+0000","last_fullsized":"2026-02-20T11:25:24.917599+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_clean_scrub_stamp":"2026-02-20T11:25:21.887889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.916690+0000","last_change":"2026-02-20T11:25:24.918954+0000","last_active":"2026-02-20T11:25:24.916690+0000","last_peered":"2026-02-20T11:25:24.916690+0000","last_clean":"2026-02-20T11:25:24.916690+0000","last_became_active":"2026-02-20T11:25:22.934874+0000","last_became_peered":"2026-02-20T11:25:22.934874+0000","last_unstale":"2026-02-20T11:25:24.916690+0000","last_undegraded":"2026-02-20T11:25:24.916690+0000","last_fullsized":"2026-02-20T11:25:24.916690+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_clean_scrub_stamp":"2026-02-20T11:25:21.887889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.917953+0000","last_change":"2026-02-20T11:25:24.918800+0000","last_active":"2026-02-20T11:25:24.917953+0000","last_peered":"2026-02-20T11:25:24.917953+0000","last_clean":"2026-02-20T11:25:24.917953+0000","last_became_active":"2026-02-20T11:25:22.939559+0000","last_became_peered":"2026-02-20T11:25:22.939559+0000","last_unstale":"2026-02-20T11:25:24.917953+0000","last_undegraded":"2026-02-20T11:25:24.917953+0000","last_fullsized":"2026-02-20T11:25:24.917953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_clean_scrub_stamp":"2026-02-20T11:25:21.887889+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.916163+0000","last_change":"2026-02-20T11:25:24.918024+0000","last_active":"2026-02-20T11:25:24.916163+0000","last_peered":"2026-02-20T11:25:24.916163+0000","last_clean":"2026-02-20T11:25:24.916163+0000","last_became_active":"2026-02-20T11:25:22.935562+0000","last_became_peered":"2026-02-20T11:25:22.935562+0000","last_unstale":"2026-02-20T11:25:24.916163+0000","last_undegraded":"2026-02-20T11:25:24.916163+0000","last_fullsized":"2026-02-20T11:25:24.916163+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_clean_scrub_stamp":"2026-02-20T11:25:21.887889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.914973+0000","last_change":"2026-02-20T11:25:24.917279+0000","last_active":"2026-02-20T11:25:24.914973+0000","last_peered":"2026-02-20T11:25:24.914973+0000","last_clean":"2026-02-20T11:25:24.914973+0000","last_became_active":"2026-02-20T11:25:22.945248+0000","last_became_peered":"2026-02-20T11:25:22.945248+0000","last_unstale":"2026-02-20T11:25:24.914973+0000","last_undegraded":"2026-02-20T11:25:24.914973+0000","last_fullsized":"2026-02-20T11:25:24.914973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_clean_scrub_stamp":"2026-02-20T11:25:21.887889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.917680+0000","last_change":"2026-02-20T11:25:24.920918+0000","last_active":"2026-02-20T11:25:24.917680+0000","last_peered":"2026-02-20T11:25:24.917680+0000","last_clean":"2026-02-20T11:25:24.917680+0000","last_became_active":"2026-02-20T11:25:22.928190+0000","last_became_peered":"2026-02-20T11:25:22.928190+0000","last_unstale":"2026-02-20T11:25:24.917680+0000","last_undegraded":"2026-02-20T11:25:24.917680+0000","last_fullsized":"2026-02-20T11:25:24.917680+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_clean_scrub_stamp":"2026-02-20T11:25:21.887889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":90,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.915439+0000","last_change":"2026-02-20T11:25:19.909953+0000","last_active":"2026-02-20T11:25:24.915439+0000","last_peered":"2026-02-20T11:25:24.915439+0000","last_clean":"2026-02-20T11:25:24.915439+0000","last_became_active":"2026-02-20T11:25:19.903209+0000","last_became_peered":"2026-02-20T11:25:19.903209+0000","last_unstale":"2026-02-20T11:25:24.915439+0000","last_undegraded":"2026-02-20T11:25:24.915439+0000","last_fullsized":"2026-02-20T11:25:24.915439+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:18.873540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:18.873540+0000","last_clean_scrub_stamp":"2026-02-20T11:25:18.873540+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253014016,"internally_reserved":0,"allocated":2953216,"data_stored":2953216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":11,"seq":47244640264,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":364,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166292,"statfs":{"total":53418655744,"available":53418283008,"internally_reserved":0,"allocated":372736,"data_stored":372736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640264,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1188,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165468,"statfs":{"total":53418655744,"available":53417439232,"internally_reserved":0,"allocated":1216512,"data_stored":1216512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1284,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165372,"statfs":{"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417431040,"internally_reserved":0,"allocated":1224704,"data_stored":1224704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418274816,"internally_reserved":0,"allocated":380928,"data_stored":380928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:25:27.616 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-20T11:25:27.691 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:27.687+0000 7ff35e75c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:27.692 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:27.690+0000 7ff35e75c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:27.800 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:25:27.801 INFO:teuthology.orchestra.run.trial038.stderr:dumped all 2026-02-20T11:25:27.809 INFO:teuthology.orchestra.run.trial038.stdout:{"pg_ready":true,"pg_map":{"version":18,"stamp":"2026-02-20T11:25:26.090743+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":156499968,"kb_used":2836,"kb_used_data":2836,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":156497132,"statfs":{"total":160255967232,"available":160253063168,"internally_reserved":0,"allocated":2904064,"data_stored":2904064,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"1.184279"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.915622+0000","last_change":"2026-02-20T11:25:24.918294+0000","last_active":"2026-02-20T11:25:24.915622+0000","last_peered":"2026-02-20T11:25:24.915622+0000","last_clean":"2026-02-20T11:25:24.915622+0000","last_became_active":"2026-02-20T11:25:22.931464+0000","last_became_peered":"2026-02-20T11:25:22.931464+0000","last_unstale":"2026-02-20T11:25:24.915622+0000","last_undegraded":"2026-02-20T11:25:24.915622+0000","last_fullsized":"2026-02-20T11:25:24.915622+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_clean_scrub_stamp":"2026-02-20T11:25:21.887889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.917638+0000","last_change":"2026-02-20T11:25:24.919809+0000","last_active":"2026-02-20T11:25:24.917638+0000","last_peered":"2026-02-20T11:25:24.917638+0000","last_clean":"2026-02-20T11:25:24.917638+0000","last_became_active":"2026-02-20T11:25:22.926970+0000","last_became_peered":"2026-02-20T11:25:22.926970+0000","last_unstale":"2026-02-20T11:25:24.917638+0000","last_undegraded":"2026-02-20T11:25:24.917638+0000","last_fullsized":"2026-02-20T11:25:24.917638+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_clean_scrub_stamp":"2026-02-20T11:25:21.887889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.917599+0000","last_change":"2026-02-20T11:25:24.920903+0000","last_active":"2026-02-20T11:25:24.917599+0000","last_peered":"2026-02-20T11:25:24.917599+0000","last_clean":"2026-02-20T11:25:24.917599+0000","last_became_active":"2026-02-20T11:25:22.930890+0000","last_became_peered":"2026-02-20T11:25:22.930890+0000","last_unstale":"2026-02-20T11:25:24.917599+0000","last_undegraded":"2026-02-20T11:25:24.917599+0000","last_fullsized":"2026-02-20T11:25:24.917599+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_clean_scrub_stamp":"2026-02-20T11:25:21.887889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.916690+0000","last_change":"2026-02-20T11:25:24.918954+0000","last_active":"2026-02-20T11:25:24.916690+0000","last_peered":"2026-02-20T11:25:24.916690+0000","last_clean":"2026-02-20T11:25:24.916690+0000","last_became_active":"2026-02-20T11:25:22.934874+0000","last_became_peered":"2026-02-20T11:25:22.934874+0000","last_unstale":"2026-02-20T11:25:24.916690+0000","last_undegraded":"2026-02-20T11:25:24.916690+0000","last_fullsized":"2026-02-20T11:25:24.916690+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_clean_scrub_stamp":"2026-02-20T11:25:21.887889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"18'2","reported_seq":17,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.917953+0000","last_change":"2026-02-20T11:25:24.918800+0000","last_active":"2026-02-20T11:25:24.917953+0000","last_peered":"2026-02-20T11:25:24.917953+0000","last_clean":"2026-02-20T11:25:24.917953+0000","last_became_active":"2026-02-20T11:25:22.939559+0000","last_became_peered":"2026-02-20T11:25:22.939559+0000","last_unstale":"2026-02-20T11:25:24.917953+0000","last_undegraded":"2026-02-20T11:25:24.917953+0000","last_fullsized":"2026-02-20T11:25:24.917953+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_clean_scrub_stamp":"2026-02-20T11:25:21.887889+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.916163+0000","last_change":"2026-02-20T11:25:24.918024+0000","last_active":"2026-02-20T11:25:24.916163+0000","last_peered":"2026-02-20T11:25:24.916163+0000","last_clean":"2026-02-20T11:25:24.916163+0000","last_became_active":"2026-02-20T11:25:22.935562+0000","last_became_peered":"2026-02-20T11:25:22.935562+0000","last_unstale":"2026-02-20T11:25:24.916163+0000","last_undegraded":"2026-02-20T11:25:24.916163+0000","last_fullsized":"2026-02-20T11:25:24.916163+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_clean_scrub_stamp":"2026-02-20T11:25:21.887889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.914973+0000","last_change":"2026-02-20T11:25:24.917279+0000","last_active":"2026-02-20T11:25:24.914973+0000","last_peered":"2026-02-20T11:25:24.914973+0000","last_clean":"2026-02-20T11:25:24.914973+0000","last_became_active":"2026-02-20T11:25:22.945248+0000","last_became_peered":"2026-02-20T11:25:22.945248+0000","last_unstale":"2026-02-20T11:25:24.914973+0000","last_undegraded":"2026-02-20T11:25:24.914973+0000","last_fullsized":"2026-02-20T11:25:24.914973+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_clean_scrub_stamp":"2026-02-20T11:25:21.887889+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"16'1","reported_seq":16,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.917680+0000","last_change":"2026-02-20T11:25:24.920918+0000","last_active":"2026-02-20T11:25:24.917680+0000","last_peered":"2026-02-20T11:25:24.917680+0000","last_clean":"2026-02-20T11:25:24.917680+0000","last_became_active":"2026-02-20T11:25:22.928190+0000","last_became_peered":"2026-02-20T11:25:22.928190+0000","last_unstale":"2026-02-20T11:25:24.917680+0000","last_undegraded":"2026-02-20T11:25:24.917680+0000","last_fullsized":"2026-02-20T11:25:24.917680+0000","mapping_epoch":15,"log_start":"0'0","ondisk_log_start":"0'0","created":15,"last_epoch_clean":16,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:21.887889+0000","last_clean_scrub_stamp":"2026-02-20T11:25:21.887889+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"13'52","reported_seq":90,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-20T11:25:24.915439+0000","last_change":"2026-02-20T11:25:19.909953+0000","last_active":"2026-02-20T11:25:24.915439+0000","last_peered":"2026-02-20T11:25:24.915439+0000","last_clean":"2026-02-20T11:25:24.915439+0000","last_became_active":"2026-02-20T11:25:19.903209+0000","last_became_peered":"2026-02-20T11:25:19.903209+0000","last_unstale":"2026-02-20T11:25:24.915439+0000","last_undegraded":"2026-02-20T11:25:24.915439+0000","last_fullsized":"2026-02-20T11:25:24.915439+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-20T11:25:18.873540+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-20T11:25:18.873540+0000","last_clean_scrub_stamp":"2026-02-20T11:25:18.873540+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253014016,"internally_reserved":0,"allocated":2953216,"data_stored":2953216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":11,"seq":47244640264,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":364,"kb_used_data":364,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166292,"statfs":{"total":53418655744,"available":53418283008,"internally_reserved":0,"allocated":372736,"data_stored":372736,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640264,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1188,"kb_used_data":1188,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165468,"statfs":{"total":53418655744,"available":53417439232,"internally_reserved":0,"allocated":1216512,"data_stored":1216512,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1284,"kb_used_data":1284,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165372,"statfs":{"total":53418655744,"available":53417340928,"internally_reserved":0,"allocated":1314816,"data_stored":1314816,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417431040,"internally_reserved":0,"allocated":1224704,"data_stored":1224704,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53418655744,"available":53417308160,"internally_reserved":0,"allocated":1347584,"data_stored":1347584,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53418655744,"available":53418274816,"internally_reserved":0,"allocated":380928,"data_stored":380928,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-20T11:25:27.811 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-20T11:25:27.811 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-20T11:25:27.811 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-20T11:25:27.811 DEBUG:teuthology.orchestra.run.trial038:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-20T11:25:27.888 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:27.885+0000 7fafb32cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:27.889 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20T11:25:27.887+0000 7fafb32cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-20T11:25:28.025 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:25:28.026 INFO:teuthology.orchestra.run.trial038.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-20T11:25:28.035 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-20T11:25:28.036 INFO:teuthology.run_tasks:Running task ssh_keys... 2026-02-20T11:25:28.153 DEBUG:teuthology.orchestra.run.trial038:> echo 'StrictHostKeyChecking no 2026-02-20T11:25:28.153 DEBUG:teuthology.orchestra.run.trial038:> ' > /home/ubuntu/.ssh/config && echo 'UserKnownHostsFile ' /dev/null >> /home/ubuntu/.ssh/config && chmod 600 /home/ubuntu/.ssh/config 2026-02-20T11:25:28.168 INFO:teuthology.task.ssh_keys:generated public key AAAAB3NzaC1yc2EAAAADAQABAAABAQDG4/dNO/5lOYBra3g5butT7mRi/glkng7zm3IZ/QllFdSONnJTGXmrTmWXejkUN3puxkJbI/wEwmJesOYVe9vG7rvMxHoq6INB2CAESmTTa699gLLEy9BdyUWsxmSNIUPdUMOaUueITEVZc6MRDUSb0P2eCltbgD4P8yoQIdQ24fBCLJrBp5xbl8J5aRCj2CLjnfxP8oAMj/MtzOdJDgJuZraP55pUefp2Ht6YRRY9GApPr85TSPFXXY34cSgAb34sv2gcffYaStuIXZey0ogb163gRGy+vpAcTVyjtA5SYozAArZukLfScG45lIogiXEacQF2TAK9bw1oarYJwoRX 2026-02-20T11:25:28.168 INFO:teuthology.task.ssh_keys:pushing keys to trial038.front.sepia.ceph.com for ubuntu 2026-02-20T11:25:28.169 DEBUG:teuthology.orchestra.run.trial038:> rm -f -- /home/ubuntu/.ssh/id_rsa 2026-02-20T11:25:28.225 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:25:28.225 DEBUG:teuthology.orchestra.run.trial038:> dd of=/home/ubuntu/.ssh/id_rsa 2026-02-20T11:25:28.225 DEBUG:teuthology.orchestra.run.trial038:> chmod 0500 /home/ubuntu/.ssh/id_rsa 2026-02-20T11:25:28.282 DEBUG:teuthology.orchestra.run.trial038:> rm -f -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-20T11:25:28.338 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:25:28.338 DEBUG:teuthology.orchestra.run.trial038:> dd of=/home/ubuntu/.ssh/id_rsa.pub 2026-02-20T11:25:28.395 DEBUG:teuthology.orchestra.run.trial038:> cp -v -a /home/ubuntu/.ssh/authorized_keys /home/ubuntu/.ssh/authorized_keys_2026-02-20_11:25:28:395229 2026-02-20T11:25:28.452 INFO:teuthology.orchestra.run.trial038.stdout:'/home/ubuntu/.ssh/authorized_keys' -> '/home/ubuntu/.ssh/authorized_keys_2026-02-20_11:25:28:395229' 2026-02-20T11:25:28.453 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:25:28.453 DEBUG:teuthology.orchestra.run.trial038:> dd of=/home/ubuntu/.ssh/authorized_keys conv=notrunc oflag=append 2026-02-20T11:25:28.508 INFO:teuthology.run_tasks:Running task cbt... 2026-02-20T11:25:28.524 INFO:tasks.cbt:cbt configuration is {'cluster': {'user': 'ubuntu', 'head': 'ubuntu@trial038.front.sepia.ceph.com', 'osds': ['ubuntu@trial038.front.sepia.ceph.com'], 'mons': ['ubuntu@trial038.front.sepia.ceph.com'], 'clients': ['ubuntu@trial038.front.sepia.ceph.com'], 'rgws': {'ubuntu@trial038.front.sepia.ceph.com': None}, 'osds_per_node': 3, 'rebuild_every_test': False, 'use_existing': True, 'is_teuthology': True, 'iterations': 1, 'tmp_dir': '/tmp/cbt', 'pool_profiles': {'replicated': {'pg_size': 256, 'pgp_size': 256, 'replication': 'replicated'}}, 'pid_dir': '/var/run/ceph'}, 'benchmarks': {'radosbench': {'concurrent_ops': 4, 'concurrent_procs': 2, 'op_size': [4194304], 'pool_monitoring_list': ['collectl'], 'pool_profile': 'replicated', 'run_monitoring_list': ['collectl'], 'time': 60, 'write_only': False}}, 'client_endpoints': None, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}} 2026-02-20T11:25:28.524 DEBUG:teuthology.orchestra.run.trial038:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/cbt 2026-02-20T11:25:28.568 DEBUG:teuthology.orchestra.run.trial038:> set -ex 2026-02-20T11:25:28.569 DEBUG:teuthology.orchestra.run.trial038:> dd of=/home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2026-02-20T11:25:28.620 DEBUG:teuthology.orchestra.run.trial038:> git clone --depth 1 -b master https://github.com/ceph/cbt.git /home/ubuntu/cephtest/cbt 2026-02-20T11:25:28.682 INFO:teuthology.orchestra.run.trial038.stderr:Cloning into '/home/ubuntu/cephtest/cbt'... 2026-02-20T11:25:29.956 INFO:tasks.cbt:Installing collectl 2026-02-20T11:25:29.957 DEBUG:teuthology.orchestra.run.trial038:> sudo mkdir collectl && cd collectl && sudo wget https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download -O collectl.tar.gz && sudo tar -xvf collectl.tar.gz && cd collectl-4.3.1 && sudo ./INSTALL 2026-02-20T11:25:30.033 INFO:teuthology.orchestra.run.trial038.stderr:--2026-02-20 11:25:30-- https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download 2026-02-20T11:25:30.167 INFO:teuthology.orchestra.run.trial038.stderr:Resolving sourceforge.net (sourceforge.net)... 104.18.12.149, 104.18.13.149, 2606:4700::6812:d95, ... 2026-02-20T11:25:30.249 INFO:teuthology.orchestra.run.trial038.stderr:Connecting to sourceforge.net (sourceforge.net)|104.18.12.149|:443... connected. 2026-02-20T11:25:30.596 INFO:teuthology.orchestra.run.trial038.stderr:HTTP request sent, awaiting response... 302 Found 2026-02-20T11:25:30.596 INFO:teuthology.orchestra.run.trial038.stderr:Location: https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABpmESqtVHKhcmRSN5FaLROLAlS3BizMAm7XBico_eWj6v_1elcTPxYrksS2bAs8OVyuEvibGAzXreMF7BmrXfxl_2PbA%3D%3D&use_mirror=pilotfiber&r= [following] 2026-02-20T11:25:30.596 INFO:teuthology.orchestra.run.trial038.stderr:--2026-02-20 11:25:30-- https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABpmESqtVHKhcmRSN5FaLROLAlS3BizMAm7XBico_eWj6v_1elcTPxYrksS2bAs8OVyuEvibGAzXreMF7BmrXfxl_2PbA%3D%3D&use_mirror=pilotfiber&r= 2026-02-20T11:25:30.731 INFO:teuthology.orchestra.run.trial038.stderr:Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 104.18.13.149, 104.18.12.149, 2606:4700::6812:d95, ... 2026-02-20T11:25:30.817 INFO:teuthology.orchestra.run.trial038.stderr:Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|104.18.13.149|:443... connected. 2026-02-20T11:25:31.150 INFO:teuthology.orchestra.run.trial038.stderr:HTTP request sent, awaiting response... 302 Found 2026-02-20T11:25:31.150 INFO:teuthology.orchestra.run.trial038.stderr:Location: https://pilotfiber.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 [following] 2026-02-20T11:25:31.151 INFO:teuthology.orchestra.run.trial038.stderr:--2026-02-20 11:25:31-- https://pilotfiber.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 2026-02-20T11:25:31.227 INFO:teuthology.orchestra.run.trial038.stderr:Resolving pilotfiber.dl.sourceforge.net (pilotfiber.dl.sourceforge.net)... 216.158.133.250 2026-02-20T11:25:31.231 INFO:teuthology.orchestra.run.trial038.stderr:Connecting to pilotfiber.dl.sourceforge.net (pilotfiber.dl.sourceforge.net)|216.158.133.250|:443... connected. 2026-02-20T11:25:31.303 INFO:teuthology.orchestra.run.trial038.stderr:HTTP request sent, awaiting response... 200 OK 2026-02-20T11:25:31.304 INFO:teuthology.orchestra.run.trial038.stderr:Length: 620693 (606K) [application/x-gzip] 2026-02-20T11:25:31.304 INFO:teuthology.orchestra.run.trial038.stderr:Saving to: ‘collectl.tar.gz’ 2026-02-20T11:25:31.304 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-20T11:25:31.324 INFO:teuthology.orchestra.run.trial038.stderr: 0K .......... .......... .......... .......... .......... 8% 2.31M 0s 2026-02-20T11:25:31.330 INFO:teuthology.orchestra.run.trial038.stderr: 50K .......... .......... .......... .......... .......... 16% 14.9M 0s 2026-02-20T11:25:31.330 INFO:teuthology.orchestra.run.trial038.stderr: 100K .......... .......... .......... .......... .......... 24% 374M 0s 2026-02-20T11:25:31.332 INFO:teuthology.orchestra.run.trial038.stderr: 150K .......... .......... .......... .......... .......... 32% 15.3M 0s 2026-02-20T11:25:31.356 INFO:teuthology.orchestra.run.trial038.stderr: 200K .......... .......... .......... .......... .......... 41% 2.93M 0s 2026-02-20T11:25:31.356 INFO:teuthology.orchestra.run.trial038.stderr: 250K .......... .......... .......... .......... .......... 49% 286M 0s 2026-02-20T11:25:31.356 INFO:teuthology.orchestra.run.trial038.stderr: 300K .......... .......... .......... .......... .......... 57% 283M 0s 2026-02-20T11:25:31.357 INFO:teuthology.orchestra.run.trial038.stderr: 350K .......... .......... .......... .......... .......... 65% 180M 0s 2026-02-20T11:25:31.357 INFO:teuthology.orchestra.run.trial038.stderr: 400K .......... .......... .......... .......... .......... 74% 18.5M 0s 2026-02-20T11:25:31.357 INFO:teuthology.orchestra.run.trial038.stderr: 450K .......... .......... .......... .......... .......... 82% 369M 0s 2026-02-20T11:25:31.357 INFO:teuthology.orchestra.run.trial038.stderr: 500K .......... .......... .......... .......... .......... 90% 391M 0s 2026-02-20T11:25:31.357 INFO:teuthology.orchestra.run.trial038.stderr: 550K .......... .......... .......... .......... .......... 98% 351M 0s 2026-02-20T11:25:31.357 INFO:teuthology.orchestra.run.trial038.stderr: 600K ...... 100% 11.4T=0.05s 2026-02-20T11:25:31.357 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-20T11:25:31.357 INFO:teuthology.orchestra.run.trial038.stderr:2026-02-20 11:25:31 (12.3 MB/s) - ‘collectl.tar.gz’ saved [620693/620693] 2026-02-20T11:25:31.357 INFO:teuthology.orchestra.run.trial038.stderr: 2026-02-20T11:25:31.371 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/ 2026-02-20T11:25:31.372 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/collectl.conf 2026-02-20T11:25:31.372 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/ 2026-02-20T11:25:31.372 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/SlowProc.html 2026-02-20T11:25:31.372 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/BuddyInfo.html 2026-02-20T11:25:31.372 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/WhySummary.html 2026-02-20T11:25:31.372 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Socket.html 2026-02-20T11:25:31.372 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/colmux.html 2026-02-20T11:25:31.372 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Tutorial.html 2026-02-20T11:25:31.372 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Gexpr.html 2026-02-20T11:25:31.372 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Import.html 2026-02-20T11:25:31.372 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/style.css 2026-02-20T11:25:31.372 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Ganglia3.jpg 2026-02-20T11:25:31.372 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Ganglia2.jpg 2026-02-20T11:25:31.372 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/FileNaming.html 2026-02-20T11:25:31.372 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/TheMath.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Environmental.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Process.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/OutputFiles.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/HiResTime.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/OutputFormats.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/slub.jpg 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Lustre.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Documentation.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Inodes.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Graphite.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/InputFiles.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Export.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Interrupts.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Startup.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Performance.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Messages.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Logging.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Data-detail.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/CPUs.html 2026-02-20T11:25:31.373 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Matrix.html 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Data-brief.html 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/SlabInfo.html 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Misc.html 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Examples.html 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/OpenStack.html 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Plotfiles.html 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/ColmuxCPU.jpg 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Tutorial-Lustre.html 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Data-verbose.html 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Data.html 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Exceptions.html 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/RunningAsAService.html 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/FAQ-collectl.html 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Ganglia.jpg 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/OperationalModes.html 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Playback.html 2026-02-20T11:25:31.374 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Hello.html 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Sockets.html 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Network.html 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Infiniband.html 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Memory.html 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/ColmuxLustre.jpg 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/Disks.html 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/docs/NfsInfo.html 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/service/ 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/service/collectl.service 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/initd/ 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/initd/collectl-suse 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/initd/collectl-debian 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/initd/collectl-generic 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/initd/collectl 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/misc.ph 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/gexpr.ph 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/statsd.ph 2026-02-20T11:25:31.375 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/envrules.std 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/hello.ph 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/vmsum.ph 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/proctree.ph 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/README-WINDOWS 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/formatit.ph 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/colmux 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/INSTALL 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/graphite.ph 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/RELEASE-collectl 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/vmstat.ph 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/README 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/man1/ 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/man1/collectl.1 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/man1/colmux.1 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/lexpr.ph 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/client.pl 2026-02-20T11:25:31.376 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/collectl 2026-02-20T11:25:31.377 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/COPYING 2026-02-20T11:25:31.377 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/vnet.ph 2026-02-20T11:25:31.377 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/GPL 2026-02-20T11:25:31.377 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/UNINSTALL 2026-02-20T11:25:31.377 INFO:teuthology.orchestra.run.trial038.stdout:collectl-4.3.1/ARTISTIC 2026-02-20T11:25:31.415 INFO:teuthology.orchestra.run.trial038.stderr:./INSTALL: line 90: chkconfig: command not found 2026-02-20T11:25:31.417 DEBUG:teuthology.orchestra.run.trial038:> sudo yum -y install librbd-devel pdsh pdsh-rcmd-ssh perf 2026-02-20T11:25:31.769 INFO:teuthology.orchestra.run.trial038.stdout:Last metadata expiration check: 0:00:59 ago on Fri 20 Feb 2026 11:24:32 AM UTC. 2026-02-20T11:25:31.854 INFO:teuthology.orchestra.run.trial038.stdout:Dependencies resolved. 2026-02-20T11:25:31.855 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:25:31.855 INFO:teuthology.orchestra.run.trial038.stdout: Package Arch Version Repository Size 2026-02-20T11:25:31.855 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:25:31.855 INFO:teuthology.orchestra.run.trial038.stdout:Installing: 2026-02-20T11:25:31.855 INFO:teuthology.orchestra.run.trial038.stdout: librbd-devel x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 24 k 2026-02-20T11:25:31.855 INFO:teuthology.orchestra.run.trial038.stdout: pdsh x86_64 2.34-7.el9 epel 118 k 2026-02-20T11:25:31.855 INFO:teuthology.orchestra.run.trial038.stdout: pdsh-rcmd-ssh x86_64 2.34-7.el9 epel 13 k 2026-02-20T11:25:31.855 INFO:teuthology.orchestra.run.trial038.stdout: perf x86_64 5.14.0-681.el9 appstream 3.2 M 2026-02-20T11:25:31.856 INFO:teuthology.orchestra.run.trial038.stdout:Installing dependencies: 2026-02-20T11:25:31.856 INFO:teuthology.orchestra.run.trial038.stdout: libradospp-devel x86_64 2:20.3.0-5357.g0c3148ca.el9 ceph 31 k 2026-02-20T11:25:31.856 INFO:teuthology.orchestra.run.trial038.stdout: libtraceevent x86_64 1.8.4-2.el9 baseos 290 k 2026-02-20T11:25:31.856 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:25:31.856 INFO:teuthology.orchestra.run.trial038.stdout:Transaction Summary 2026-02-20T11:25:31.856 INFO:teuthology.orchestra.run.trial038.stdout:================================================================================ 2026-02-20T11:25:31.856 INFO:teuthology.orchestra.run.trial038.stdout:Install 6 Packages 2026-02-20T11:25:31.856 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:25:31.856 INFO:teuthology.orchestra.run.trial038.stdout:Total download size: 3.7 M 2026-02-20T11:25:31.856 INFO:teuthology.orchestra.run.trial038.stdout:Installed size: 14 M 2026-02-20T11:25:31.856 INFO:teuthology.orchestra.run.trial038.stdout:Downloading Packages: 2026-02-20T11:25:32.642 INFO:teuthology.orchestra.run.trial038.stdout:(1/6): libradospp-devel-20.3.0-5357.g0c3148ca.e 92 kB/s | 31 kB 00:00 2026-02-20T11:25:32.643 INFO:teuthology.orchestra.run.trial038.stdout:(2/6): librbd-devel-20.3.0-5357.g0c3148ca.el9.x 70 kB/s | 24 kB 00:00 2026-02-20T11:25:32.813 INFO:teuthology.orchestra.run.trial038.stdout:(3/6): libtraceevent-1.8.4-2.el9.x86_64.rpm 569 kB/s | 290 kB 00:00 2026-02-20T11:25:33.044 INFO:teuthology.orchestra.run.trial038.stdout:(4/6): pdsh-rcmd-ssh-2.34-7.el9.x86_64.rpm 57 kB/s | 13 kB 00:00 2026-02-20T11:25:33.118 INFO:teuthology.orchestra.run.trial038.stdout:(5/6): pdsh-2.34-7.el9.x86_64.rpm 250 kB/s | 118 kB 00:00 2026-02-20T11:25:33.720 INFO:teuthology.orchestra.run.trial038.stdout:(6/6): perf-5.14.0-681.el9.x86_64.rpm 3.0 MB/s | 3.2 MB 00:01 2026-02-20T11:25:33.723 INFO:teuthology.orchestra.run.trial038.stdout:-------------------------------------------------------------------------------- 2026-02-20T11:25:33.723 INFO:teuthology.orchestra.run.trial038.stdout:Total 2.0 MB/s | 3.7 MB 00:01 2026-02-20T11:25:33.747 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction check 2026-02-20T11:25:33.757 INFO:teuthology.orchestra.run.trial038.stdout:Transaction check succeeded. 2026-02-20T11:25:33.757 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction test 2026-02-20T11:25:33.814 INFO:teuthology.orchestra.run.trial038.stdout:Transaction test succeeded. 2026-02-20T11:25:33.815 INFO:teuthology.orchestra.run.trial038.stdout:Running transaction 2026-02-20T11:25:33.888 INFO:teuthology.orchestra.run.trial038.stdout: Preparing : 1/1 2026-02-20T11:25:33.893 INFO:teuthology.orchestra.run.trial038.stdout: Installing : pdsh-rcmd-ssh-2.34-7.el9.x86_64 1/6 2026-02-20T11:25:33.933 INFO:teuthology.orchestra.run.trial038.stdout: Installing : pdsh-2.34-7.el9.x86_64 2/6 2026-02-20T11:25:33.938 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libtraceevent-1.8.4-2.el9.x86_64 3/6 2026-02-20T11:25:33.940 INFO:teuthology.orchestra.run.trial038.stdout: Installing : libradospp-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 4/6 2026-02-20T11:25:34.105 INFO:teuthology.orchestra.run.trial038.stdout: Installing : librbd-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 5/6 2026-02-20T11:25:34.125 INFO:teuthology.orchestra.run.trial038.stdout: Installing : perf-5.14.0-681.el9.x86_64 6/6 2026-02-20T11:25:34.236 INFO:teuthology.orchestra.run.trial038.stdout: Running scriptlet: perf-5.14.0-681.el9.x86_64 6/6 2026-02-20T11:25:34.236 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libradospp-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 1/6 2026-02-20T11:25:34.236 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : librbd-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 2/6 2026-02-20T11:25:34.236 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : libtraceevent-1.8.4-2.el9.x86_64 3/6 2026-02-20T11:25:34.236 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : perf-5.14.0-681.el9.x86_64 4/6 2026-02-20T11:25:34.236 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : pdsh-2.34-7.el9.x86_64 5/6 2026-02-20T11:25:34.297 INFO:teuthology.orchestra.run.trial038.stdout: Verifying : pdsh-rcmd-ssh-2.34-7.el9.x86_64 6/6 2026-02-20T11:25:34.297 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:25:34.298 INFO:teuthology.orchestra.run.trial038.stdout:Installed: 2026-02-20T11:25:34.298 INFO:teuthology.orchestra.run.trial038.stdout: libradospp-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:25:34.298 INFO:teuthology.orchestra.run.trial038.stdout: librbd-devel-2:20.3.0-5357.g0c3148ca.el9.x86_64 2026-02-20T11:25:34.298 INFO:teuthology.orchestra.run.trial038.stdout: libtraceevent-1.8.4-2.el9.x86_64 2026-02-20T11:25:34.298 INFO:teuthology.orchestra.run.trial038.stdout: pdsh-2.34-7.el9.x86_64 2026-02-20T11:25:34.298 INFO:teuthology.orchestra.run.trial038.stdout: pdsh-rcmd-ssh-2.34-7.el9.x86_64 2026-02-20T11:25:34.298 INFO:teuthology.orchestra.run.trial038.stdout: perf-5.14.0-681.el9.x86_64 2026-02-20T11:25:34.298 INFO:teuthology.orchestra.run.trial038.stdout: 2026-02-20T11:25:34.298 INFO:teuthology.orchestra.run.trial038.stdout:Complete! 2026-02-20T11:25:34.328 DEBUG:teuthology.orchestra.run.trial038:> python3 -m pip install -r /home/ubuntu/cephtest/cbt/requirements.txt 2026-02-20T11:25:34.465 INFO:teuthology.orchestra.run.trial038.stdout:Defaulting to user installation because normal site-packages is not writeable 2026-02-20T11:25:34.476 INFO:teuthology.orchestra.run.trial038.stdout:Requirement already satisfied: pyyaml in /usr/lib64/python3.9/site-packages (from -r /home/ubuntu/cephtest/cbt/requirements.txt (line 1)) (5.4.1) 2026-02-20T11:25:34.476 INFO:teuthology.orchestra.run.trial038.stdout:Requirement already satisfied: lxml in /usr/lib64/python3.9/site-packages (from -r /home/ubuntu/cephtest/cbt/requirements.txt (line 2)) (4.6.5) 2026-02-20T11:25:35.315 INFO:teuthology.orchestra.run.trial038.stdout:Collecting matplotlib 2026-02-20T11:25:35.651 INFO:teuthology.orchestra.run.trial038.stdout: Downloading matplotlib-3.9.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.3 MB) 2026-02-20T11:25:36.712 INFO:teuthology.orchestra.run.trial038.stdout:Collecting fonttools>=4.22.0 2026-02-20T11:25:36.795 INFO:teuthology.orchestra.run.trial038.stdout: Downloading fonttools-4.60.2-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.8 MB) 2026-02-20T11:25:36.893 INFO:teuthology.orchestra.run.trial038.stdout:Requirement already satisfied: python-dateutil>=2.7 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (2.9.0.post0) 2026-02-20T11:25:36.893 INFO:teuthology.orchestra.run.trial038.stdout:Requirement already satisfied: pyparsing>=2.3.1 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (2.4.7) 2026-02-20T11:25:36.893 INFO:teuthology.orchestra.run.trial038.stdout:Requirement already satisfied: numpy>=1.23 in /usr/lib64/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (1.23.5) 2026-02-20T11:25:37.130 INFO:teuthology.orchestra.run.trial038.stdout:Collecting kiwisolver>=1.3.1 2026-02-20T11:25:37.219 INFO:teuthology.orchestra.run.trial038.stdout: Downloading kiwisolver-1.4.7-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB) 2026-02-20T11:25:37.487 INFO:teuthology.orchestra.run.trial038.stdout:Collecting contourpy>=1.0.1 2026-02-20T11:25:37.569 INFO:teuthology.orchestra.run.trial038.stdout: Downloading contourpy-1.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (321 kB) 2026-02-20T11:25:37.660 INFO:teuthology.orchestra.run.trial038.stdout:Collecting cycler>=0.10 2026-02-20T11:25:37.740 INFO:teuthology.orchestra.run.trial038.stdout: Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB) 2026-02-20T11:25:37.741 INFO:teuthology.orchestra.run.trial038.stdout:Requirement already satisfied: packaging>=20.0 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (20.9) 2026-02-20T11:25:37.852 INFO:teuthology.orchestra.run.trial038.stdout:Collecting importlib-resources>=3.2.0 2026-02-20T11:25:37.935 INFO:teuthology.orchestra.run.trial038.stdout: Downloading importlib_resources-6.5.2-py3-none-any.whl (37 kB) 2026-02-20T11:25:38.431 INFO:teuthology.orchestra.run.trial038.stdout:Collecting pillow>=8 2026-02-20T11:25:38.515 INFO:teuthology.orchestra.run.trial038.stdout: Downloading pillow-11.3.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (6.6 MB) 2026-02-20T11:25:38.746 INFO:teuthology.orchestra.run.trial038.stdout:Collecting zipp>=3.1.0 2026-02-20T11:25:38.826 INFO:teuthology.orchestra.run.trial038.stdout: Downloading zipp-3.23.0-py3-none-any.whl (10 kB) 2026-02-20T11:25:38.836 INFO:teuthology.orchestra.run.trial038.stdout:Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (1.15.0) 2026-02-20T11:25:38.953 INFO:teuthology.orchestra.run.trial038.stdout:Installing collected packages: zipp, pillow, kiwisolver, importlib-resources, fonttools, cycler, contourpy, matplotlib 2026-02-20T11:25:40.012 INFO:teuthology.orchestra.run.trial038.stdout:Successfully installed contourpy-1.3.0 cycler-0.12.1 fonttools-4.60.2 importlib-resources-6.5.2 kiwisolver-1.4.7 matplotlib-3.9.4 pillow-11.3.0 zipp-3.23.0 2026-02-20T11:25:40.391 INFO:tasks.cbt:benchmark: radosbench 2026-02-20T11:25:40.391 DEBUG:teuthology.orchestra.run.trial038:> sudo /sbin/sysctl -q -w kernel.perf_event_paranoid=0 2026-02-20T11:25:40.416 DEBUG:teuthology.orchestra.run.trial038:> /home/ubuntu/cephtest/cbt/cbt.py -a /home/ubuntu/cephtest/archive/cbt /home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2026-02-20T11:25:40.531 INFO:teuthology.orchestra.run.trial038.stderr:11:25:40 - DEBUG - cbt - Settings.cluster: 2026-02-20T11:25:40.531 INFO:teuthology.orchestra.run.trial038.stderr: {'archive_dir': '/home/ubuntu/cephtest/archive/cbt', 2026-02-20T11:25:40.531 INFO:teuthology.orchestra.run.trial038.stderr: 'clients': ['ubuntu@trial038.front.sepia.ceph.com'], 2026-02-20T11:25:40.531 INFO:teuthology.orchestra.run.trial038.stderr: 'conf_file': '/etc/ceph/ceph.conf', 2026-02-20T11:25:40.531 INFO:teuthology.orchestra.run.trial038.stderr: 'head': 'ubuntu@trial038.front.sepia.ceph.com', 2026-02-20T11:25:40.531 INFO:teuthology.orchestra.run.trial038.stderr: 'is_teuthology': True, 2026-02-20T11:25:40.531 INFO:teuthology.orchestra.run.trial038.stderr: 'iterations': 1, 2026-02-20T11:25:40.531 INFO:teuthology.orchestra.run.trial038.stderr: 'mons': ['ubuntu@trial038.front.sepia.ceph.com'], 2026-02-20T11:25:40.531 INFO:teuthology.orchestra.run.trial038.stderr: 'osds': ['ubuntu@trial038.front.sepia.ceph.com'], 2026-02-20T11:25:40.532 INFO:teuthology.orchestra.run.trial038.stderr: 'osds_per_node': 3, 2026-02-20T11:25:40.532 INFO:teuthology.orchestra.run.trial038.stderr: 'pid_dir': '/var/run/ceph', 2026-02-20T11:25:40.532 INFO:teuthology.orchestra.run.trial038.stderr: 'pool_profiles': {'replicated': {'pg_size': 256, 2026-02-20T11:25:40.532 INFO:teuthology.orchestra.run.trial038.stderr: 'pgp_size': 256, 2026-02-20T11:25:40.532 INFO:teuthology.orchestra.run.trial038.stderr: 'replication': 'replicated'}}, 2026-02-20T11:25:40.532 INFO:teuthology.orchestra.run.trial038.stderr: 'rebuild_every_test': False, 2026-02-20T11:25:40.532 INFO:teuthology.orchestra.run.trial038.stderr: 'rgws': {'ubuntu@trial038.front.sepia.ceph.com': None}, 2026-02-20T11:25:40.532 INFO:teuthology.orchestra.run.trial038.stderr: 'tmp_dir': '/tmp/cbt', 2026-02-20T11:25:40.532 INFO:teuthology.orchestra.run.trial038.stderr: 'use_existing': True, 2026-02-20T11:25:40.532 INFO:teuthology.orchestra.run.trial038.stderr: 'user': 'ubuntu'} 2026-02-20T11:25:40.532 INFO:teuthology.orchestra.run.trial038.stderr:11:25:40 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d 2026-02-20T11:25:40.532 INFO:teuthology.orchestra.run.trial038.stderr:11:25:40 - INFO - cbt - Pausing for 60s for idle monitoring. 2026-02-20T11:25:40.533 INFO:teuthology.orchestra.run.trial038.stderr:11:25:40 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/collectl 2026-02-20T11:25:40.702 INFO:teuthology.orchestra.run.trial038.stderr:11:25:40 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-20T11:25:40.703 INFO:teuthology.orchestra.run.trial038.stderr:11:25:40 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/perf 2026-02-20T11:25:40.865 INFO:teuthology.orchestra.run.trial038.stderr:11:25:40 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-20T11:25:40.865 INFO:teuthology.orchestra.run.trial038.stderr:11:25:40 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/perf/perf_stat.${pid} & done 2026-02-20T11:26:40.890 INFO:teuthology.orchestra.run.trial038.stderr:11:26:40 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-20T11:26:41.083 INFO:teuthology.orchestra.run.trial038.stderr:11:26:41 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-20T11:26:41.283 INFO:teuthology.orchestra.run.trial038.stderr:11:26:41 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/perf/perf.data 2026-02-20T11:26:41.285 INFO:teuthology.orchestra.run.trial038.stderr:11:26:41 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/idle_monitoring/perf/perf_stat.* 2026-02-20T11:26:41.287 INFO:teuthology.orchestra.run.trial038.stderr:11:26:41 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/* 2026-02-20T11:26:41.461 INFO:teuthology.orchestra.run.trial038.stderr:11:26:41 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial038.front.sepia.ceph.com -r /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d 2026-02-20T11:26:41.625 INFO:teuthology.orchestra.run.trial038.stderr:11:26:41 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com sudo killall -9 rados 2026-02-20T11:26:41.802 INFO:teuthology.orchestra.run.trial038.stderr:11:26:41 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d 2026-02-20T11:26:41.802 INFO:teuthology.orchestra.run.trial038.stderr:11:26:41 - INFO - cbt - Skipping existing test in /home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d. 2026-02-20T11:26:41.802 INFO:teuthology.orchestra.run.trial038.stderr:11:26:41 - INFO - cbt - Running benchmark /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004 2026-02-20T11:26:41.803 INFO:teuthology.orchestra.run.trial038.stderr:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-839a075d 2026-02-20T11:26:41.803 INFO:teuthology.orchestra.run.trial038.stderr:{'concurrent_ops': 4, 'concurrent_procs': 2, 'pool_profile': 'replicated', 'time': 60, 'write_only': False, 'op_size': 4194304, 'pool_monitoring_list': 'collectl', 'run_monitoring_list': 'collectl', 'benchmark': 'radosbench', 'iteration': 0} == iteration 0 == 2026-02-20T11:26:41.803 INFO:teuthology.orchestra.run.trial038.stderr:11:26:41 - INFO - cbt - Setting OSD Read Ahead to: 0 2026-02-20T11:26:41.803 INFO:teuthology.orchestra.run.trial038.stderr:11:26:41 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com find /dev/disk/by-partlabel/osd-device-*data -exec readlink {} \; | cut -d"/" -f 3 | sed "s/[0-9]$//" | xargs -I{} sudo sh -c "echo 0 > /sys/block/'{}'/queue/read_ahead_kb" 2026-02-20T11:26:41.803 INFO:teuthology.orchestra.run.trial038.stderr:11:26:41 - DEBUG - cbt - Cleaning existing temporary run directory: /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004 2026-02-20T11:26:41.804 INFO:teuthology.orchestra.run.trial038.stderr:11:26:41 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com sudo rm -rf /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004 2026-02-20T11:26:41.977 INFO:teuthology.orchestra.run.trial038.stderr:11:26:41 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/collectl 2026-02-20T11:26:42.141 INFO:teuthology.orchestra.run.trial038.stderr:11:26:42 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-20T11:26:42.143 INFO:teuthology.orchestra.run.trial038.stderr:11:26:42 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/perf 2026-02-20T11:26:42.312 INFO:teuthology.orchestra.run.trial038.stderr:11:26:42 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-20T11:26:42.312 INFO:teuthology.orchestra.run.trial038.stderr:11:26:42 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/perf/perf_stat.${pid} & done 2026-02-20T11:26:42.314 INFO:teuthology.orchestra.run.trial038.stderr:11:26:42 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool delete rados-bench-cbt rados-bench-cbt --yes-i-really-really-mean-it 2026-02-20T11:26:42.668 INFO:teuthology.orchestra.run.trial038.stderr:11:26:42 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool create rados-bench-cbt 256 256 2026-02-20T11:26:43.199 INFO:teuthology.orchestra.run.trial038.stderr:11:26:43 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool application enable rados-bench-cbt radosbench 2026-02-20T11:26:44.205 INFO:teuthology.orchestra.run.trial038.stderr:11:26:44 - INFO - cbt - Checking Health after pool creation. 2026-02-20T11:26:49.210 INFO:teuthology.orchestra.run.trial038.stderr:11:26:49 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com /usr/bin/ceph -c /etc/ceph/ceph.conf health 2026-02-20T11:26:49.566 INFO:teuthology.orchestra.run.trial038.stderr:11:26:49 - INFO - cbt - Final Pool Health Check. 2026-02-20T11:26:54.572 INFO:teuthology.orchestra.run.trial038.stderr:11:26:54 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com /usr/bin/ceph -c /etc/ceph/ceph.conf health 2026-02-20T11:26:54.936 INFO:teuthology.orchestra.run.trial038.stderr:11:26:54 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-20T11:26:55.113 INFO:teuthology.orchestra.run.trial038.stderr:11:26:55 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-20T11:26:55.292 INFO:teuthology.orchestra.run.trial038.stderr:11:26:55 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/perf/perf.data 2026-02-20T11:26:55.294 INFO:teuthology.orchestra.run.trial038.stderr:11:26:55 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/pool_monitoring/perf/perf_stat.* 2026-02-20T11:26:55.296 INFO:teuthology.orchestra.run.trial038.stderr:11:26:55 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com sync 2026-02-20T11:26:55.459 INFO:teuthology.orchestra.run.trial038.stderr:11:26:55 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com echo 3 | sudo tee /proc/sys/vm/drop_caches 2026-02-20T11:26:56.485 INFO:teuthology.orchestra.run.trial038.stderr:11:26:56 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -v 2026-02-20T11:26:56.696 INFO:teuthology.orchestra.run.trial038.stderr:11:26:56 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write 2026-02-20T11:26:56.861 INFO:teuthology.orchestra.run.trial038.stderr:11:26:56 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf daemon osd.0 config show > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/ceph_settings.out 2026-02-20T11:26:57.157 INFO:teuthology.orchestra.run.trial038.stderr:11:26:57 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/collectl 2026-02-20T11:26:57.323 INFO:teuthology.orchestra.run.trial038.stderr:11:26:57 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-20T11:26:57.324 INFO:teuthology.orchestra.run.trial038.stderr:11:26:57 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/perf 2026-02-20T11:26:57.489 INFO:teuthology.orchestra.run.trial038.stderr:11:26:57 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-20T11:26:57.489 INFO:teuthology.orchestra.run.trial038.stderr:11:26:57 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/perf/perf_stat.${pid} & done 2026-02-20T11:26:57.491 INFO:teuthology.orchestra.run.trial038.stderr:11:26:57 - INFO - cbt - Running radosbench write test. 2026-02-20T11:26:57.491 INFO:teuthology.orchestra.run.trial038.stderr:11:26:57 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench -b 4194304 60 write --concurrent-ios 4 --run-name `hostname -f`-0 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/objecter.0.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/output.0 2026-02-20T11:26:57.493 INFO:teuthology.orchestra.run.trial038.stderr:11:26:57 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial038.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench -b 4194304 60 write --concurrent-ios 4 --run-name `hostname -f`-1 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/objecter.1.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-04194304/concurrent_ops-00000004/write/output.1 2026-02-20T11:27:15.397 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~0s 2026-02-20T11:27:20.800 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~5s 2026-02-20T11:27:26.203 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~11s 2026-02-20T11:27:31.607 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~16s 2026-02-20T11:27:37.010 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~22s 2026-02-20T11:27:42.413 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~27s 2026-02-20T11:27:47.816 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~32s 2026-02-20T11:27:53.218 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~38s 2026-02-20T11:27:58.622 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~43s 2026-02-20T11:28:04.025 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~49s 2026-02-20T11:28:09.427 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~54s 2026-02-20T11:28:14.831 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~59s 2026-02-20T11:28:20.133 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~0s 2026-02-20T11:28:20.134 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~65s 2026-02-20T11:28:25.436 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~5s 2026-02-20T11:28:25.436 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~70s 2026-02-20T11:28:30.739 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~11s 2026-02-20T11:28:30.739 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~75s 2026-02-20T11:28:36.041 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~16s 2026-02-20T11:28:36.041 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~81s 2026-02-20T11:28:41.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~21s 2026-02-20T11:28:41.344 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~86s 2026-02-20T11:28:46.646 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~27s 2026-02-20T11:28:46.647 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~91s 2026-02-20T11:28:51.948 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~32s 2026-02-20T11:28:51.949 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~97s 2026-02-20T11:28:57.250 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~37s 2026-02-20T11:28:57.251 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~102s 2026-02-20T11:29:02.553 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~42s 2026-02-20T11:29:02.553 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~107s 2026-02-20T11:29:07.856 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~48s 2026-02-20T11:29:07.856 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~113s 2026-02-20T11:29:13.159 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~53s 2026-02-20T11:29:13.159 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~118s 2026-02-20T11:29:15.684 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:29:15.682+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:29:18.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~58s 2026-02-20T11:29:18.462 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~123s 2026-02-20T11:29:20.684 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:29:20.682+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:29:23.764 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~64s 2026-02-20T11:29:23.765 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~128s 2026-02-20T11:29:25.685 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:29:25.683+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:29:29.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~69s 2026-02-20T11:29:29.068 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~134s 2026-02-20T11:29:30.685 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:29:30.683+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:29:34.370 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~74s 2026-02-20T11:29:34.371 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~139s 2026-02-20T11:29:35.685 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:29:35.684+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:29:39.673 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~80s 2026-02-20T11:29:39.673 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~144s 2026-02-20T11:29:40.686 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:29:40.684+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:29:44.976 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~85s 2026-02-20T11:29:44.976 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~150s 2026-02-20T11:29:45.686 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:29:45.685+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:29:50.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~90s 2026-02-20T11:29:50.279 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~155s 2026-02-20T11:29:50.686 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:29:50.684+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:29:55.581 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~95s 2026-02-20T11:29:55.581 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~160s 2026-02-20T11:29:55.687 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:29:55.684+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:30:00.687 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:30:00.685+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:30:00.884 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~101s 2026-02-20T11:30:00.884 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~166s 2026-02-20T11:30:05.687 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:30:05.685+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:30:06.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~106s 2026-02-20T11:30:06.186 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~171s 2026-02-20T11:30:10.688 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:30:10.686+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:30:11.489 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~111s 2026-02-20T11:30:11.489 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~176s 2026-02-20T11:30:15.688 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:30:15.686+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:30:16.792 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~117s 2026-02-20T11:30:16.792 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~181s 2026-02-20T11:30:20.689 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:30:20.688+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:30:22.094 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~122s 2026-02-20T11:30:22.095 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~187s 2026-02-20T11:30:25.689 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:30:25.687+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:30:27.398 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~127s 2026-02-20T11:30:27.398 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~192s 2026-02-20T11:30:30.690 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:30:30.688+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:30:32.700 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~133s 2026-02-20T11:30:32.700 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~197s 2026-02-20T11:30:35.690 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:30:35.688+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:30:38.003 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~138s 2026-02-20T11:30:38.003 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~203s 2026-02-20T11:30:40.691 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:30:40.689+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:30:43.306 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~143s 2026-02-20T11:30:43.306 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~208s 2026-02-20T11:30:45.691 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:30:45.689+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:30:48.609 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~148s 2026-02-20T11:30:48.609 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~213s 2026-02-20T11:30:50.692 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:30:50.690+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:30:53.911 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~154s 2026-02-20T11:30:53.911 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~219s 2026-02-20T11:30:55.692 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:30:55.690+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:30:59.215 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~159s 2026-02-20T11:30:59.215 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~224s 2026-02-20T11:31:00.693 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:31:00.690+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:31:04.517 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~164s 2026-02-20T11:31:04.517 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~229s 2026-02-20T11:31:05.693 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:31:05.691+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:31:09.819 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~170s 2026-02-20T11:31:09.820 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~235s 2026-02-20T11:31:10.693 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:31:10.691+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:31:15.122 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~175s 2026-02-20T11:31:15.122 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~240s 2026-02-20T11:31:15.694 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:31:15.692+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:31:20.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~180s 2026-02-20T11:31:20.425 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~245s 2026-02-20T11:31:20.694 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:31:20.693+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:31:25.695 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:31:25.693+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:31:25.727 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~186s 2026-02-20T11:31:25.728 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~250s 2026-02-20T11:31:30.695 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:31:30.693+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:31:31.029 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~191s 2026-02-20T11:31:31.030 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~256s 2026-02-20T11:31:35.695 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:31:35.693+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:31:36.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~196s 2026-02-20T11:31:36.332 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~261s 2026-02-20T11:31:40.696 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:31:40.694+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:31:41.635 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~202s 2026-02-20T11:31:41.635 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~266s 2026-02-20T11:31:45.696 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:31:45.694+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:31:46.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~207s 2026-02-20T11:31:46.938 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~272s 2026-02-20T11:31:50.697 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:31:50.695+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:31:52.240 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~212s 2026-02-20T11:31:52.241 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~277s 2026-02-20T11:31:55.697 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:31:55.696+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:31:57.543 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~217s 2026-02-20T11:31:57.543 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~282s 2026-02-20T11:32:00.698 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:32:00.697+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:32:02.846 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~223s 2026-02-20T11:32:02.846 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~288s 2026-02-20T11:32:05.698 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:32:05.696+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:32:08.148 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~228s 2026-02-20T11:32:08.149 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~293s 2026-02-20T11:32:10.699 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:32:10.697+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:32:13.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~233s 2026-02-20T11:32:13.452 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~298s 2026-02-20T11:32:15.699 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:32:15.697+0000 7f847c9f6640 -1 mon.a@0(leader) e1 get_health_metrics reporting 1 slow ops, oldest is osd_failure(failed timeout osd.0 v2:10.20.193.38:6807/1023039484 for 29sec e27 v27) 2026-02-20T11:32:18.755 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.0 is failed for ~239s 2026-02-20T11:32:18.755 INFO:tasks.daemonwatchdog.daemon_watchdog:daemon ceph.osd.2 is failed for ~303s 2026-02-20T11:32:18.755 INFO:tasks.daemonwatchdog.daemon_watchdog:BARK! unmounting mounts and killing all daemons 2026-02-20T11:32:19.058 INFO:tasks.ceph.osd.1:Sent signal 15 2026-02-20T11:32:19.058 INFO:tasks.ceph.mon.a:Sent signal 15 2026-02-20T11:32:19.058 INFO:tasks.ceph.mgr.x:Sent signal 15 2026-02-20T11:32:19.059 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:32:19.057+0000 7f847f9fc640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mon -f --cluster ceph -i a (PID: 39941) UID: 0 2026-02-20T11:32:19.059 INFO:tasks.ceph.mon.a.trial038.stderr:2026-02-20T11:32:19.057+0000 7f847f9fc640 -1 mon.a@0(leader) e1 *** Got Signal Terminated *** 2026-02-20T11:32:19.060 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-20T11:32:19.057+0000 7f4cf73f8640 -1 received signal: Terminated from /usr/bin/python3 /bin/daemon-helper kill ceph-mgr -f --cluster ceph -i x (PID: 39945) UID: 0 2026-02-20T11:32:19.060 INFO:tasks.ceph.mgr.x.trial038.stderr:2026-02-20T11:32:19.057+0000 7f4cf73f8640 -1 mgr handle_mgr_signal *** Got signal Terminated *** 2026-02-20T17:16:51.067 DEBUG:teuthology.exit:Got signal 15; running 1 handler... 2026-02-20T17:16:51.085 DEBUG:teuthology.task.console_log:Killing console logger for trial038 2026-02-20T17:16:51.086 DEBUG:teuthology.exit:Finished running handlers