2026-02-17T09:54:11.828 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-17T09:54:11.845 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T09:54:11.926 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/matan-2026-02-17_08:51:22-crimson-rados-wip-sjust-seastore-2026-02-16-1771285632-distro-debug-trial/52100 branch: wip-sjust-seastore-2026-02-16-1771285632 description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} perf_ignorelist settings/optimized workloads/radosbench_4K_rand_read} email: null first_in_suite: false flavor: debug job_id: '52100' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: matan-2026-02-17_08:51:22-crimson-rados-wip-sjust-seastore-2026-02-16-1771285632-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: wip-sjust-seastore-2026-02-16-1771285632 ceph: conf: global: auth client required: none auth cluster required: none auth service required: none auth supported: none debug asok: 0/0 debug auth: 0/0 debug bdev: 0/0 debug bluefs: 0/0 debug bluestore: 0/0 debug buffer: 0/0 debug civetweb: 0/0 debug client: 0/0 debug compressor: 0/0 debug context: 0/0 debug crush: 0/0 debug crypto: 0/0 debug dpdk: 0/0 debug eventtrace: 0/0 debug filer: 0/0 debug filestore: 0/0 debug finisher: 0/0 debug fuse: 0/0 debug heartbeatmap: 0/0 debug javaclient: 0/0 debug journal: 0/0 debug journaler: 0/0 debug kstore: 0/0 debug leveldb: 0/0 debug lockdep: 0/0 debug mds: 0/0 debug mds balancer: 0/0 debug mds locker: 0/0 debug mds log: 0/0 debug mds log expire: 0/0 debug mds migrator: 0/0 debug memdb: 0/0 debug mgr: 0/0 debug mgrc: 0/0 debug mon: 0/0 debug monc: 0/0 debug ms: 0/0 debug objclass: 0/0 debug objectcacher: 0/0 debug objecter: 0/0 debug optracker: 0/0 debug osd: 0/0 debug paxos: 0/0 debug perfcounter: 0/0 debug rados: 0/0 debug rbd: 0/0 debug rbd mirror: 0/0 debug rbd replay: 0/0 debug refs: 0/0 debug rgw: 0/0 debug rgw lifecycle: 0/0 debug rgw notification: 0/0 debug rgw sync: 0/0 debug rocksdb: 0/0 debug striper: 0/0 debug throttle: 0/0 debug timer: 0/0 debug tp: 0/0 enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 0/0 debug ms: 0/0 debug paxos: 0/0 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug filestore: 0/0 debug journal: 0/0 debug ms: 0/0 debug osd: 0/0 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore main device type: RANDOM_BLOCK_SSD seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) - \(PG_ - \(OSD_ - \(OBJECT_ - overall HEALTH sha1: b58fab16a61897dbc214a1865300c7cef586ea65 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log global: osd crush chooseleaf type: 0 osd pool default pg num: 128 osd pool default pgp num: 128 osd pool default size: 2 mon: {} install: ceph: flavor: debug sha1: b58fab16a61897dbc214a1865300c7cef586ea65 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: ebc89147fc1a1731b84ef7e7003e19f704ba15c0 owner: scheduled_matan@soko04.front.sepia.ceph.com priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - mgr.x - osd.0 - osd.1 - osd.2 - client.0 seed: 2314 sha1: b58fab16a61897dbc214a1865300c7cef586ea65 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_ebc89147fc1a1731b84ef7e7003e19f704ba15c0/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: ebc89147fc1a1731b84ef7e7003e19f704ba15c0 targets: trial103.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCcwH+06X39oKcH6RUHgOLQ7XUqt/xbjS5DGxwrZ+qCsHpgSxFnPJQ9Fcw3+GPzTyIQNOZIkxDmFVn1EEahoK6U= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: osd: debug monc: 20 wait-for-scrub: false - ssh_keys: null - cbt: benchmarks: radosbench: concurrent_ops: 4 concurrent_procs: 2 op_size: - 4096 pool_monitoring_list: - collectl pool_profile: replicated readmode: rand run_monitoring_list: - collectl time: 60 write_only: false cluster: iterations: 1 osds_per_node: 3 pool_profiles: replicated: pg_size: 256 pgp_size: 256 replication: replicated user: ubuntu monitoring_profiles: perf: args: stat -p {pid} -o {perf_dir}/perf_stat.{pid} nodes: - osds perf_cmd: perf pid_glob: crimson-osd.*.pid teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-17_08:51:22 tube: trial user: matan verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3682 2026-02-17T09:54:11.926 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_ebc89147fc1a1731b84ef7e7003e19f704ba15c0/qa; will attempt to use it 2026-02-17T09:54:11.926 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_ebc89147fc1a1731b84ef7e7003e19f704ba15c0/qa/tasks 2026-02-17T09:54:11.926 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-17T09:54:11.928 INFO:teuthology.task.internal:Checking packages... 2026-02-17T09:54:11.928 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash 'b58fab16a61897dbc214a1865300c7cef586ea65' 2026-02-17T09:54:11.928 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-17T09:54:11.928 INFO:teuthology.packaging:ref: None 2026-02-17T09:54:11.928 INFO:teuthology.packaging:tag: None 2026-02-17T09:54:11.928 INFO:teuthology.packaging:branch: wip-sjust-seastore-2026-02-16-1771285632 2026-02-17T09:54:11.928 INFO:teuthology.packaging:sha1: b58fab16a61897dbc214a1865300c7cef586ea65 2026-02-17T09:54:11.928 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=wip-sjust-seastore-2026-02-16-1771285632 2026-02-17T09:54:12.050 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5345.gb58fab16 2026-02-17T09:54:12.052 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-17T09:54:12.063 INFO:teuthology.task.internal:no buildpackages task found 2026-02-17T09:54:12.063 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-17T09:54:12.076 INFO:teuthology.task.internal:Saving configuration 2026-02-17T09:54:12.089 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-17T09:54:12.101 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-17T09:54:12.173 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial103.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/matan-2026-02-17_08:51:22-crimson-rados-wip-sjust-seastore-2026-02-16-1771285632-distro-debug-trial/52100', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-17 09:50:49.444765', 'locked_by': 'scheduled_matan@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCcwH+06X39oKcH6RUHgOLQ7XUqt/xbjS5DGxwrZ+qCsHpgSxFnPJQ9Fcw3+GPzTyIQNOZIkxDmFVn1EEahoK6U='} 2026-02-17T09:54:12.173 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-17T09:54:12.185 INFO:teuthology.task.internal:roles: ubuntu@trial103.front.sepia.ceph.com - ['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0'] 2026-02-17T09:54:12.185 INFO:teuthology.run_tasks:Running task console_log... 2026-02-17T09:54:12.259 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7640907b3d80>, signals=[15]) 2026-02-17T09:54:12.259 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-17T09:54:12.271 INFO:teuthology.task.internal:Opening connections... 2026-02-17T09:54:12.271 DEBUG:teuthology.task.internal:connecting to ubuntu@trial103.front.sepia.ceph.com 2026-02-17T09:54:12.272 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T09:54:12.336 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-17T09:54:12.348 DEBUG:teuthology.orchestra.run.trial103:> uname -m 2026-02-17T09:54:12.413 INFO:teuthology.orchestra.run.trial103.stdout:x86_64 2026-02-17T09:54:12.413 DEBUG:teuthology.orchestra.run.trial103:> cat /etc/os-release 2026-02-17T09:54:12.473 INFO:teuthology.orchestra.run.trial103.stdout:NAME="CentOS Stream" 2026-02-17T09:54:12.473 INFO:teuthology.orchestra.run.trial103.stdout:VERSION="9" 2026-02-17T09:54:12.473 INFO:teuthology.orchestra.run.trial103.stdout:ID="centos" 2026-02-17T09:54:12.473 INFO:teuthology.orchestra.run.trial103.stdout:ID_LIKE="rhel fedora" 2026-02-17T09:54:12.473 INFO:teuthology.orchestra.run.trial103.stdout:VERSION_ID="9" 2026-02-17T09:54:12.473 INFO:teuthology.orchestra.run.trial103.stdout:PLATFORM_ID="platform:el9" 2026-02-17T09:54:12.473 INFO:teuthology.orchestra.run.trial103.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-17T09:54:12.473 INFO:teuthology.orchestra.run.trial103.stdout:ANSI_COLOR="0;31" 2026-02-17T09:54:12.473 INFO:teuthology.orchestra.run.trial103.stdout:LOGO="fedora-logo-icon" 2026-02-17T09:54:12.473 INFO:teuthology.orchestra.run.trial103.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-17T09:54:12.473 INFO:teuthology.orchestra.run.trial103.stdout:HOME_URL="https://centos.org/" 2026-02-17T09:54:12.473 INFO:teuthology.orchestra.run.trial103.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-17T09:54:12.473 INFO:teuthology.orchestra.run.trial103.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-17T09:54:12.473 INFO:teuthology.orchestra.run.trial103.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-17T09:54:12.474 INFO:teuthology.lock.ops:Updating trial103.front.sepia.ceph.com on lock server 2026-02-17T09:54:12.567 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-17T09:54:12.580 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-17T09:54:12.592 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-17T09:54:12.592 DEBUG:teuthology.orchestra.run.trial103:> test '!' -e /home/ubuntu/cephtest 2026-02-17T09:54:12.607 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-17T09:54:12.619 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-17T09:54:12.619 DEBUG:teuthology.orchestra.run.trial103:> test -z $(ls -A /var/lib/ceph) 2026-02-17T09:54:12.668 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-17T09:54:12.778 INFO:teuthology.run_tasks:Running task kernel... 2026-02-17T09:54:12.803 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-17T09:54:12.803 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-17T09:54:12.803 DEBUG:teuthology.orchestra.run.trial103:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-17T09:54:12.817 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T09:54:12.818 DEBUG:teuthology.orchestra.run.trial103:> uname -r 2026-02-17T09:54:12.877 INFO:teuthology.orchestra.run.trial103.stdout:5.14.0-645.el9.x86_64 2026-02-17T09:54:12.877 INFO:teuthology.task.kernel:Running kernel on trial103: 5.14.0-645.el9.x86_64 2026-02-17T09:54:12.878 DEBUG:teuthology.orchestra.run.trial103:> sudo yum install -y kernel 2026-02-17T09:54:14.818 INFO:teuthology.orchestra.run.trial103.stdout:CentOS Stream 9 - BaseOS 6.1 MB/s | 8.9 MB 00:01 2026-02-17T09:54:17.884 INFO:teuthology.orchestra.run.trial103.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-17T09:54:25.570 INFO:teuthology.orchestra.run.trial103.stdout:CentOS Stream 9 - CRB 1.5 MB/s | 7.8 MB 00:05 2026-02-17T09:54:27.218 INFO:teuthology.orchestra.run.trial103.stdout:CentOS Stream 9 - Extras packages 22 kB/s | 20 kB 00:00 2026-02-17T09:54:28.127 INFO:teuthology.orchestra.run.trial103.stdout:Extra Packages for Enterprise Linux 24 MB/s | 20 MB 00:00 2026-02-17T09:54:31.257 INFO:teuthology.orchestra.run.trial103.stdout:lab-extras 2.4 MB/s | 50 kB 00:00 2026-02-17T09:54:32.267 INFO:teuthology.orchestra.run.trial103.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-17T09:54:32.287 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T09:54:32.288 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T09:54:32.288 INFO:teuthology.orchestra.run.trial103.stdout: Package Architecture Version Repository Size 2026-02-17T09:54:32.289 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T09:54:32.289 INFO:teuthology.orchestra.run.trial103.stdout:Installing: 2026-02-17T09:54:32.289 INFO:teuthology.orchestra.run.trial103.stdout: kernel x86_64 5.14.0-681.el9 baseos 803 k 2026-02-17T09:54:32.289 INFO:teuthology.orchestra.run.trial103.stdout:Installing dependencies: 2026-02-17T09:54:32.289 INFO:teuthology.orchestra.run.trial103.stdout: kernel-core x86_64 5.14.0-681.el9 baseos 17 M 2026-02-17T09:54:32.289 INFO:teuthology.orchestra.run.trial103.stdout: kernel-modules x86_64 5.14.0-681.el9 baseos 40 M 2026-02-17T09:54:32.289 INFO:teuthology.orchestra.run.trial103.stdout: kernel-modules-core x86_64 5.14.0-681.el9 baseos 31 M 2026-02-17T09:54:32.289 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T09:54:32.289 INFO:teuthology.orchestra.run.trial103.stdout:Transaction Summary 2026-02-17T09:54:32.289 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T09:54:32.290 INFO:teuthology.orchestra.run.trial103.stdout:Install 4 Packages 2026-02-17T09:54:32.290 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T09:54:32.290 INFO:teuthology.orchestra.run.trial103.stdout:Total download size: 88 M 2026-02-17T09:54:32.290 INFO:teuthology.orchestra.run.trial103.stdout:Installed size: 135 M 2026-02-17T09:54:32.290 INFO:teuthology.orchestra.run.trial103.stdout:Downloading Packages: 2026-02-17T09:54:32.719 INFO:teuthology.orchestra.run.trial103.stdout:(1/4): kernel-5.14.0-681.el9.x86_64.rpm 6.2 MB/s | 803 kB 00:00 2026-02-17T09:54:32.837 INFO:teuthology.orchestra.run.trial103.stdout:(2/4): kernel-core-5.14.0-681.el9.x86_64.rpm 70 MB/s | 17 MB 00:00 2026-02-17T09:54:32.882 INFO:teuthology.orchestra.run.trial103.stdout:(3/4): kernel-modules-core-5.14.0-681.el9.x86_6 188 MB/s | 31 MB 00:00 2026-02-17T09:54:32.950 INFO:teuthology.orchestra.run.trial103.stdout:(4/4): kernel-modules-5.14.0-681.el9.x86_64.rpm 111 MB/s | 40 MB 00:00 2026-02-17T09:54:32.950 INFO:teuthology.orchestra.run.trial103.stdout:-------------------------------------------------------------------------------- 2026-02-17T09:54:32.950 INFO:teuthology.orchestra.run.trial103.stdout:Total 134 MB/s | 88 MB 00:00 2026-02-17T09:54:33.138 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction check 2026-02-17T09:54:33.337 INFO:teuthology.orchestra.run.trial103.stdout:Transaction check succeeded. 2026-02-17T09:54:33.337 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction test 2026-02-17T09:54:33.688 INFO:teuthology.orchestra.run.trial103.stdout:Transaction test succeeded. 2026-02-17T09:54:33.688 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction 2026-02-17T09:54:34.773 INFO:teuthology.orchestra.run.trial103.stdout: Preparing : 1/1 2026-02-17T09:54:35.055 INFO:teuthology.orchestra.run.trial103.stdout: Installing : kernel-modules-core-5.14.0-681.el9.x86_64 1/4 2026-02-17T09:54:35.074 INFO:teuthology.orchestra.run.trial103.stdout: Installing : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T09:54:36.855 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T09:54:36.885 INFO:teuthology.orchestra.run.trial103.stdout: Installing : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T09:54:39.908 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T09:54:39.915 INFO:teuthology.orchestra.run.trial103.stdout: Installing : kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T09:54:42.945 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T09:55:13.375 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: kernel-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T09:55:13.396 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: kernel-modules-5.14.0-681.el9.x86_64 4/4 2026-02-17T09:55:13.461 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: kernel-5.14.0-681.el9.x86_64 4/4 2026-02-17T09:55:13.461 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : kernel-5.14.0-681.el9.x86_64 1/4 2026-02-17T09:55:13.461 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : kernel-core-5.14.0-681.el9.x86_64 2/4 2026-02-17T09:55:13.461 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : kernel-modules-5.14.0-681.el9.x86_64 3/4 2026-02-17T09:55:13.513 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : kernel-modules-core-5.14.0-681.el9.x86_64 4/4 2026-02-17T09:55:13.513 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T09:55:13.513 INFO:teuthology.orchestra.run.trial103.stdout:Installed: 2026-02-17T09:55:13.514 INFO:teuthology.orchestra.run.trial103.stdout: kernel-5.14.0-681.el9.x86_64 2026-02-17T09:55:13.514 INFO:teuthology.orchestra.run.trial103.stdout: kernel-core-5.14.0-681.el9.x86_64 2026-02-17T09:55:13.514 INFO:teuthology.orchestra.run.trial103.stdout: kernel-modules-5.14.0-681.el9.x86_64 2026-02-17T09:55:13.514 INFO:teuthology.orchestra.run.trial103.stdout: kernel-modules-core-5.14.0-681.el9.x86_64 2026-02-17T09:55:13.514 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T09:55:13.514 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T09:55:13.574 DEBUG:teuthology.orchestra.run.trial103:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T09:55:13.600 INFO:teuthology.orchestra.run.trial103.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T09:55:13.602 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T09:55:13.602 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-645.el9.x86_64 Expected: 5.14.0-681.el9.x86_64 2026-02-17T09:55:13.602 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-17T09:55:13.602 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-17T09:55:13.602 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-17T09:55:13.602 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial103.front.sepia.ceph.com, path=None, version=distro) 2026-02-17T09:55:13.602 DEBUG:teuthology.orchestra.run.trial103:> rpm -q kernel | sort -rV | head -n 1 2026-02-17T09:55:13.665 INFO:teuthology.orchestra.run.trial103.stdout:kernel-5.14.0-681.el9.x86_64 2026-02-17T09:55:13.666 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-681.el9.x86_64 2026-02-17T09:55:13.667 DEBUG:teuthology.orchestra.run.trial103:> sudo rpm -qi grub2-tools 2026-02-17T09:55:13.736 INFO:teuthology.orchestra.run.trial103.stdout:Name : grub2-tools 2026-02-17T09:55:13.736 INFO:teuthology.orchestra.run.trial103.stdout:Epoch : 1 2026-02-17T09:55:13.736 INFO:teuthology.orchestra.run.trial103.stdout:Version : 2.06 2026-02-17T09:55:13.736 INFO:teuthology.orchestra.run.trial103.stdout:Release : 118.el9 2026-02-17T09:55:13.737 INFO:teuthology.orchestra.run.trial103.stdout:Architecture: x86_64 2026-02-17T09:55:13.737 INFO:teuthology.orchestra.run.trial103.stdout:Install Date: Thu 04 Dec 2025 11:54:21 PM UTC 2026-02-17T09:55:13.737 INFO:teuthology.orchestra.run.trial103.stdout:Group : Unspecified 2026-02-17T09:55:13.737 INFO:teuthology.orchestra.run.trial103.stdout:Size : 8393909 2026-02-17T09:55:13.737 INFO:teuthology.orchestra.run.trial103.stdout:License : GPLv3+ 2026-02-17T09:55:13.737 INFO:teuthology.orchestra.run.trial103.stdout:Signature : RSA/SHA256, Wed 05 Nov 2025 09:13:15 AM UTC, Key ID 05b555b38483c65d 2026-02-17T09:55:13.737 INFO:teuthology.orchestra.run.trial103.stdout:Source RPM : grub2-2.06-118.el9.src.rpm 2026-02-17T09:55:13.737 INFO:teuthology.orchestra.run.trial103.stdout:Build Date : Tue 04 Nov 2025 03:44:14 PM UTC 2026-02-17T09:55:13.737 INFO:teuthology.orchestra.run.trial103.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-17T09:55:13.737 INFO:teuthology.orchestra.run.trial103.stdout:Packager : builder@centos.org 2026-02-17T09:55:13.737 INFO:teuthology.orchestra.run.trial103.stdout:Vendor : CentOS 2026-02-17T09:55:13.737 INFO:teuthology.orchestra.run.trial103.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-17T09:55:13.738 INFO:teuthology.orchestra.run.trial103.stdout:Summary : Support tools for GRUB. 2026-02-17T09:55:13.738 INFO:teuthology.orchestra.run.trial103.stdout:Description : 2026-02-17T09:55:13.738 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T09:55:13.738 INFO:teuthology.orchestra.run.trial103.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-17T09:55:13.738 INFO:teuthology.orchestra.run.trial103.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-17T09:55:13.738 INFO:teuthology.orchestra.run.trial103.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-17T09:55:13.738 INFO:teuthology.orchestra.run.trial103.stdout:hardware devices. 2026-02-17T09:55:13.738 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T09:55:13.738 INFO:teuthology.orchestra.run.trial103.stdout:This subpackage provides tools for support of all platforms. 2026-02-17T09:55:13.739 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-17T09:55:13.740 INFO:teuthology.task.kernel:Updating grub on trial103 to boot 5.14.0-681.el9.x86_64 2026-02-17T09:55:13.740 DEBUG:teuthology.orchestra.run.trial103:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-17T09:55:13.805 INFO:teuthology.orchestra.run.trial103.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T09:55:13.806 DEBUG:teuthology.orchestra.run.trial103:> sudo command -v grubby && echo yes || echo no 2026-02-17T09:55:13.870 INFO:teuthology.orchestra.run.trial103.stdout:/sbin/grubby 2026-02-17T09:55:13.872 INFO:teuthology.orchestra.run.trial103.stdout:yes 2026-02-17T09:55:13.872 DEBUG:teuthology.orchestra.run.trial103:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-681.el9.x86_64" | head -n 1 2026-02-17T09:55:13.936 INFO:teuthology.orchestra.run.trial103.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-17T09:55:13.936 DEBUG:teuthology.orchestra.run.trial103:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64 2026-02-17T09:55:14.010 DEBUG:teuthology.orchestra.run.trial103:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-17T09:55:14.220 INFO:teuthology.orchestra.run.trial103.stderr:Generating grub configuration file ... 2026-02-17T09:55:14.282 INFO:teuthology.orchestra.run.trial103.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-17T09:55:14.289 INFO:teuthology.orchestra.run.trial103.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-17T09:55:14.540 INFO:teuthology.orchestra.run.trial103.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-17T09:55:14.546 INFO:teuthology.orchestra.run.trial103.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-17T09:55:14.579 INFO:teuthology.orchestra.run.trial103.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-17T09:55:14.585 INFO:teuthology.orchestra.run.trial103.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-17T09:55:14.608 INFO:teuthology.orchestra.run.trial103.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-17T09:55:14.614 INFO:teuthology.orchestra.run.trial103.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-17T09:55:14.695 INFO:teuthology.orchestra.run.trial103.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-17T09:55:14.703 INFO:teuthology.orchestra.run.trial103.stderr:done 2026-02-17T09:55:14.705 DEBUG:teuthology.orchestra.run.trial103:> test -d /sys/firmware/efi 2026-02-17T09:55:14.721 DEBUG:teuthology.orchestra.run.trial103:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-17T09:55:14.786 INFO:teuthology.orchestra.run.trial103.stdout:/boot/efi/EFI/BOOT 2026-02-17T09:55:14.787 INFO:teuthology.orchestra.run.trial103.stdout:/boot/efi/EFI/centos 2026-02-17T09:55:14.788 DEBUG:teuthology.orchestra.run.trial103:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-17T09:55:14.852 INFO:teuthology.orchestra.run.trial103.stdout:no 2026-02-17T09:55:14.853 DEBUG:teuthology.orchestra.run.trial103:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-17T09:55:14.917 INFO:teuthology.orchestra.run.trial103.stdout:yes 2026-02-17T09:55:14.918 DEBUG:teuthology.orchestra.run.trial103:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-17T09:55:14.985 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial103 2026-02-17T09:55:14.985 DEBUG:teuthology.orchestra.run.trial103:> sudo bash -c ' 2026-02-17T09:55:14.985 DEBUG:teuthology.orchestra.run.trial103:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-17T09:55:14.985 DEBUG:teuthology.orchestra.run.trial103:> ' 2026-02-17T09:55:45.028 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-17T09:55:45.029 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial103.front.sepia.ceph.com' 2026-02-17T09:55:45.030 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T09:56:45.091 DEBUG:teuthology.orchestra.remote:timed out 2026-02-17T09:56:54.101 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial103.front.sepia.ceph.com' 2026-02-17T09:56:54.102 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T09:56:54.207 DEBUG:teuthology.orchestra.run.trial103:> true 2026-02-17T09:56:54.361 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial103.front.sepia.ceph.com' 2026-02-17T09:56:54.361 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-681.el9.x86_64"... 2026-02-17T09:56:54.361 DEBUG:teuthology.orchestra.run.trial103:> uname -r 2026-02-17T09:56:54.417 INFO:teuthology.orchestra.run.trial103.stdout:5.14.0-681.el9.x86_64 2026-02-17T09:56:54.418 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-681.el9.x86_64 vs 5.14.0-681.el9.x86_64 2026-02-17T09:56:54.418 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-17T09:56:54.418 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-17T09:56:55.419 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-17T09:56:55.420 DEBUG:teuthology.orchestra.run.trial103:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-17T09:56:55.449 INFO:teuthology.orchestra.run.trial103.stdout:ttyS1 2026-02-17T09:56:55.471 DEBUG:teuthology.parallel:result is None 2026-02-17T09:56:55.471 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-17T09:56:55.484 INFO:teuthology.task.internal:Creating test directory... 2026-02-17T09:56:55.484 DEBUG:teuthology.orchestra.run.trial103:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-17T09:56:55.527 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-17T09:56:55.539 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-17T09:56:55.553 INFO:teuthology.task.internal:Creating archive directory... 2026-02-17T09:56:55.553 DEBUG:teuthology.orchestra.run.trial103:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-17T09:56:55.590 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-17T09:56:55.602 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-17T09:56:55.602 DEBUG:teuthology.orchestra.run.trial103:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-17T09:56:55.648 INFO:teuthology.orchestra.run.trial103.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T09:56:55.656 INFO:teuthology.orchestra.run.trial103.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-17T09:56:55.658 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-17T09:56:55.670 INFO:teuthology.task.internal:Configuring sudo... 2026-02-17T09:56:55.671 DEBUG:teuthology.orchestra.run.trial103:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-17T09:56:55.722 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-17T09:56:55.739 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-17T09:56:55.739 DEBUG:teuthology.orchestra.run.trial103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-17T09:56:55.780 DEBUG:teuthology.orchestra.run.trial103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T09:56:55.942 DEBUG:teuthology.orchestra.run.trial103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T09:56:55.967 DEBUG:teuthology.orchestra.run.trial103:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T09:56:56.113 DEBUG:teuthology.orchestra.run.trial103:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T09:56:56.138 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T09:56:56.138 DEBUG:teuthology.orchestra.run.trial103:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-17T09:56:56.202 DEBUG:teuthology.orchestra.run.trial103:> sudo service rsyslog restart 2026-02-17T09:56:56.270 INFO:teuthology.orchestra.run.trial103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T09:56:56.668 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-17T09:56:56.681 INFO:teuthology.task.internal:Starting timer... 2026-02-17T09:56:56.682 INFO:teuthology.run_tasks:Running task pcp... 2026-02-17T09:56:56.704 INFO:teuthology.run_tasks:Running task selinux... 2026-02-17T09:56:56.722 DEBUG:teuthology.orchestra.run.trial103:> sudo service auditd rotate 2026-02-17T09:56:56.751 INFO:teuthology.orchestra.run.trial103.stdout:Rotating logs: 2026-02-17T09:56:56.753 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-17T09:56:56.753 DEBUG:teuthology.orchestra.run.trial103:> /usr/sbin/getenforce 2026-02-17T09:56:56.811 INFO:teuthology.orchestra.run.trial103.stdout:Permissive 2026-02-17T09:56:56.811 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial103.front.sepia.ceph.com': 'permissive'} 2026-02-17T09:56:56.812 DEBUG:teuthology.orchestra.run.trial103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-17T09:56:56.877 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T09:56:56.877 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-17T09:56:56.878 DEBUG:teuthology.orchestra.run.trial103:> sudo /usr/sbin/setenforce permissive 2026-02-17T09:56:56.941 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-17T09:56:56.958 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-17T09:56:56.973 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-17T09:56:56.974 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial103.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-17T09:59:44.954 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial103.front.sepia.ceph.com')] 2026-02-17T09:59:44.955 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial103.front.sepia.ceph.com' 2026-02-17T09:59:44.957 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial103.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-17T09:59:45.066 DEBUG:teuthology.orchestra.run.trial103:> true 2026-02-17T09:59:45.141 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial103.front.sepia.ceph.com' 2026-02-17T09:59:45.141 INFO:teuthology.run_tasks:Running task clock... 2026-02-17T09:59:45.155 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-17T09:59:45.155 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-17T09:59:45.156 DEBUG:teuthology.orchestra.run.trial103:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T09:59:45.209 INFO:teuthology.orchestra.run.trial103.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-17T09:59:45.221 INFO:teuthology.orchestra.run.trial103.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-17T09:59:45.241 INFO:teuthology.orchestra.run.trial103.stderr:sudo: ntpd: command not found 2026-02-17T09:59:45.250 INFO:teuthology.orchestra.run.trial103.stdout:506 Cannot talk to daemon 2026-02-17T09:59:45.262 INFO:teuthology.orchestra.run.trial103.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-17T09:59:45.274 INFO:teuthology.orchestra.run.trial103.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-17T09:59:45.327 INFO:teuthology.orchestra.run.trial103.stderr:bash: line 1: ntpq: command not found 2026-02-17T09:59:45.331 INFO:teuthology.orchestra.run.trial103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T09:59:45.332 INFO:teuthology.orchestra.run.trial103.stdout:=============================================================================== 2026-02-17T09:59:45.332 INFO:teuthology.orchestra.run.trial103.stdout:^? lax1.us.ntp.li 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-17T09:59:45.333 INFO:teuthology.run_tasks:Running task install... 2026-02-17T09:59:45.347 DEBUG:teuthology.task.install:project ceph 2026-02-17T09:59:45.347 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': 'b58fab16a61897dbc214a1865300c7cef586ea65'}} 2026-02-17T09:59:45.347 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': 'b58fab16a61897dbc214a1865300c7cef586ea65'} 2026-02-17T09:59:45.347 INFO:teuthology.task.install:Using flavor: debug 2026-02-17T09:59:45.357 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-17T09:59:45.357 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-17T09:59:45.357 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b58fab16a61897dbc214a1865300c7cef586ea65', 'tag': None, 'wait_for_package': False} 2026-02-17T09:59:45.358 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=b58fab16a61897dbc214a1865300c7cef586ea65 2026-02-17T09:59:45.519 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-sjust-seastore-2026-02-16-1771285632/b58fab16a61897dbc214a1865300c7cef586ea65/centos/9/flavors/debug/ 2026-02-17T09:59:45.519 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5345.gb58fab16 2026-02-17T09:59:45.829 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-sjust-seastore-2026-02-16-1771285632/b58fab16a61897dbc214a1865300c7cef586ea65/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-sjust-seastore-2026-02-16-1771285632/b58fab16a61897dbc214a1865300c7cef586ea65/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-sjust-seastore-2026-02-16-1771285632/b58fab16a61897dbc214a1865300c7cef586ea65/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-17T09:59:45.830 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T09:59:45.830 DEBUG:teuthology.orchestra.run.trial103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-17T09:59:45.855 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-17T09:59:45.855 DEBUG:teuthology.orchestra.run.trial103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b58fab16a61897dbc214a1865300c7cef586ea65/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-17T09:59:45.923 DEBUG:teuthology.orchestra.run.trial103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-17T09:59:45.994 DEBUG:teuthology.orchestra.run.trial103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-17T09:59:46.058 INFO:teuthology.orchestra.run.trial103.stdout:check_obsoletes = 1 2026-02-17T09:59:46.060 DEBUG:teuthology.orchestra.run.trial103:> sudo yum clean all 2026-02-17T09:59:46.210 INFO:teuthology.orchestra.run.trial103.stdout:41 files removed 2026-02-17T09:59:46.224 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-radosgw 2026-02-17T09:59:47.052 INFO:teuthology.orchestra.run.trial103.stdout:ceph packages for x86_64 137 kB/s | 94 kB 00:00 2026-02-17T09:59:47.664 INFO:teuthology.orchestra.run.trial103.stdout:ceph noarch packages 42 kB/s | 24 kB 00:00 2026-02-17T09:59:48.145 INFO:teuthology.orchestra.run.trial103.stdout:ceph source packages 4.5 kB/s | 2.1 kB 00:00 2026-02-17T09:59:48.809 INFO:teuthology.orchestra.run.trial103.stdout:CentOS Stream 9 - BaseOS 14 MB/s | 8.9 MB 00:00 2026-02-17T10:00:17.058 INFO:teuthology.orchestra.run.trial103.stdout:CentOS Stream 9 - AppStream 991 kB/s | 27 MB 00:27 2026-02-17T10:00:20.516 INFO:teuthology.orchestra.run.trial103.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 7.8 MB 00:01 2026-02-17T10:00:21.447 INFO:teuthology.orchestra.run.trial103.stdout:CentOS Stream 9 - Extras packages 104 kB/s | 20 kB 00:00 2026-02-17T10:00:22.456 INFO:teuthology.orchestra.run.trial103.stdout:Extra Packages for Enterprise Linux 21 MB/s | 20 MB 00:00 2026-02-17T10:00:25.630 INFO:teuthology.orchestra.run.trial103.stdout:lab-extras 1.7 MB/s | 50 kB 00:00 2026-02-17T10:00:26.858 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:00:26.859 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:00:26.859 INFO:teuthology.orchestra.run.trial103.stdout: Package Arch Version Repository Size 2026-02-17T10:00:26.859 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:00:26.859 INFO:teuthology.orchestra.run.trial103.stdout:Installing: 2026-02-17T10:00:26.860 INFO:teuthology.orchestra.run.trial103.stdout: ceph-radosgw x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 11 M 2026-02-17T10:00:26.860 INFO:teuthology.orchestra.run.trial103.stdout:Upgrading: 2026-02-17T10:00:26.860 INFO:teuthology.orchestra.run.trial103.stdout: librados2 x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 3.3 M 2026-02-17T10:00:26.860 INFO:teuthology.orchestra.run.trial103.stdout: librbd1 x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 2.7 M 2026-02-17T10:00:26.860 INFO:teuthology.orchestra.run.trial103.stdout:Installing dependencies: 2026-02-17T10:00:26.860 INFO:teuthology.orchestra.run.trial103.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-17T10:00:26.860 INFO:teuthology.orchestra.run.trial103.stdout: ceph-base x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 5.1 M 2026-02-17T10:00:26.860 INFO:teuthology.orchestra.run.trial103.stdout: ceph-common x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 21 M 2026-02-17T10:00:26.860 INFO:teuthology.orchestra.run.trial103.stdout: ceph-selinux x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 25 k 2026-02-17T10:00:26.860 INFO:teuthology.orchestra.run.trial103.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-17T10:00:26.860 INFO:teuthology.orchestra.run.trial103.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-17T10:00:26.861 INFO:teuthology.orchestra.run.trial103.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-17T10:00:26.861 INFO:teuthology.orchestra.run.trial103.stdout: libcephfs2 x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 859 k 2026-02-17T10:00:26.861 INFO:teuthology.orchestra.run.trial103.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-17T10:00:26.861 INFO:teuthology.orchestra.run.trial103.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-17T10:00:26.861 INFO:teuthology.orchestra.run.trial103.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-17T10:00:26.861 INFO:teuthology.orchestra.run.trial103.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-17T10:00:26.861 INFO:teuthology.orchestra.run.trial103.stdout: libradosstriper1 x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 263 k 2026-02-17T10:00:26.861 INFO:teuthology.orchestra.run.trial103.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-17T10:00:26.861 INFO:teuthology.orchestra.run.trial103.stdout: librgw2 x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 5.6 M 2026-02-17T10:00:26.861 INFO:teuthology.orchestra.run.trial103.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-17T10:00:26.861 INFO:teuthology.orchestra.run.trial103.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-17T10:00:26.861 INFO:teuthology.orchestra.run.trial103.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-17T10:00:26.861 INFO:teuthology.orchestra.run.trial103.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-17T10:00:26.862 INFO:teuthology.orchestra.run.trial103.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-17T10:00:26.862 INFO:teuthology.orchestra.run.trial103.stdout: python3-ceph-argparse x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 45 k 2026-02-17T10:00:26.862 INFO:teuthology.orchestra.run.trial103.stdout: python3-ceph-common x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 185 k 2026-02-17T10:00:26.862 INFO:teuthology.orchestra.run.trial103.stdout: python3-cephfs x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 204 k 2026-02-17T10:00:26.862 INFO:teuthology.orchestra.run.trial103.stdout: python3-rados x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 325 k 2026-02-17T10:00:26.862 INFO:teuthology.orchestra.run.trial103.stdout: python3-rbd x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 306 k 2026-02-17T10:00:26.862 INFO:teuthology.orchestra.run.trial103.stdout: python3-rgw x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 100 k 2026-02-17T10:00:26.862 INFO:teuthology.orchestra.run.trial103.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-17T10:00:26.862 INFO:teuthology.orchestra.run.trial103.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-17T10:00:26.862 INFO:teuthology.orchestra.run.trial103.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-17T10:00:26.862 INFO:teuthology.orchestra.run.trial103.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-17T10:00:26.862 INFO:teuthology.orchestra.run.trial103.stdout:Installing weak dependencies: 2026-02-17T10:00:26.863 INFO:teuthology.orchestra.run.trial103.stdout: libcephfs-daemon x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 37 k 2026-02-17T10:00:26.863 INFO:teuthology.orchestra.run.trial103.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 26 k 2026-02-17T10:00:26.863 INFO:teuthology.orchestra.run.trial103.stdout: nvme-cli x86_64 2.13-1.el9 baseos 1.1 M 2026-02-17T10:00:26.863 INFO:teuthology.orchestra.run.trial103.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-17T10:00:26.863 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:00:26.863 INFO:teuthology.orchestra.run.trial103.stdout:Transaction Summary 2026-02-17T10:00:26.863 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:00:26.863 INFO:teuthology.orchestra.run.trial103.stdout:Install 35 Packages 2026-02-17T10:00:26.863 INFO:teuthology.orchestra.run.trial103.stdout:Upgrade 2 Packages 2026-02-17T10:00:26.863 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:00:26.864 INFO:teuthology.orchestra.run.trial103.stdout:Total download size: 61 M 2026-02-17T10:00:26.864 INFO:teuthology.orchestra.run.trial103.stdout:Downloading Packages: 2026-02-17T10:00:27.749 INFO:teuthology.orchestra.run.trial103.stdout:(1/37): ceph-base-20.3.0-5345.gb58fab16.el9.x86 7.4 MB/s | 5.1 MB 00:00 2026-02-17T10:00:27.780 INFO:teuthology.orchestra.run.trial103.stdout:(2/37): ceph-selinux-20.3.0-5345.gb58fab16.el9. 804 kB/s | 25 kB 00:00 2026-02-17T10:00:27.813 INFO:teuthology.orchestra.run.trial103.stdout:(3/37): libcephfs-daemon-20.3.0-5345.gb58fab16. 1.1 MB/s | 37 kB 00:00 2026-02-17T10:00:27.847 INFO:teuthology.orchestra.run.trial103.stdout:(4/37): libcephfs-proxy2-20.3.0-5345.gb58fab16. 763 kB/s | 26 kB 00:00 2026-02-17T10:00:27.931 INFO:teuthology.orchestra.run.trial103.stdout:(5/37): libcephfs2-20.3.0-5345.gb58fab16.el9.x8 10 MB/s | 859 kB 00:00 2026-02-17T10:00:27.972 INFO:teuthology.orchestra.run.trial103.stdout:(6/37): libradosstriper1-20.3.0-5345.gb58fab16. 6.2 MB/s | 263 kB 00:00 2026-02-17T10:00:28.191 INFO:teuthology.orchestra.run.trial103.stdout:(7/37): ceph-radosgw-20.3.0-5345.gb58fab16.el9. 9.9 MB/s | 11 MB 00:01 2026-02-17T10:00:28.233 INFO:teuthology.orchestra.run.trial103.stdout:(8/37): python3-ceph-argparse-20.3.0-5345.gb58f 1.1 MB/s | 45 kB 00:00 2026-02-17T10:00:28.277 INFO:teuthology.orchestra.run.trial103.stdout:(9/37): python3-ceph-common-20.3.0-5345.gb58fab 4.0 MB/s | 185 kB 00:00 2026-02-17T10:00:28.325 INFO:teuthology.orchestra.run.trial103.stdout:(10/37): python3-cephfs-20.3.0-5345.gb58fab16.e 4.2 MB/s | 204 kB 00:00 2026-02-17T10:00:28.376 INFO:teuthology.orchestra.run.trial103.stdout:(11/37): python3-rados-20.3.0-5345.gb58fab16.el 6.2 MB/s | 325 kB 00:00 2026-02-17T10:00:28.394 INFO:teuthology.orchestra.run.trial103.stdout:(12/37): librgw2-20.3.0-5345.gb58fab16.el9.x86_ 13 MB/s | 5.6 MB 00:00 2026-02-17T10:00:28.424 INFO:teuthology.orchestra.run.trial103.stdout:(13/37): python3-rbd-20.3.0-5345.gb58fab16.el9. 6.2 MB/s | 306 kB 00:00 2026-02-17T10:00:28.434 INFO:teuthology.orchestra.run.trial103.stdout:(14/37): python3-rgw-20.3.0-5345.gb58fab16.el9. 2.5 MB/s | 100 kB 00:00 2026-02-17T10:00:28.459 INFO:teuthology.orchestra.run.trial103.stdout:(15/37): mailcap-2.1.49-5.el9.noarch.rpm 947 kB/s | 33 kB 00:00 2026-02-17T10:00:28.488 INFO:teuthology.orchestra.run.trial103.stdout:(16/37): nvme-cli-2.13-1.el9.x86_64.rpm 20 MB/s | 1.1 MB 00:00 2026-02-17T10:00:28.495 INFO:teuthology.orchestra.run.trial103.stdout:(17/37): pciutils-3.7.0-7.el9.x86_64.rpm 2.6 MB/s | 93 kB 00:00 2026-02-17T10:00:28.562 INFO:teuthology.orchestra.run.trial103.stdout:(18/37): ceph-common-20.3.0-5345.gb58fab16.el9. 14 MB/s | 21 MB 00:01 2026-02-17T10:00:28.622 INFO:teuthology.orchestra.run.trial103.stdout:(19/37): boost-program-options-1.75.0-13.el9.x8 778 kB/s | 104 kB 00:00 2026-02-17T10:00:28.639 INFO:teuthology.orchestra.run.trial103.stdout:(20/37): libnbd-1.20.3-4.el9.x86_64.rpm 1.1 MB/s | 164 kB 00:00 2026-02-17T10:00:28.656 INFO:teuthology.orchestra.run.trial103.stdout:(21/37): librabbitmq-0.11.0-7.el9.x86_64.rpm 1.3 MB/s | 45 kB 00:00 2026-02-17T10:00:28.703 INFO:teuthology.orchestra.run.trial103.stdout:(22/37): libpmemobj-1.12.1-1.el9.x86_64.rpm 1.1 MB/s | 160 kB 00:00 2026-02-17T10:00:28.749 INFO:teuthology.orchestra.run.trial103.stdout:(23/37): lttng-ust-2.12.0-6.el9.x86_64.rpm 3.1 MB/s | 292 kB 00:00 2026-02-17T10:00:28.782 INFO:teuthology.orchestra.run.trial103.stdout:(24/37): qatlib-25.08.0-2.el9.x86_64.rpm 3.0 MB/s | 240 kB 00:00 2026-02-17T10:00:28.785 INFO:teuthology.orchestra.run.trial103.stdout:(25/37): librdkafka-1.6.1-102.el9.x86_64.rpm 4.4 MB/s | 662 kB 00:00 2026-02-17T10:00:28.792 INFO:teuthology.orchestra.run.trial103.stdout:(26/37): qatlib-service-25.08.0-2.el9.x86_64.rp 860 kB/s | 37 kB 00:00 2026-02-17T10:00:28.831 INFO:teuthology.orchestra.run.trial103.stdout:(27/37): qatzip-libs-1.3.1-1.el9.x86_64.rpm 1.3 MB/s | 66 kB 00:00 2026-02-17T10:00:28.942 INFO:teuthology.orchestra.run.trial103.stdout:(28/37): libarrow-doc-9.0.0-15.el9.noarch.rpm 224 kB/s | 25 kB 00:00 2026-02-17T10:00:28.975 INFO:teuthology.orchestra.run.trial103.stdout:(29/37): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.6 MB/s | 308 kB 00:00 2026-02-17T10:00:28.980 INFO:teuthology.orchestra.run.trial103.stdout:(30/37): liboath-2.6.12-1.el9.x86_64.rpm 1.3 MB/s | 49 kB 00:00 2026-02-17T10:00:29.001 INFO:teuthology.orchestra.run.trial103.stdout:(31/37): libunwind-1.6.2-1.el9.x86_64.rpm 2.6 MB/s | 67 kB 00:00 2026-02-17T10:00:29.044 INFO:teuthology.orchestra.run.trial103.stdout:(32/37): parquet-libs-9.0.0-15.el9.x86_64.rpm 13 MB/s | 838 kB 00:00 2026-02-17T10:00:29.048 INFO:teuthology.orchestra.run.trial103.stdout:(33/37): re2-20211101-20.el9.x86_64.rpm 4.0 MB/s | 191 kB 00:00 2026-02-17T10:00:29.072 INFO:teuthology.orchestra.run.trial103.stdout:(34/37): libarrow-9.0.0-15.el9.x86_64.rpm 16 MB/s | 4.4 MB 00:00 2026-02-17T10:00:29.094 INFO:teuthology.orchestra.run.trial103.stdout:(35/37): thrift-0.15.0-4.el9.x86_64.rpm 32 MB/s | 1.6 MB 00:00 2026-02-17T10:00:29.227 INFO:teuthology.orchestra.run.trial103.stdout:(36/37): librbd1-20.3.0-5345.gb58fab16.el9.x86_ 17 MB/s | 2.7 MB 00:00 2026-02-17T10:00:29.266 INFO:teuthology.orchestra.run.trial103.stdout:(37/37): librados2-20.3.0-5345.gb58fab16.el9.x8 15 MB/s | 3.3 MB 00:00 2026-02-17T10:00:29.270 INFO:teuthology.orchestra.run.trial103.stdout:-------------------------------------------------------------------------------- 2026-02-17T10:00:29.270 INFO:teuthology.orchestra.run.trial103.stdout:Total 25 MB/s | 61 MB 00:02 2026-02-17T10:00:29.366 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction check 2026-02-17T10:00:29.385 INFO:teuthology.orchestra.run.trial103.stdout:Transaction check succeeded. 2026-02-17T10:00:29.385 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction test 2026-02-17T10:00:29.618 INFO:teuthology.orchestra.run.trial103.stdout:Transaction test succeeded. 2026-02-17T10:00:29.618 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction 2026-02-17T10:00:29.914 INFO:teuthology.orchestra.run.trial103.stdout: Preparing : 1/1 2026-02-17T10:00:30.054 INFO:teuthology.orchestra.run.trial103.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/39 2026-02-17T10:00:30.058 INFO:teuthology.orchestra.run.trial103.stdout: Upgrading : librados2-2:20.3.0-5345.gb58fab16.el9.x86_64 2/39 2026-02-17T10:00:30.107 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: librados2-2:20.3.0-5345.gb58fab16.el9.x86_64 2/39 2026-02-17T10:00:30.108 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libcephfs2-2:20.3.0-5345.gb58fab16.el9.x86_64 3/39 2026-02-17T10:00:30.121 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: libcephfs2-2:20.3.0-5345.gb58fab16.el9.x86_64 3/39 2026-02-17T10:00:30.124 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libcephfs-daemon-2:20.3.0-5345.gb58fab16.el9.x86_6 4/39 2026-02-17T10:00:30.124 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libcephfs-proxy2-2:20.3.0-5345.gb58fab16.el9.x86_6 5/39 2026-02-17T10:00:30.151 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5345.gb58fab16.el9.x86_6 5/39 2026-02-17T10:00:30.154 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-rados-2:20.3.0-5345.gb58fab16.el9.x86_64 6/39 2026-02-17T10:00:30.161 INFO:teuthology.orchestra.run.trial103.stdout: Installing : liboath-2.6.12-1.el9.x86_64 7/39 2026-02-17T10:00:30.164 INFO:teuthology.orchestra.run.trial103.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 8/39 2026-02-17T10:00:30.167 INFO:teuthology.orchestra.run.trial103.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 9/39 2026-02-17T10:00:30.171 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 10/39 2026-02-17T10:00:30.182 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-ceph-argparse-2:20.3.0-5345.gb58fab16.el9. 11/39 2026-02-17T10:00:30.195 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-cephfs-2:20.3.0-5345.gb58fab16.el9.x86_64 12/39 2026-02-17T10:00:30.196 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libradosstriper1-2:20.3.0-5345.gb58fab16.el9.x86_6 13/39 2026-02-17T10:00:30.216 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5345.gb58fab16.el9.x86_6 13/39 2026-02-17T10:00:30.220 INFO:teuthology.orchestra.run.trial103.stdout: Installing : thrift-0.15.0-4.el9.x86_64 14/39 2026-02-17T10:00:30.253 INFO:teuthology.orchestra.run.trial103.stdout: Installing : re2-1:20211101-20.el9.x86_64 15/39 2026-02-17T10:00:30.257 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 16/39 2026-02-17T10:00:30.263 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 17/39 2026-02-17T10:00:30.265 INFO:teuthology.orchestra.run.trial103.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 18/39 2026-02-17T10:00:30.269 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 19/39 2026-02-17T10:00:30.384 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-17T10:00:30.385 INFO:teuthology.orchestra.run.trial103.stdout: Upgrading : librbd1-2:20.3.0-5345.gb58fab16.el9.x86_64 21/39 2026-02-17T10:00:30.413 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: librbd1-2:20.3.0-5345.gb58fab16.el9.x86_64 21/39 2026-02-17T10:00:30.416 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-rbd-2:20.3.0-5345.gb58fab16.el9.x86_64 22/39 2026-02-17T10:00:30.422 INFO:teuthology.orchestra.run.trial103.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 23/39 2026-02-17T10:00:30.671 INFO:teuthology.orchestra.run.trial103.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 24/39 2026-02-17T10:00:30.672 INFO:teuthology.orchestra.run.trial103.stdout: Installing : librgw2-2:20.3.0-5345.gb58fab16.el9.x86_64 25/39 2026-02-17T10:00:30.688 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: librgw2-2:20.3.0-5345.gb58fab16.el9.x86_64 25/39 2026-02-17T10:00:30.692 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-rgw-2:20.3.0-5345.gb58fab16.el9.x86_64 26/39 2026-02-17T10:00:30.698 INFO:teuthology.orchestra.run.trial103.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 27/39 2026-02-17T10:00:30.701 INFO:teuthology.orchestra.run.trial103.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 28/39 2026-02-17T10:00:30.701 INFO:teuthology.orchestra.run.trial103.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-17T10:00:30.719 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 29/39 2026-02-17T10:00:30.833 INFO:teuthology.orchestra.run.trial103.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 30/39 2026-02-17T10:00:30.835 INFO:teuthology.orchestra.run.trial103.stdout: Installing : nvme-cli-2.13-1.el9.x86_64 31/39 2026-02-17T10:00:31.111 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: nvme-cli-2.13-1.el9.x86_64 31/39 2026-02-17T10:00:31.111 INFO:teuthology.orchestra.run.trial103.stdout:Created symlink /etc/systemd/system/default.target.wants/nvmefc-boot-connections.service → /usr/lib/systemd/system/nvmefc-boot-connections.service. 2026-02-17T10:00:31.111 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:00:31.156 INFO:teuthology.orchestra.run.trial103.stdout: Installing : mailcap-2.1.49-5.el9.noarch 32/39 2026-02-17T10:00:31.158 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-ceph-common-2:20.3.0-5345.gb58fab16.el9.x8 33/39 2026-02-17T10:00:32.100 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-common-2:20.3.0-5345.gb58fab16.el9.x86_64 34/39 2026-02-17T10:00:32.102 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-common-2:20.3.0-5345.gb58fab16.el9.x86_64 34/39 2026-02-17T10:00:32.366 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-common-2:20.3.0-5345.gb58fab16.el9.x86_64 34/39 2026-02-17T10:00:32.369 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-base-2:20.3.0-5345.gb58fab16.el9.x86_64 35/39 2026-02-17T10:00:32.406 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-base-2:20.3.0-5345.gb58fab16.el9.x86_64 35/39 2026-02-17T10:00:32.406 INFO:teuthology.orchestra.run.trial103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-17T10:00:32.407 INFO:teuthology.orchestra.run.trial103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-17T10:00:32.407 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:00:32.410 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-selinux-2:20.3.0-5345.gb58fab16.el9.x86_64 36/39 2026-02-17T10:00:37.007 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5345.gb58fab16.el9.x86_64 36/39 2026-02-17T10:00:37.008 INFO:teuthology.orchestra.run.trial103.stdout:skipping the directory /sys 2026-02-17T10:00:37.008 INFO:teuthology.orchestra.run.trial103.stdout:skipping the directory /proc 2026-02-17T10:00:37.008 INFO:teuthology.orchestra.run.trial103.stdout:skipping the directory /mnt 2026-02-17T10:00:37.008 INFO:teuthology.orchestra.run.trial103.stdout:skipping the directory /var/tmp 2026-02-17T10:00:37.008 INFO:teuthology.orchestra.run.trial103.stdout:skipping the directory /home 2026-02-17T10:00:37.008 INFO:teuthology.orchestra.run.trial103.stdout:skipping the directory /root 2026-02-17T10:00:37.008 INFO:teuthology.orchestra.run.trial103.stdout:skipping the directory /tmp 2026-02-17T10:00:37.008 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:00:37.500 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-radosgw-2:20.3.0-5345.gb58fab16.el9.x86_64 37/39 2026-02-17T10:00:37.524 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5345.gb58fab16.el9.x86_64 37/39 2026-02-17T10:00:37.524 INFO:teuthology.orchestra.run.trial103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T10:00:37.524 INFO:teuthology.orchestra.run.trial103.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-17T10:00:37.524 INFO:teuthology.orchestra.run.trial103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T10:00:37.524 INFO:teuthology.orchestra.run.trial103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-17T10:00:37.525 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:00:37.526 INFO:teuthology.orchestra.run.trial103.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-17T10:00:37.538 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 38/39 2026-02-17T10:00:37.538 INFO:teuthology.orchestra.run.trial103.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-17T10:00:37.592 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 39/39 2026-02-17T10:00:37.593 INFO:teuthology.orchestra.run.trial103.stdout:Creating group 'qat' with GID 992. 2026-02-17T10:00:37.593 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:00:37.881 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-base-2:20.3.0-5345.gb58fab16.el9.x86_64 1/39 2026-02-17T10:00:37.881 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-common-2:20.3.0-5345.gb58fab16.el9.x86_64 2/39 2026-02-17T10:00:37.881 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-radosgw-2:20.3.0-5345.gb58fab16.el9.x86_64 3/39 2026-02-17T10:00:37.881 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-selinux-2:20.3.0-5345.gb58fab16.el9.x86_64 4/39 2026-02-17T10:00:37.881 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libcephfs-daemon-2:20.3.0-5345.gb58fab16.el9.x86_6 5/39 2026-02-17T10:00:37.881 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5345.gb58fab16.el9.x86_6 6/39 2026-02-17T10:00:37.881 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libcephfs2-2:20.3.0-5345.gb58fab16.el9.x86_64 7/39 2026-02-17T10:00:37.881 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libradosstriper1-2:20.3.0-5345.gb58fab16.el9.x86_6 8/39 2026-02-17T10:00:37.881 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : librgw2-2:20.3.0-5345.gb58fab16.el9.x86_64 9/39 2026-02-17T10:00:37.881 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5345.gb58fab16.el9. 10/39 2026-02-17T10:00:37.881 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-ceph-common-2:20.3.0-5345.gb58fab16.el9.x8 11/39 2026-02-17T10:00:37.881 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-cephfs-2:20.3.0-5345.gb58fab16.el9.x86_64 12/39 2026-02-17T10:00:37.881 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-rados-2:20.3.0-5345.gb58fab16.el9.x86_64 13/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-rbd-2:20.3.0-5345.gb58fab16.el9.x86_64 14/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-rgw-2:20.3.0-5345.gb58fab16.el9.x86_64 15/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 16/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : nvme-cli-2.13-1.el9.x86_64 17/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 18/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 19/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 20/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 21/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 22/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 23/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 24/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 25/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 26/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 27/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 28/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 29/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 30/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 31/39 2026-02-17T10:00:37.882 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 32/39 2026-02-17T10:00:37.883 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 33/39 2026-02-17T10:00:37.883 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : re2-1:20211101-20.el9.x86_64 34/39 2026-02-17T10:00:37.883 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 35/39 2026-02-17T10:00:37.883 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : librados2-2:20.3.0-5345.gb58fab16.el9.x86_64 36/39 2026-02-17T10:00:37.883 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 37/39 2026-02-17T10:00:37.883 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : librbd1-2:20.3.0-5345.gb58fab16.el9.x86_64 38/39 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 39/39 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout:Upgraded: 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: librados2-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: librbd1-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout:Installed: 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: ceph-base-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: ceph-common-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: ceph-radosgw-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: ceph-selinux-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: libcephfs-daemon-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: libcephfs-proxy2-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: libcephfs2-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: libradosstriper1-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: librgw2-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-17T10:00:37.945 INFO:teuthology.orchestra.run.trial103.stdout: nvme-cli-2.13-1.el9.x86_64 2026-02-17T10:00:37.946 INFO:teuthology.orchestra.run.trial103.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-17T10:00:37.946 INFO:teuthology.orchestra.run.trial103.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-17T10:00:37.946 INFO:teuthology.orchestra.run.trial103.stdout: python3-ceph-argparse-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.946 INFO:teuthology.orchestra.run.trial103.stdout: python3-ceph-common-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.946 INFO:teuthology.orchestra.run.trial103.stdout: python3-cephfs-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.946 INFO:teuthology.orchestra.run.trial103.stdout: python3-rados-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.946 INFO:teuthology.orchestra.run.trial103.stdout: python3-rbd-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.946 INFO:teuthology.orchestra.run.trial103.stdout: python3-rgw-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:37.946 INFO:teuthology.orchestra.run.trial103.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-17T10:00:37.946 INFO:teuthology.orchestra.run.trial103.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-17T10:00:37.946 INFO:teuthology.orchestra.run.trial103.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-17T10:00:37.946 INFO:teuthology.orchestra.run.trial103.stdout: re2-1:20211101-20.el9.x86_64 2026-02-17T10:00:37.946 INFO:teuthology.orchestra.run.trial103.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-17T10:00:37.946 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:00:37.946 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:00:37.985 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-test 2026-02-17T10:00:38.295 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:13 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:00:38.377 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:00:38.378 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:00:38.378 INFO:teuthology.orchestra.run.trial103.stdout: Package Arch Version Repository Size 2026-02-17T10:00:38.378 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:00:38.378 INFO:teuthology.orchestra.run.trial103.stdout:Installing: 2026-02-17T10:00:38.379 INFO:teuthology.orchestra.run.trial103.stdout: ceph-test x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 125 M 2026-02-17T10:00:38.379 INFO:teuthology.orchestra.run.trial103.stdout:Installing dependencies: 2026-02-17T10:00:38.379 INFO:teuthology.orchestra.run.trial103.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-17T10:00:38.379 INFO:teuthology.orchestra.run.trial103.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-17T10:00:38.379 INFO:teuthology.orchestra.run.trial103.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-17T10:00:38.379 INFO:teuthology.orchestra.run.trial103.stdout: libcephsqlite x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 153 k 2026-02-17T10:00:38.379 INFO:teuthology.orchestra.run.trial103.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-17T10:00:38.379 INFO:teuthology.orchestra.run.trial103.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-17T10:00:38.379 INFO:teuthology.orchestra.run.trial103.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-17T10:00:38.379 INFO:teuthology.orchestra.run.trial103.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-17T10:00:38.380 INFO:teuthology.orchestra.run.trial103.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-17T10:00:38.380 INFO:teuthology.orchestra.run.trial103.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-17T10:00:38.380 INFO:teuthology.orchestra.run.trial103.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-17T10:00:38.380 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:00:38.380 INFO:teuthology.orchestra.run.trial103.stdout:Transaction Summary 2026-02-17T10:00:38.380 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:00:38.380 INFO:teuthology.orchestra.run.trial103.stdout:Install 12 Packages 2026-02-17T10:00:38.380 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:00:38.381 INFO:teuthology.orchestra.run.trial103.stdout:Total download size: 129 M 2026-02-17T10:00:38.381 INFO:teuthology.orchestra.run.trial103.stdout:Installed size: 963 M 2026-02-17T10:00:38.381 INFO:teuthology.orchestra.run.trial103.stdout:Downloading Packages: 2026-02-17T10:00:38.994 INFO:teuthology.orchestra.run.trial103.stdout:(1/12): libcephsqlite-20.3.0-5345.gb58fab16.el9 421 kB/s | 153 kB 00:00 2026-02-17T10:00:39.426 INFO:teuthology.orchestra.run.trial103.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 53 kB/s | 23 kB 00:00 2026-02-17T10:00:39.689 INFO:teuthology.orchestra.run.trial103.stdout:(3/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 358 kB/s | 94 kB 00:00 2026-02-17T10:00:40.074 INFO:teuthology.orchestra.run.trial103.stdout:(4/12): libasan8-15.2.1-7.el9.x86_64.rpm 1.3 MB/s | 525 kB 00:00 2026-02-17T10:00:40.176 INFO:teuthology.orchestra.run.trial103.stdout:(5/12): libubsan-11.5.0-14.el9.x86_64.rpm 1.7 MB/s | 176 kB 00:00 2026-02-17T10:00:40.279 INFO:teuthology.orchestra.run.trial103.stdout:(6/12): libxslt-1.1.34-12.el9.x86_64.rpm 2.2 MB/s | 233 kB 00:00 2026-02-17T10:00:40.464 INFO:teuthology.orchestra.run.trial103.stdout:(7/12): protobuf-3.14.0-17.el9.x86_64.rpm 5.4 MB/s | 1.0 MB 00:00 2026-02-17T10:00:40.567 INFO:teuthology.orchestra.run.trial103.stdout:(8/12): socat-1.7.4.1-8.el9.x86_64.rpm 2.9 MB/s | 303 kB 00:00 2026-02-17T10:00:40.668 INFO:teuthology.orchestra.run.trial103.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 627 kB/s | 64 kB 00:00 2026-02-17T10:00:41.051 INFO:teuthology.orchestra.run.trial103.stdout:(10/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 318 kB/s | 122 kB 00:00 2026-02-17T10:00:41.953 INFO:teuthology.orchestra.run.trial103.stdout:(11/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 647 kB/s | 2.1 MB 00:03 2026-02-17T10:00:44.298 INFO:teuthology.orchestra.run.trial103.stdout:(12/12): ceph-test-20.3.0-5345.gb58fab16.el9.x8 22 MB/s | 125 MB 00:05 2026-02-17T10:00:44.300 INFO:teuthology.orchestra.run.trial103.stdout:-------------------------------------------------------------------------------- 2026-02-17T10:00:44.300 INFO:teuthology.orchestra.run.trial103.stdout:Total 22 MB/s | 129 MB 00:05 2026-02-17T10:00:44.382 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction check 2026-02-17T10:00:44.394 INFO:teuthology.orchestra.run.trial103.stdout:Transaction check succeeded. 2026-02-17T10:00:44.394 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction test 2026-02-17T10:00:44.818 INFO:teuthology.orchestra.run.trial103.stdout:Transaction test succeeded. 2026-02-17T10:00:44.818 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction 2026-02-17T10:00:45.267 INFO:teuthology.orchestra.run.trial103.stdout: Preparing : 1/1 2026-02-17T10:00:45.275 INFO:teuthology.orchestra.run.trial103.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-17T10:00:45.284 INFO:teuthology.orchestra.run.trial103.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-17T10:00:45.289 INFO:teuthology.orchestra.run.trial103.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-17T10:00:45.293 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-17T10:00:45.295 INFO:teuthology.orchestra.run.trial103.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-17T10:00:45.300 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-17T10:00:45.301 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-17T10:00:45.318 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-17T10:00:45.320 INFO:teuthology.orchestra.run.trial103.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-17T10:00:45.330 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-17T10:00:45.339 INFO:teuthology.orchestra.run.trial103.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-17T10:00:45.340 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libcephsqlite-2:20.3.0-5345.gb58fab16.el9.x86_64 11/12 2026-02-17T10:00:52.140 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5345.gb58fab16.el9.x86_64 11/12 2026-02-17T10:00:52.158 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-test-2:20.3.0-5345.gb58fab16.el9.x86_64 12/12 2026-02-17T10:00:52.921 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-test-2:20.3.0-5345.gb58fab16.el9.x86_64 12/12 2026-02-17T10:00:52.921 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-test-2:20.3.0-5345.gb58fab16.el9.x86_64 1/12 2026-02-17T10:00:52.921 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libcephsqlite-2:20.3.0-5345.gb58fab16.el9.x86_64 2/12 2026-02-17T10:00:52.921 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-17T10:00:52.921 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-17T10:00:52.921 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-17T10:00:52.921 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-17T10:00:52.922 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-17T10:00:52.922 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-17T10:00:52.922 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-17T10:00:52.922 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-17T10:00:52.922 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-17T10:00:52.975 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-17T10:00:52.975 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:00:52.975 INFO:teuthology.orchestra.run.trial103.stdout:Installed: 2026-02-17T10:00:52.975 INFO:teuthology.orchestra.run.trial103.stdout: ceph-test-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:52.975 INFO:teuthology.orchestra.run.trial103.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-17T10:00:52.975 INFO:teuthology.orchestra.run.trial103.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-17T10:00:52.976 INFO:teuthology.orchestra.run.trial103.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-17T10:00:52.976 INFO:teuthology.orchestra.run.trial103.stdout: libcephsqlite-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:00:52.976 INFO:teuthology.orchestra.run.trial103.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-17T10:00:52.976 INFO:teuthology.orchestra.run.trial103.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-17T10:00:52.976 INFO:teuthology.orchestra.run.trial103.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-17T10:00:52.976 INFO:teuthology.orchestra.run.trial103.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-17T10:00:52.976 INFO:teuthology.orchestra.run.trial103.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-17T10:00:52.976 INFO:teuthology.orchestra.run.trial103.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-17T10:00:52.976 INFO:teuthology.orchestra.run.trial103.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-17T10:00:52.977 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:00:52.977 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:00:53.020 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph 2026-02-17T10:00:53.327 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:28 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:00:53.415 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:00:53.417 INFO:teuthology.orchestra.run.trial103.stdout:======================================================================================= 2026-02-17T10:00:53.417 INFO:teuthology.orchestra.run.trial103.stdout: Package Arch Version Repository Size 2026-02-17T10:00:53.417 INFO:teuthology.orchestra.run.trial103.stdout:======================================================================================= 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout:Installing: 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: ceph x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 6.5 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout:Installing dependencies: 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5345.gb58fab16.el9 ceph-noarch 57 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mds x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 2.0 M 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mgr x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 937 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5345.gb58fab16.el9 ceph-noarch 297 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mon x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 4.9 M 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: ceph-osd x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 163 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: ceph-osd-classic x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 15 M 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: ceph-osd-crimson x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 85 M 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5345.gb58fab16.el9 ceph-noarch 17 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: cephadm noarch 2:20.3.0-5345.gb58fab16.el9 ceph-noarch 1.0 M 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: cryptsetup x86_64 2.8.1-2.el9 baseos 350 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: python3-devel x86_64 3.9.25-2.el9 appstream 244 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-17T10:00:53.418 INFO:teuthology.orchestra.run.trial103.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-isodate noarch 0.6.1-3.el9 epel 56 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-jaraco-collections noarch 3.0.0-8.el9 epel 23 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-lxml x86_64 4.6.5-3.el9 appstream 1.2 M 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-msgpack x86_64 1.0.3-2.el9 epel 86 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-pyasn1 noarch 0.4.8-6.el9 appstream 159 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-pyasn1-modules noarch 0.4.8-6.el9 appstream 279 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-xmlsec x86_64 1.3.13-1.el9 epel 48 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: xmlsec1 x86_64 1.2.29-13.el9 appstream 189 k 2026-02-17T10:00:53.419 INFO:teuthology.orchestra.run.trial103.stdout: xmlsec1-openssl x86_64 1.2.29-13.el9 appstream 90 k 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout:Installing weak dependencies: 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5345.gb58fab16.el9 ceph-noarch 184 k 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5345.gb58fab16.el9 ceph-noarch 51 M 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5345.gb58fab16.el9 ceph-noarch 7.4 M 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mgr-k8sevents noarch 2:20.3.0-5345.gb58fab16.el9 ceph-noarch 22 k 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mgr-rook noarch 2:20.3.0-5345.gb58fab16.el9 ceph-noarch 50 k 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout: ceph-volume noarch 2:20.3.0-5345.gb58fab16.el9 ceph-noarch 301 k 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout: python3-influxdb noarch 5.3.1-1.el9 epel 139 k 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout: python3-saml noarch 1.16.0-1.el9 epel 125 k 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout:Transaction Summary 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout:======================================================================================= 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout:Install 96 Packages 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout:Total download size: 223 M 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout:Installed size: 1.5 G 2026-02-17T10:00:53.420 INFO:teuthology.orchestra.run.trial103.stdout:Downloading Packages: 2026-02-17T10:00:54.603 INFO:teuthology.orchestra.run.trial103.stdout:(1/96): ceph-20.3.0-5345.gb58fab16.el9.x86_64.r 32 kB/s | 6.5 kB 00:00 2026-02-17T10:00:55.041 INFO:teuthology.orchestra.run.trial103.stdout:(2/96): ceph-mds-20.3.0-5345.gb58fab16.el9.x86_ 3.2 MB/s | 2.0 MB 00:00 2026-02-17T10:00:55.100 INFO:teuthology.orchestra.run.trial103.stdout:(3/96): ceph-osd-20.3.0-5345.gb58fab16.el9.x86_ 2.7 MB/s | 163 kB 00:00 2026-02-17T10:00:55.103 INFO:teuthology.orchestra.run.trial103.stdout:(4/96): ceph-mgr-20.3.0-5345.gb58fab16.el9.x86_ 1.3 MB/s | 937 kB 00:00 2026-02-17T10:00:56.699 INFO:teuthology.orchestra.run.trial103.stdout:(5/96): ceph-osd-classic-20.3.0-5345.gb58fab16. 9.2 MB/s | 15 MB 00:01 2026-02-17T10:00:56.747 INFO:teuthology.orchestra.run.trial103.stdout:(6/96): ceph-grafana-dashboards-20.3.0-5345.gb5 1.1 MB/s | 57 kB 00:00 2026-02-17T10:00:56.800 INFO:teuthology.orchestra.run.trial103.stdout:(7/96): ceph-mgr-cephadm-20.3.0-5345.gb58fab16. 3.4 MB/s | 184 kB 00:00 2026-02-17T10:00:57.528 INFO:teuthology.orchestra.run.trial103.stdout:(8/96): ceph-mon-20.3.0-5345.gb58fab16.el9.x86_ 1.7 MB/s | 4.9 MB 00:02 2026-02-17T10:01:01.636 INFO:teuthology.orchestra.run.trial103.stdout:(9/96): ceph-mgr-diskprediction-local-20.3.0-53 1.8 MB/s | 7.4 MB 00:04 2026-02-17T10:01:01.706 INFO:teuthology.orchestra.run.trial103.stdout:(10/96): ceph-mgr-k8sevents-20.3.0-5345.gb58fab 316 kB/s | 22 kB 00:00 2026-02-17T10:01:01.877 INFO:teuthology.orchestra.run.trial103.stdout:(11/96): ceph-mgr-modules-core-20.3.0-5345.gb58 1.7 MB/s | 297 kB 00:00 2026-02-17T10:01:01.923 INFO:teuthology.orchestra.run.trial103.stdout:(12/96): ceph-mgr-rook-20.3.0-5345.gb58fab16.el 1.1 MB/s | 50 kB 00:00 2026-02-17T10:01:01.968 INFO:teuthology.orchestra.run.trial103.stdout:(13/96): ceph-prometheus-alerts-20.3.0-5345.gb5 384 kB/s | 17 kB 00:00 2026-02-17T10:01:02.110 INFO:teuthology.orchestra.run.trial103.stdout:(14/96): ceph-volume-20.3.0-5345.gb58fab16.el9. 2.1 MB/s | 301 kB 00:00 2026-02-17T10:01:02.545 INFO:teuthology.orchestra.run.trial103.stdout:(15/96): cephadm-20.3.0-5345.gb58fab16.el9.noar 2.3 MB/s | 1.0 MB 00:00 2026-02-17T10:01:02.572 INFO:teuthology.orchestra.run.trial103.stdout:(16/96): cryptsetup-2.8.1-2.el9.x86_64.rpm 13 MB/s | 350 kB 00:00 2026-02-17T10:01:02.577 INFO:teuthology.orchestra.run.trial103.stdout:(17/96): ledmon-libs-1.1.0-3.el9.x86_64.rpm 8.9 MB/s | 40 kB 00:00 2026-02-17T10:01:02.582 INFO:teuthology.orchestra.run.trial103.stdout:(18/96): libconfig-1.7.2-9.el9.x86_64.rpm 15 MB/s | 72 kB 00:00 2026-02-17T10:01:02.592 INFO:teuthology.orchestra.run.trial103.stdout:(19/96): libgfortran-11.5.0-14.el9.x86_64.rpm 81 MB/s | 794 kB 00:00 2026-02-17T10:01:02.599 INFO:teuthology.orchestra.run.trial103.stdout:(20/96): libquadmath-11.5.0-14.el9.x86_64.rpm 24 MB/s | 184 kB 00:00 2026-02-17T10:01:02.606 INFO:teuthology.orchestra.run.trial103.stdout:(21/96): python3-cffi-1.14.5-5.el9.x86_64.rpm 35 MB/s | 253 kB 00:00 2026-02-17T10:01:02.639 INFO:teuthology.orchestra.run.trial103.stdout:(22/96): python3-cryptography-36.0.1-5.el9.x86_ 39 MB/s | 1.2 MB 00:00 2026-02-17T10:01:02.645 INFO:teuthology.orchestra.run.trial103.stdout:(23/96): python3-ply-3.11-14.el9.noarch.rpm 17 MB/s | 106 kB 00:00 2026-02-17T10:01:02.651 INFO:teuthology.orchestra.run.trial103.stdout:(24/96): python3-pycparser-2.20-6.el9.noarch.rp 21 MB/s | 135 kB 00:00 2026-02-17T10:01:02.658 INFO:teuthology.orchestra.run.trial103.stdout:(25/96): python3-pyparsing-2.4.7-9.el9.noarch.r 23 MB/s | 150 kB 00:00 2026-02-17T10:01:02.665 INFO:teuthology.orchestra.run.trial103.stdout:(26/96): python3-requests-2.25.1-10.el9.noarch. 18 MB/s | 126 kB 00:00 2026-02-17T10:01:02.672 INFO:teuthology.orchestra.run.trial103.stdout:(27/96): python3-urllib3-1.26.5-7.el9.noarch.rp 32 MB/s | 218 kB 00:00 2026-02-17T10:01:02.679 INFO:teuthology.orchestra.run.trial103.stdout:(28/96): unzip-6.0-59.el9.x86_64.rpm 25 MB/s | 182 kB 00:00 2026-02-17T10:01:02.686 INFO:teuthology.orchestra.run.trial103.stdout:(29/96): zip-3.0-35.el9.x86_64.rpm 40 MB/s | 266 kB 00:00 2026-02-17T10:01:03.403 INFO:teuthology.orchestra.run.trial103.stdout:(30/96): flexiblas-3.0.4-9.el9.x86_64.rpm 41 kB/s | 30 kB 00:00 2026-02-17T10:01:07.065 INFO:teuthology.orchestra.run.trial103.stdout:(31/96): ceph-mgr-dashboard-20.3.0-5345.gb58fab 4.9 MB/s | 51 MB 00:10 2026-02-17T10:01:07.079 INFO:teuthology.orchestra.run.trial103.stdout:(32/96): flexiblas-netlib-3.0.4-9.el9.x86_64.rp 833 kB/s | 3.0 MB 00:03 2026-02-17T10:01:07.229 INFO:teuthology.orchestra.run.trial103.stdout:(33/96): flexiblas-openblas-openmp-3.0.4-9.el9. 90 kB/s | 15 kB 00:00 2026-02-17T10:01:07.571 INFO:teuthology.orchestra.run.trial103.stdout:(34/96): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 501 kB/s | 246 kB 00:00 2026-02-17T10:01:07.736 INFO:teuthology.orchestra.run.trial103.stdout:(35/96): openblas-0.3.29-1.el9.x86_64.rpm 257 kB/s | 42 kB 00:00 2026-02-17T10:01:08.460 INFO:teuthology.orchestra.run.trial103.stdout:(36/96): lua-5.4.4-4.el9.x86_64.rpm 153 kB/s | 188 kB 00:01 2026-02-17T10:01:14.104 INFO:teuthology.orchestra.run.trial103.stdout:(37/96): openblas-openmp-0.3.29-1.el9.x86_64.rp 850 kB/s | 5.3 MB 00:06 2026-02-17T10:01:14.604 INFO:teuthology.orchestra.run.trial103.stdout:(38/96): python3-devel-3.9.25-2.el9.x86_64.rpm 489 kB/s | 244 kB 00:00 2026-02-17T10:01:15.022 INFO:teuthology.orchestra.run.trial103.stdout:(39/96): python3-jinja2-2.11.3-8.el9.noarch.rpm 595 kB/s | 249 kB 00:00 2026-02-17T10:01:15.187 INFO:teuthology.orchestra.run.trial103.stdout:(40/96): python3-jmespath-1.0.1-1.el9.noarch.rp 288 kB/s | 48 kB 00:00 2026-02-17T10:01:15.519 INFO:teuthology.orchestra.run.trial103.stdout:(41/96): python3-libstoragemgmt-1.10.1-1.el9.x8 533 kB/s | 177 kB 00:00 2026-02-17T10:01:16.471 INFO:teuthology.orchestra.run.trial103.stdout:(42/96): python3-babel-2.9.1-2.el9.noarch.rpm 762 kB/s | 6.0 MB 00:08 2026-02-17T10:01:16.741 INFO:teuthology.orchestra.run.trial103.stdout:(43/96): ceph-osd-crimson-20.3.0-5345.gb58fab16 3.9 MB/s | 85 MB 00:21 2026-02-17T10:01:16.742 INFO:teuthology.orchestra.run.trial103.stdout:(44/96): python3-markupsafe-1.1.1-12.el9.x86_64 128 kB/s | 35 kB 00:00 2026-02-17T10:01:17.118 INFO:teuthology.orchestra.run.trial103.stdout:(45/96): python3-lxml-4.6.5-3.el9.x86_64.rpm 783 kB/s | 1.2 MB 00:01 2026-02-17T10:01:17.365 INFO:teuthology.orchestra.run.trial103.stdout:(46/96): python3-packaging-20.9-5.el9.noarch.rp 313 kB/s | 77 kB 00:00 2026-02-17T10:01:17.775 INFO:teuthology.orchestra.run.trial103.stdout:(47/96): python3-protobuf-3.14.0-17.el9.noarch. 652 kB/s | 267 kB 00:00 2026-02-17T10:01:17.886 INFO:teuthology.orchestra.run.trial103.stdout:(48/96): python3-numpy-f2py-1.23.5-2.el9.x86_64 386 kB/s | 442 kB 00:01 2026-02-17T10:01:18.103 INFO:teuthology.orchestra.run.trial103.stdout:(49/96): python3-pyasn1-0.4.8-6.el9.noarch.rpm 484 kB/s | 159 kB 00:00 2026-02-17T10:01:18.268 INFO:teuthology.orchestra.run.trial103.stdout:(50/96): python3-requests-oauthlib-1.3.0-12.el9 326 kB/s | 54 kB 00:00 2026-02-17T10:01:18.294 INFO:teuthology.orchestra.run.trial103.stdout:(51/96): python3-pyasn1-modules-0.4.8-6.el9.noa 685 kB/s | 279 kB 00:00 2026-02-17T10:01:18.458 INFO:teuthology.orchestra.run.trial103.stdout:(52/96): python3-toml-0.10.2-6.el9.noarch.rpm 252 kB/s | 42 kB 00:00 2026-02-17T10:01:18.783 INFO:teuthology.orchestra.run.trial103.stdout:(53/96): xmlsec1-1.2.29-13.el9.x86_64.rpm 580 kB/s | 189 kB 00:00 2026-02-17T10:01:19.028 INFO:teuthology.orchestra.run.trial103.stdout:(54/96): xmlsec1-openssl-1.2.29-13.el9.x86_64.r 370 kB/s | 90 kB 00:00 2026-02-17T10:01:19.281 INFO:teuthology.orchestra.run.trial103.stdout:(55/96): lua-devel-5.4.4-4.el9.x86_64.rpm 88 kB/s | 22 kB 00:00 2026-02-17T10:01:19.756 INFO:teuthology.orchestra.run.trial103.stdout:(56/96): protobuf-compiler-3.14.0-17.el9.x86_64 1.8 MB/s | 862 kB 00:00 2026-02-17T10:01:19.836 INFO:teuthology.orchestra.run.trial103.stdout:(57/96): abseil-cpp-20211102.0-4.el9.x86_64.rpm 6.8 MB/s | 551 kB 00:00 2026-02-17T10:01:19.841 INFO:teuthology.orchestra.run.trial103.stdout:(58/96): grpc-data-1.46.7-10.el9.noarch.rpm 3.5 MB/s | 19 kB 00:00 2026-02-17T10:01:19.854 INFO:teuthology.orchestra.run.trial103.stdout:(59/96): luarocks-3.9.2-5.el9.noarch.rpm 12 MB/s | 151 kB 00:00 2026-02-17T10:01:19.890 INFO:teuthology.orchestra.run.trial103.stdout:(60/96): python3-asyncssh-2.13.2-5.el9.noarch.r 15 MB/s | 548 kB 00:00 2026-02-17T10:01:19.895 INFO:teuthology.orchestra.run.trial103.stdout:(61/96): python3-autocommand-2.2.2-8.el9.noarch 6.9 MB/s | 29 kB 00:00 2026-02-17T10:01:19.901 INFO:teuthology.orchestra.run.trial103.stdout:(62/96): python3-backports-tarfile-1.2.0-1.el9. 8.9 MB/s | 60 kB 00:00 2026-02-17T10:01:19.905 INFO:teuthology.orchestra.run.trial103.stdout:(63/96): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 11 MB/s | 43 kB 00:00 2026-02-17T10:01:19.910 INFO:teuthology.orchestra.run.trial103.stdout:(64/96): python3-cachetools-4.2.4-1.el9.noarch. 7.9 MB/s | 32 kB 00:00 2026-02-17T10:01:19.914 INFO:teuthology.orchestra.run.trial103.stdout:(65/96): python3-certifi-2023.05.07-4.el9.noarc 3.5 MB/s | 14 kB 00:00 2026-02-17T10:01:19.926 INFO:teuthology.orchestra.run.trial103.stdout:(66/96): python3-cheroot-10.0.1-4.el9.noarch.rp 13 MB/s | 173 kB 00:00 2026-02-17T10:01:19.948 INFO:teuthology.orchestra.run.trial103.stdout:(67/96): python3-cherrypy-18.6.1-2.el9.noarch.r 17 MB/s | 358 kB 00:00 2026-02-17T10:01:19.961 INFO:teuthology.orchestra.run.trial103.stdout:(68/96): python3-google-auth-2.45.0-1.el9.noarc 19 MB/s | 254 kB 00:00 2026-02-17T10:01:20.039 INFO:teuthology.orchestra.run.trial103.stdout:(69/96): python3-grpcio-1.46.7-10.el9.x86_64.rp 26 MB/s | 2.0 MB 00:00 2026-02-17T10:01:20.047 INFO:teuthology.orchestra.run.trial103.stdout:(70/96): python3-grpcio-tools-1.46.7-10.el9.x86 19 MB/s | 144 kB 00:00 2026-02-17T10:01:20.054 INFO:teuthology.orchestra.run.trial103.stdout:(71/96): python3-influxdb-5.3.1-1.el9.noarch.rp 19 MB/s | 139 kB 00:00 2026-02-17T10:01:20.059 INFO:teuthology.orchestra.run.trial103.stdout:(72/96): python3-isodate-0.6.1-3.el9.noarch.rpm 13 MB/s | 56 kB 00:00 2026-02-17T10:01:20.062 INFO:teuthology.orchestra.run.trial103.stdout:(73/96): python3-jaraco-8.2.1-3.el9.noarch.rpm 3.0 MB/s | 11 kB 00:00 2026-02-17T10:01:20.066 INFO:teuthology.orchestra.run.trial103.stdout:(74/96): python3-jaraco-classes-3.2.1-5.el9.noa 4.9 MB/s | 18 kB 00:00 2026-02-17T10:01:20.069 INFO:teuthology.orchestra.run.trial103.stdout:(75/96): python3-jaraco-collections-3.0.0-8.el9 6.4 MB/s | 23 kB 00:00 2026-02-17T10:01:20.073 INFO:teuthology.orchestra.run.trial103.stdout:(76/96): python3-jaraco-context-6.0.1-3.el9.noa 5.4 MB/s | 20 kB 00:00 2026-02-17T10:01:20.077 INFO:teuthology.orchestra.run.trial103.stdout:(77/96): python3-jaraco-functools-3.5.0-2.el9.n 5.1 MB/s | 19 kB 00:00 2026-02-17T10:01:20.080 INFO:teuthology.orchestra.run.trial103.stdout:(78/96): python3-jaraco-text-4.0.0-2.el9.noarch 7.4 MB/s | 26 kB 00:00 2026-02-17T10:01:20.114 INFO:teuthology.orchestra.run.trial103.stdout:(79/96): python3-kubernetes-26.1.0-3.el9.noarch 30 MB/s | 1.0 MB 00:00 2026-02-17T10:01:20.118 INFO:teuthology.orchestra.run.trial103.stdout:(80/96): python3-more-itertools-8.12.0-2.el9.no 19 MB/s | 79 kB 00:00 2026-02-17T10:01:20.122 INFO:teuthology.orchestra.run.trial103.stdout:(81/96): python3-msgpack-1.0.3-2.el9.x86_64.rpm 21 MB/s | 86 kB 00:00 2026-02-17T10:01:20.126 INFO:teuthology.orchestra.run.trial103.stdout:(82/96): python3-natsort-7.1.1-5.el9.noarch.rpm 16 MB/s | 58 kB 00:00 2026-02-17T10:01:20.130 INFO:teuthology.orchestra.run.trial103.stdout:(83/96): python3-portend-3.1.0-2.el9.noarch.rpm 4.6 MB/s | 16 kB 00:00 2026-02-17T10:01:20.134 INFO:teuthology.orchestra.run.trial103.stdout:(84/96): python3-pyOpenSSL-21.0.0-1.el9.noarch. 23 MB/s | 90 kB 00:00 2026-02-17T10:01:20.138 INFO:teuthology.orchestra.run.trial103.stdout:(85/96): python3-repoze-lru-0.7-16.el9.noarch.r 7.9 MB/s | 31 kB 00:00 2026-02-17T10:01:20.145 INFO:teuthology.orchestra.run.trial103.stdout:(86/96): python3-routes-2.5.1-5.el9.noarch.rpm 26 MB/s | 188 kB 00:00 2026-02-17T10:01:20.149 INFO:teuthology.orchestra.run.trial103.stdout:(87/96): python3-rsa-4.9-2.el9.noarch.rpm 15 MB/s | 59 kB 00:00 2026-02-17T10:01:20.155 INFO:teuthology.orchestra.run.trial103.stdout:(88/96): python3-saml-1.16.0-1.el9.noarch.rpm 18 MB/s | 125 kB 00:00 2026-02-17T10:01:20.159 INFO:teuthology.orchestra.run.trial103.stdout:(89/96): python3-tempora-5.0.0-2.el9.noarch.rpm 9.8 MB/s | 36 kB 00:00 2026-02-17T10:01:20.163 INFO:teuthology.orchestra.run.trial103.stdout:(90/96): python3-typing-extensions-4.15.0-1.el9 22 MB/s | 86 kB 00:00 2026-02-17T10:01:20.167 INFO:teuthology.orchestra.run.trial103.stdout:(91/96): python3-websocket-client-1.2.3-2.el9.n 22 MB/s | 90 kB 00:00 2026-02-17T10:01:20.171 INFO:teuthology.orchestra.run.trial103.stdout:(92/96): python3-xmlsec-1.3.13-1.el9.x86_64.rpm 13 MB/s | 48 kB 00:00 2026-02-17T10:01:20.175 INFO:teuthology.orchestra.run.trial103.stdout:(93/96): python3-xmltodict-0.12.0-15.el9.noarch 6.3 MB/s | 22 kB 00:00 2026-02-17T10:01:20.178 INFO:teuthology.orchestra.run.trial103.stdout:(94/96): python3-zc-lockfile-2.0-10.el9.noarch. 5.6 MB/s | 20 kB 00:00 2026-02-17T10:01:24.185 INFO:teuthology.orchestra.run.trial103.stdout:(95/96): python3-numpy-1.23.5-2.el9.x86_64.rpm 843 kB/s | 6.1 MB 00:07 2026-02-17T10:01:41.459 INFO:teuthology.orchestra.run.trial103.stdout:(96/96): python3-scipy-1.9.3-2.el9.x86_64.rpm 851 kB/s | 19 MB 00:23 2026-02-17T10:01:41.463 INFO:teuthology.orchestra.run.trial103.stdout:-------------------------------------------------------------------------------- 2026-02-17T10:01:41.463 INFO:teuthology.orchestra.run.trial103.stdout:Total 4.6 MB/s | 223 MB 00:48 2026-02-17T10:01:41.837 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction check 2026-02-17T10:01:41.875 INFO:teuthology.orchestra.run.trial103.stdout:Transaction check succeeded. 2026-02-17T10:01:41.876 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction test 2026-02-17T10:01:42.664 INFO:teuthology.orchestra.run.trial103.stdout:Transaction test succeeded. 2026-02-17T10:01:42.664 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction 2026-02-17T10:01:43.476 INFO:teuthology.orchestra.run.trial103.stdout: Preparing : 1/1 2026-02-17T10:01:43.478 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/96 2026-02-17T10:01:43.482 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/96 2026-02-17T10:01:43.504 INFO:teuthology.orchestra.run.trial103.stdout: Installing : xmlsec1-1.2.29-13.el9.x86_64 3/96 2026-02-17T10:01:43.537 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-pyasn1-0.4.8-6.el9.noarch 4/96 2026-02-17T10:01:43.543 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-lxml-4.6.5-3.el9.x86_64 5/96 2026-02-17T10:01:43.545 INFO:teuthology.orchestra.run.trial103.stdout: Installing : lua-5.4.4-4.el9.x86_64 6/96 2026-02-17T10:01:43.549 INFO:teuthology.orchestra.run.trial103.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 7/96 2026-02-17T10:01:43.573 INFO:teuthology.orchestra.run.trial103.stdout: Installing : unzip-6.0-59.el9.x86_64 8/96 2026-02-17T10:01:43.586 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 9/96 2026-02-17T10:01:43.589 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 10/96 2026-02-17T10:01:43.595 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 11/96 2026-02-17T10:01:43.596 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 12/96 2026-02-17T10:01:43.608 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 13/96 2026-02-17T10:01:43.614 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 14/96 2026-02-17T10:01:43.638 INFO:teuthology.orchestra.run.trial103.stdout: Installing : zip-3.0-35.el9.x86_64 15/96 2026-02-17T10:01:43.641 INFO:teuthology.orchestra.run.trial103.stdout: Installing : luarocks-3.9.2-5.el9.noarch 16/96 2026-02-17T10:01:43.692 INFO:teuthology.orchestra.run.trial103.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 17/96 2026-02-17T10:01:43.705 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-pyasn1-modules-0.4.8-6.el9.noarch 18/96 2026-02-17T10:01:43.707 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-rsa-4.9-2.el9.noarch 19/96 2026-02-17T10:01:43.711 INFO:teuthology.orchestra.run.trial103.stdout: Installing : xmlsec1-openssl-1.2.29-13.el9.x86_64 20/96 2026-02-17T10:01:43.717 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-xmlsec-1.3.13-1.el9.x86_64 21/96 2026-02-17T10:01:43.721 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 22/96 2026-02-17T10:01:43.724 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 23/96 2026-02-17T10:01:43.739 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 24/96 2026-02-17T10:01:43.743 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 25/96 2026-02-17T10:01:43.749 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 26/96 2026-02-17T10:01:43.760 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 27/96 2026-02-17T10:01:43.769 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 28/96 2026-02-17T10:01:43.775 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 29/96 2026-02-17T10:01:43.795 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-msgpack-1.0.3-2.el9.x86_64 30/96 2026-02-17T10:01:43.807 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-influxdb-5.3.1-1.el9.noarch 31/96 2026-02-17T10:01:43.822 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-isodate-0.6.1-3.el9.noarch 32/96 2026-02-17T10:01:43.827 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-saml-1.16.0-1.el9.noarch 33/96 2026-02-17T10:01:43.834 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 34/96 2026-02-17T10:01:43.875 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 35/96 2026-02-17T10:01:44.203 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 36/96 2026-02-17T10:01:44.213 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 37/96 2026-02-17T10:01:44.216 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 38/96 2026-02-17T10:01:44.223 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 39/96 2026-02-17T10:01:44.224 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 40/96 2026-02-17T10:01:44.251 INFO:teuthology.orchestra.run.trial103.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 41/96 2026-02-17T10:01:44.294 INFO:teuthology.orchestra.run.trial103.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 42/96 2026-02-17T10:01:44.304 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 43/96 2026-02-17T10:01:44.309 INFO:teuthology.orchestra.run.trial103.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 44/96 2026-02-17T10:01:44.313 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 45/96 2026-02-17T10:01:44.318 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 46/96 2026-02-17T10:01:44.322 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 47/96 2026-02-17T10:01:44.329 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 48/96 2026-02-17T10:01:44.332 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 49/96 2026-02-17T10:01:44.360 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 50/96 2026-02-17T10:01:44.371 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 51/96 2026-02-17T10:01:44.376 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 52/96 2026-02-17T10:01:44.383 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 53/96 2026-02-17T10:01:44.417 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 54/96 2026-02-17T10:01:44.640 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-devel-3.9.25-2.el9.x86_64 55/96 2026-02-17T10:01:44.663 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 56/96 2026-02-17T10:01:44.667 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 57/96 2026-02-17T10:01:44.725 INFO:teuthology.orchestra.run.trial103.stdout: Installing : openblas-0.3.29-1.el9.x86_64 58/96 2026-02-17T10:01:44.727 INFO:teuthology.orchestra.run.trial103.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 59/96 2026-02-17T10:01:44.748 INFO:teuthology.orchestra.run.trial103.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 60/96 2026-02-17T10:01:45.073 INFO:teuthology.orchestra.run.trial103.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 61/96 2026-02-17T10:01:45.145 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 62/96 2026-02-17T10:01:45.813 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 63/96 2026-02-17T10:01:45.830 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 64/96 2026-02-17T10:01:45.840 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 65/96 2026-02-17T10:01:45.847 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-packaging-20.9-5.el9.noarch 66/96 2026-02-17T10:01:45.860 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-ply-3.11-14.el9.noarch 67/96 2026-02-17T10:01:45.875 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 68/96 2026-02-17T10:01:45.950 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 69/96 2026-02-17T10:01:45.959 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 70/96 2026-02-17T10:01:45.982 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 71/96 2026-02-17T10:01:46.012 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 72/96 2026-02-17T10:01:46.062 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 73/96 2026-02-17T10:01:46.068 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 74/96 2026-02-17T10:01:46.073 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 75/96 2026-02-17T10:01:46.074 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 76/96 2026-02-17T10:01:46.091 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-17T10:01:46.092 INFO:teuthology.orchestra.run.trial103.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-17T10:01:46.092 INFO:teuthology.orchestra.run.trial103.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-17T10:01:46.092 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:01:46.099 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-17T10:01:46.124 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 77/96 2026-02-17T10:01:46.125 INFO:teuthology.orchestra.run.trial103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-17T10:01:46.125 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:01:46.158 INFO:teuthology.orchestra.run.trial103.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 78/96 2026-02-17T10:01:46.746 INFO:teuthology.orchestra.run.trial103.stdout: Installing : cryptsetup-2.8.1-2.el9.x86_64 79/96 2026-02-17T10:01:52.003 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-osd-classic-2:20.3.0-5345.gb58fab16.el9.x86_6 80/96 2026-02-17T10:01:52.067 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-osd-crimson-2:20.3.0-5345.gb58fab16.el9.x86_6 81/96 2026-02-17T10:01:52.070 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-volume-2:20.3.0-5345.gb58fab16.el9.noarch 82/96 2026-02-17T10:01:52.083 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-volume-2:20.3.0-5345.gb58fab16.el9.noarch 82/96 2026-02-17T10:01:52.084 INFO:teuthology.orchestra.run.trial103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T10:01:52.084 INFO:teuthology.orchestra.run.trial103.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-17T10:01:52.084 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:01:52.094 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-osd-2:20.3.0-5345.gb58fab16.el9.x86_64 83/96 2026-02-17T10:01:52.117 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-osd-2:20.3.0-5345.gb58fab16.el9.x86_64 83/96 2026-02-17T10:01:52.117 INFO:teuthology.orchestra.run.trial103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T10:01:52.117 INFO:teuthology.orchestra.run.trial103.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-17T10:01:52.117 INFO:teuthology.orchestra.run.trial103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T10:01:52.117 INFO:teuthology.orchestra.run.trial103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-17T10:01:52.117 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:01:52.172 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: cephadm-2:20.3.0-5345.gb58fab16.el9.noarch 84/96 2026-02-17T10:01:52.174 INFO:teuthology.orchestra.run.trial103.stdout: Installing : cephadm-2:20.3.0-5345.gb58fab16.el9.noarch 84/96 2026-02-17T10:01:52.183 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5345.gb58fab16.el9 85/96 2026-02-17T10:01:52.207 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5345.gb58fab16.el 86/96 2026-02-17T10:01:52.208 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5345.gb58fab16.el9.noarc 87/96 2026-02-17T10:01:55.089 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5345.gb58fab16.el9.noarc 87/96 2026-02-17T10:01:55.096 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5345.gb58fab16.el9.noa 88/96 2026-02-17T10:01:55.547 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5345.gb58fab16.el9.noa 88/96 2026-02-17T10:01:55.548 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5345.gb58fa 89/96 2026-02-17T10:01:55.562 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5345.gb58fa 89/96 2026-02-17T10:01:55.562 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-mgr-k8sevents-2:20.3.0-5345.gb58fab16.el9.noa 90/96 2026-02-17T10:01:55.624 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-mgr-k8sevents-2:20.3.0-5345.gb58fab16.el9.noa 90/96 2026-02-17T10:01:55.664 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5345.gb58fab16.el9. 91/96 2026-02-17T10:01:55.665 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-mgr-2:20.3.0-5345.gb58fab16.el9.x86_64 92/96 2026-02-17T10:01:55.685 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5345.gb58fab16.el9.x86_64 92/96 2026-02-17T10:01:55.685 INFO:teuthology.orchestra.run.trial103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T10:01:55.686 INFO:teuthology.orchestra.run.trial103.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-17T10:01:55.686 INFO:teuthology.orchestra.run.trial103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T10:01:55.686 INFO:teuthology.orchestra.run.trial103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-17T10:01:55.686 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:01:55.698 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-mgr-rook-2:20.3.0-5345.gb58fab16.el9.noarch 93/96 2026-02-17T10:01:55.911 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5345.gb58fab16.el9.noarch 93/96 2026-02-17T10:01:55.913 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-mon-2:20.3.0-5345.gb58fab16.el9.x86_64 94/96 2026-02-17T10:01:55.934 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-mon-2:20.3.0-5345.gb58fab16.el9.x86_64 94/96 2026-02-17T10:01:55.934 INFO:teuthology.orchestra.run.trial103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T10:01:55.935 INFO:teuthology.orchestra.run.trial103.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-17T10:01:55.935 INFO:teuthology.orchestra.run.trial103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T10:01:55.935 INFO:teuthology.orchestra.run.trial103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-17T10:01:55.935 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:01:56.017 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-mds-2:20.3.0-5345.gb58fab16.el9.x86_64 95/96 2026-02-17T10:01:56.040 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-mds-2:20.3.0-5345.gb58fab16.el9.x86_64 95/96 2026-02-17T10:01:56.040 INFO:teuthology.orchestra.run.trial103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T10:01:56.040 INFO:teuthology.orchestra.run.trial103.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-17T10:01:56.040 INFO:teuthology.orchestra.run.trial103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T10:01:56.040 INFO:teuthology.orchestra.run.trial103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-17T10:01:56.041 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:01:56.041 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-2:20.3.0-5345.gb58fab16.el9.x86_64 96/96 2026-02-17T10:01:56.044 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5345.gb58fab16.el9.x86_6 96/96 2026-02-17T10:01:56.068 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5345.gb58fab16.el9.x86_6 96/96 2026-02-17T10:01:57.440 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-2:20.3.0-5345.gb58fab16.el9.x86_64 96/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-2:20.3.0-5345.gb58fab16.el9.x86_64 1/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-mds-2:20.3.0-5345.gb58fab16.el9.x86_64 2/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-mgr-2:20.3.0-5345.gb58fab16.el9.x86_64 3/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-mon-2:20.3.0-5345.gb58fab16.el9.x86_64 4/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-osd-2:20.3.0-5345.gb58fab16.el9.x86_64 5/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-osd-classic-2:20.3.0-5345.gb58fab16.el9.x86_6 6/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5345.gb58fab16.el9.x86_6 7/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5345.gb58fab16.el 8/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5345.gb58fab16.el9.noarc 9/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5345.gb58fab16.el9.noa 10/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5345.gb58fa 11/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-mgr-k8sevents-2:20.3.0-5345.gb58fab16.el9.noa 12/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5345.gb58fab16.el9. 13/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5345.gb58fab16.el9.noarch 14/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5345.gb58fab16.el9 15/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-volume-2:20.3.0-5345.gb58fab16.el9.noarch 16/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : cephadm-2:20.3.0-5345.gb58fab16.el9.noarch 17/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : cryptsetup-2.8.1-2.el9.x86_64 18/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 19/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 20/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 21/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 22/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 23/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 24/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-ply-3.11-14.el9.noarch 25/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 26/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 27/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 28/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 29/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : unzip-6.0-59.el9.x86_64 30/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : zip-3.0-35.el9.x86_64 31/96 2026-02-17T10:01:57.441 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 32/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 33/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 34/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 35/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : lua-5.4.4-4.el9.x86_64 36/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 37/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 38/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 39/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-devel-3.9.25-2.el9.x86_64 40/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 41/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 42/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 43/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-lxml-4.6.5-3.el9.x86_64 44/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 45/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 46/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 47/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 48/96 2026-02-17T10:01:57.443 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 49/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-pyasn1-0.4.8-6.el9.noarch 50/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-pyasn1-modules-0.4.8-6.el9.noarch 51/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 52/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 53/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 54/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : xmlsec1-1.2.29-13.el9.x86_64 55/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : xmlsec1-openssl-1.2.29-13.el9.x86_64 56/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 57/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 58/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 59/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 60/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 61/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 62/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 63/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 64/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 65/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 66/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 67/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 68/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 69/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 70/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 71/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 72/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-influxdb-5.3.1-1.el9.noarch 73/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-isodate-0.6.1-3.el9.noarch 74/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 75/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 76/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 77/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 78/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 79/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 80/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 81/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 82/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-msgpack-1.0.3-2.el9.x86_64 83/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 84/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 85/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 86/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 87/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 88/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 89/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-saml-1.16.0-1.el9.noarch 90/96 2026-02-17T10:01:57.444 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 91/96 2026-02-17T10:01:57.445 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 92/96 2026-02-17T10:01:57.445 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 93/96 2026-02-17T10:01:57.445 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-xmlsec-1.3.13-1.el9.x86_64 94/96 2026-02-17T10:01:57.445 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 95/96 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 96/96 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout:Installed: 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: ceph-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: ceph-grafana-dashboards-2:20.3.0-5345.gb58fab16.el9.noarch 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mds-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mgr-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mgr-cephadm-2:20.3.0-5345.gb58fab16.el9.noarch 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mgr-dashboard-2:20.3.0-5345.gb58fab16.el9.noarch 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5345.gb58fab16.el9.noarch 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mgr-k8sevents-2:20.3.0-5345.gb58fab16.el9.noarch 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mgr-modules-core-2:20.3.0-5345.gb58fab16.el9.noarch 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mgr-rook-2:20.3.0-5345.gb58fab16.el9.noarch 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: ceph-mon-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: ceph-osd-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: ceph-osd-classic-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: ceph-osd-crimson-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: ceph-prometheus-alerts-2:20.3.0-5345.gb58fab16.el9.noarch 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: ceph-volume-2:20.3.0-5345.gb58fab16.el9.noarch 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: cephadm-2:20.3.0-5345.gb58fab16.el9.noarch 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: cryptsetup-2.8.1-2.el9.x86_64 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-17T10:01:57.513 INFO:teuthology.orchestra.run.trial103.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: lua-5.4.4-4.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-devel-3.9.25-2.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-influxdb-5.3.1-1.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-isodate-0.6.1-3.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-17T10:01:57.514 INFO:teuthology.orchestra.run.trial103.stdout: python3-lxml-4.6.5-3.el9.x86_64 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-msgpack-1.0.3-2.el9.x86_64 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-ply-3.11-14.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-pyasn1-0.4.8-6.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-pyasn1-modules-0.4.8-6.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-saml-1.16.0-1.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-xmlsec-1.3.13-1.el9.x86_64 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: unzip-6.0-59.el9.x86_64 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: xmlsec1-1.2.29-13.el9.x86_64 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: xmlsec1-openssl-1.2.29-13.el9.x86_64 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: zip-3.0-35.el9.x86_64 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:01:57.515 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:01:57.579 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-base 2026-02-17T10:01:57.890 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:32 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:01:57.956 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-base-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:01:57.973 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:01:57.975 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:01:57.975 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:01:57.996 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install cephadm 2026-02-17T10:01:58.310 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:33 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:01:58.375 INFO:teuthology.orchestra.run.trial103.stdout:Package cephadm-2:20.3.0-5345.gb58fab16.el9.noarch is already installed. 2026-02-17T10:01:58.393 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:01:58.393 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:01:58.393 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:01:58.414 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-immutable-object-cache 2026-02-17T10:01:58.721 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:33 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:01:58.804 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:01:58.804 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:01:58.804 INFO:teuthology.orchestra.run.trial103.stdout: Package Arch Version Repo Size 2026-02-17T10:01:58.804 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:01:58.804 INFO:teuthology.orchestra.run.trial103.stdout:Installing: 2026-02-17T10:01:58.804 INFO:teuthology.orchestra.run.trial103.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 149 k 2026-02-17T10:01:58.804 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:01:58.804 INFO:teuthology.orchestra.run.trial103.stdout:Transaction Summary 2026-02-17T10:01:58.804 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:01:58.804 INFO:teuthology.orchestra.run.trial103.stdout:Install 1 Package 2026-02-17T10:01:58.804 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:01:58.804 INFO:teuthology.orchestra.run.trial103.stdout:Total download size: 149 k 2026-02-17T10:01:58.804 INFO:teuthology.orchestra.run.trial103.stdout:Installed size: 455 k 2026-02-17T10:01:58.804 INFO:teuthology.orchestra.run.trial103.stdout:Downloading Packages: 2026-02-17T10:01:58.890 INFO:teuthology.orchestra.run.trial103.stdout:ceph-immutable-object-cache-20.3.0-5345.gb58fab 1.7 MB/s | 149 kB 00:00 2026-02-17T10:01:58.891 INFO:teuthology.orchestra.run.trial103.stdout:-------------------------------------------------------------------------------- 2026-02-17T10:01:58.891 INFO:teuthology.orchestra.run.trial103.stdout:Total 1.7 MB/s | 149 kB 00:00 2026-02-17T10:01:58.891 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction check 2026-02-17T10:01:58.895 INFO:teuthology.orchestra.run.trial103.stdout:Transaction check succeeded. 2026-02-17T10:01:58.896 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction test 2026-02-17T10:01:58.926 INFO:teuthology.orchestra.run.trial103.stdout:Transaction test succeeded. 2026-02-17T10:01:58.927 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction 2026-02-17T10:01:58.981 INFO:teuthology.orchestra.run.trial103.stdout: Preparing : 1/1 2026-02-17T10:01:58.983 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5345.gb58fab16. 1/1 2026-02-17T10:01:59.006 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5345.gb58fab16. 1/1 2026-02-17T10:01:59.006 INFO:teuthology.orchestra.run.trial103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T10:01:59.006 INFO:teuthology.orchestra.run.trial103.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-17T10:01:59.006 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:01:59.302 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5345.gb58fab16. 1/1 2026-02-17T10:01:59.302 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:01:59.302 INFO:teuthology.orchestra.run.trial103.stdout:Installed: 2026-02-17T10:01:59.302 INFO:teuthology.orchestra.run.trial103.stdout: ceph-immutable-object-cache-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:01:59.303 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:01:59.303 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:01:59.330 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-mgr 2026-02-17T10:01:59.640 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:34 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:01:59.705 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-mgr-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:01:59.723 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:01:59.724 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:01:59.724 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:01:59.745 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-mgr-dashboard 2026-02-17T10:02:00.061 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:00.127 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-mgr-dashboard-2:20.3.0-5345.gb58fab16.el9.noarch is already installed. 2026-02-17T10:02:00.144 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:00.145 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:02:00.145 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:00.167 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-17T10:02:00.475 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:00.541 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5345.gb58fab16.el9.noarch is already installed. 2026-02-17T10:02:00.558 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:00.558 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:02:00.559 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:00.580 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-mgr-rook 2026-02-17T10:02:00.889 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:35 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:00.955 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-mgr-rook-2:20.3.0-5345.gb58fab16.el9.noarch is already installed. 2026-02-17T10:02:00.973 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:00.974 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:02:00.974 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:00.996 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-mgr-cephadm 2026-02-17T10:02:01.301 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:36 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:01.365 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-mgr-cephadm-2:20.3.0-5345.gb58fab16.el9.noarch is already installed. 2026-02-17T10:02:01.382 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:01.383 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:02:01.383 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:01.404 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-osd 2026-02-17T10:02:01.715 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:36 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:01.779 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-osd-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:02:01.796 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:01.797 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:02:01.797 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:01.818 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-osd-classic 2026-02-17T10:02:02.129 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:37 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:02.196 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-osd-classic-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:02:02.213 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:02.214 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:02:02.214 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:02.236 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-fuse 2026-02-17T10:02:02.547 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:37 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:02.632 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:02.633 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:02.633 INFO:teuthology.orchestra.run.trial103.stdout: Package Arch Version Repo Size 2026-02-17T10:02:02.633 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:02.633 INFO:teuthology.orchestra.run.trial103.stdout:Installing: 2026-02-17T10:02:02.633 INFO:teuthology.orchestra.run.trial103.stdout: ceph-fuse x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 914 k 2026-02-17T10:02:02.633 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:02.633 INFO:teuthology.orchestra.run.trial103.stdout:Transaction Summary 2026-02-17T10:02:02.633 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:02.633 INFO:teuthology.orchestra.run.trial103.stdout:Install 1 Package 2026-02-17T10:02:02.634 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:02.634 INFO:teuthology.orchestra.run.trial103.stdout:Total download size: 914 k 2026-02-17T10:02:02.634 INFO:teuthology.orchestra.run.trial103.stdout:Installed size: 2.9 M 2026-02-17T10:02:02.634 INFO:teuthology.orchestra.run.trial103.stdout:Downloading Packages: 2026-02-17T10:02:02.753 INFO:teuthology.orchestra.run.trial103.stdout:ceph-fuse-20.3.0-5345.gb58fab16.el9.x86_64.rpm 7.4 MB/s | 914 kB 00:00 2026-02-17T10:02:02.754 INFO:teuthology.orchestra.run.trial103.stdout:-------------------------------------------------------------------------------- 2026-02-17T10:02:02.754 INFO:teuthology.orchestra.run.trial103.stdout:Total 7.4 MB/s | 914 kB 00:00 2026-02-17T10:02:02.754 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction check 2026-02-17T10:02:02.759 INFO:teuthology.orchestra.run.trial103.stdout:Transaction check succeeded. 2026-02-17T10:02:02.759 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction test 2026-02-17T10:02:02.792 INFO:teuthology.orchestra.run.trial103.stdout:Transaction test succeeded. 2026-02-17T10:02:02.792 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction 2026-02-17T10:02:02.882 INFO:teuthology.orchestra.run.trial103.stdout: Preparing : 1/1 2026-02-17T10:02:02.899 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-fuse-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:03.139 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:03.195 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-fuse-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:03.195 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:03.195 INFO:teuthology.orchestra.run.trial103.stdout:Installed: 2026-02-17T10:02:03.196 INFO:teuthology.orchestra.run.trial103.stdout: ceph-fuse-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:02:03.196 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:03.196 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:03.224 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-volume 2026-02-17T10:02:03.538 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:03.602 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-volume-2:20.3.0-5345.gb58fab16.el9.noarch is already installed. 2026-02-17T10:02:03.619 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:03.620 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:02:03.620 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:03.642 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install librados-devel 2026-02-17T10:02:03.954 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:38 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:04.036 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:04.036 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:04.036 INFO:teuthology.orchestra.run.trial103.stdout: Package Arch Version Repo Size 2026-02-17T10:02:04.037 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:04.037 INFO:teuthology.orchestra.run.trial103.stdout:Installing: 2026-02-17T10:02:04.037 INFO:teuthology.orchestra.run.trial103.stdout: librados-devel x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 124 k 2026-02-17T10:02:04.037 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:04.037 INFO:teuthology.orchestra.run.trial103.stdout:Transaction Summary 2026-02-17T10:02:04.037 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:04.037 INFO:teuthology.orchestra.run.trial103.stdout:Install 1 Package 2026-02-17T10:02:04.037 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:04.037 INFO:teuthology.orchestra.run.trial103.stdout:Total download size: 124 k 2026-02-17T10:02:04.037 INFO:teuthology.orchestra.run.trial103.stdout:Installed size: 434 k 2026-02-17T10:02:04.037 INFO:teuthology.orchestra.run.trial103.stdout:Downloading Packages: 2026-02-17T10:02:04.122 INFO:teuthology.orchestra.run.trial103.stdout:librados-devel-20.3.0-5345.gb58fab16.el9.x86_64 1.4 MB/s | 124 kB 00:00 2026-02-17T10:02:04.122 INFO:teuthology.orchestra.run.trial103.stdout:-------------------------------------------------------------------------------- 2026-02-17T10:02:04.122 INFO:teuthology.orchestra.run.trial103.stdout:Total 1.4 MB/s | 124 kB 00:00 2026-02-17T10:02:04.123 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction check 2026-02-17T10:02:04.125 INFO:teuthology.orchestra.run.trial103.stdout:Transaction check succeeded. 2026-02-17T10:02:04.125 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction test 2026-02-17T10:02:04.158 INFO:teuthology.orchestra.run.trial103.stdout:Transaction test succeeded. 2026-02-17T10:02:04.158 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction 2026-02-17T10:02:04.213 INFO:teuthology.orchestra.run.trial103.stdout: Preparing : 1/1 2026-02-17T10:02:04.229 INFO:teuthology.orchestra.run.trial103.stdout: Installing : librados-devel-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:04.309 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: librados-devel-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:04.367 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : librados-devel-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:04.368 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:04.368 INFO:teuthology.orchestra.run.trial103.stdout:Installed: 2026-02-17T10:02:04.368 INFO:teuthology.orchestra.run.trial103.stdout: librados-devel-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:02:04.368 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:04.368 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:04.397 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install libcephfs2 2026-02-17T10:02:04.702 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:39 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:04.766 INFO:teuthology.orchestra.run.trial103.stdout:Package libcephfs2-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:02:04.784 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:04.784 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:02:04.784 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:04.806 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install libcephfs-devel 2026-02-17T10:02:05.120 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:05.203 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:05.204 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:05.204 INFO:teuthology.orchestra.run.trial103.stdout: Package Arch Version Repo Size 2026-02-17T10:02:05.204 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:05.204 INFO:teuthology.orchestra.run.trial103.stdout:Installing: 2026-02-17T10:02:05.204 INFO:teuthology.orchestra.run.trial103.stdout: libcephfs-devel x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 37 k 2026-02-17T10:02:05.204 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:05.204 INFO:teuthology.orchestra.run.trial103.stdout:Transaction Summary 2026-02-17T10:02:05.204 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:05.204 INFO:teuthology.orchestra.run.trial103.stdout:Install 1 Package 2026-02-17T10:02:05.205 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:05.205 INFO:teuthology.orchestra.run.trial103.stdout:Total download size: 37 k 2026-02-17T10:02:05.205 INFO:teuthology.orchestra.run.trial103.stdout:Installed size: 171 k 2026-02-17T10:02:05.205 INFO:teuthology.orchestra.run.trial103.stdout:Downloading Packages: 2026-02-17T10:02:05.269 INFO:teuthology.orchestra.run.trial103.stdout:libcephfs-devel-20.3.0-5345.gb58fab16.el9.x86_6 574 kB/s | 37 kB 00:00 2026-02-17T10:02:05.270 INFO:teuthology.orchestra.run.trial103.stdout:-------------------------------------------------------------------------------- 2026-02-17T10:02:05.270 INFO:teuthology.orchestra.run.trial103.stdout:Total 567 kB/s | 37 kB 00:00 2026-02-17T10:02:05.270 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction check 2026-02-17T10:02:05.272 INFO:teuthology.orchestra.run.trial103.stdout:Transaction check succeeded. 2026-02-17T10:02:05.272 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction test 2026-02-17T10:02:05.274 INFO:teuthology.orchestra.run.trial103.stdout:Transaction test succeeded. 2026-02-17T10:02:05.274 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction 2026-02-17T10:02:05.296 INFO:teuthology.orchestra.run.trial103.stdout: Preparing : 1/1 2026-02-17T10:02:05.312 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libcephfs-devel-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:05.368 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:05.426 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libcephfs-devel-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:05.426 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:05.426 INFO:teuthology.orchestra.run.trial103.stdout:Installed: 2026-02-17T10:02:05.426 INFO:teuthology.orchestra.run.trial103.stdout: libcephfs-devel-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:02:05.427 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:05.427 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:05.454 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install librados2 2026-02-17T10:02:05.765 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:40 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:05.830 INFO:teuthology.orchestra.run.trial103.stdout:Package librados2-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:02:05.848 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:05.849 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:02:05.849 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:05.871 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install librbd1 2026-02-17T10:02:06.180 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:06.246 INFO:teuthology.orchestra.run.trial103.stdout:Package librbd1-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:02:06.263 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:06.264 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:02:06.264 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:06.287 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install python3-rados 2026-02-17T10:02:06.598 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:41 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:06.663 INFO:teuthology.orchestra.run.trial103.stdout:Package python3-rados-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:02:06.682 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:06.682 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:02:06.682 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:06.704 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install python3-rgw 2026-02-17T10:02:07.024 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:07.089 INFO:teuthology.orchestra.run.trial103.stdout:Package python3-rgw-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:02:07.107 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:07.107 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:02:07.108 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:07.130 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install python3-cephfs 2026-02-17T10:02:07.445 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:07.511 INFO:teuthology.orchestra.run.trial103.stdout:Package python3-cephfs-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:02:07.528 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:07.529 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:02:07.529 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:07.551 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install python3-rbd 2026-02-17T10:02:07.863 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:42 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:07.927 INFO:teuthology.orchestra.run.trial103.stdout:Package python3-rbd-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:02:07.944 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:07.945 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:02:07.945 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:07.967 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install rbd-fuse 2026-02-17T10:02:08.283 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:43 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:08.365 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:08.366 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:08.366 INFO:teuthology.orchestra.run.trial103.stdout: Package Architecture Version Repository Size 2026-02-17T10:02:08.366 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:08.366 INFO:teuthology.orchestra.run.trial103.stdout:Installing: 2026-02-17T10:02:08.366 INFO:teuthology.orchestra.run.trial103.stdout: rbd-fuse x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 87 k 2026-02-17T10:02:08.366 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:08.366 INFO:teuthology.orchestra.run.trial103.stdout:Transaction Summary 2026-02-17T10:02:08.366 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:08.366 INFO:teuthology.orchestra.run.trial103.stdout:Install 1 Package 2026-02-17T10:02:08.367 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:08.367 INFO:teuthology.orchestra.run.trial103.stdout:Total download size: 87 k 2026-02-17T10:02:08.367 INFO:teuthology.orchestra.run.trial103.stdout:Installed size: 236 k 2026-02-17T10:02:08.367 INFO:teuthology.orchestra.run.trial103.stdout:Downloading Packages: 2026-02-17T10:02:08.672 INFO:teuthology.orchestra.run.trial103.stdout:rbd-fuse-20.3.0-5345.gb58fab16.el9.x86_64.rpm 284 kB/s | 87 kB 00:00 2026-02-17T10:02:08.672 INFO:teuthology.orchestra.run.trial103.stdout:-------------------------------------------------------------------------------- 2026-02-17T10:02:08.672 INFO:teuthology.orchestra.run.trial103.stdout:Total 283 kB/s | 87 kB 00:00 2026-02-17T10:02:08.673 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction check 2026-02-17T10:02:08.676 INFO:teuthology.orchestra.run.trial103.stdout:Transaction check succeeded. 2026-02-17T10:02:08.677 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction test 2026-02-17T10:02:08.707 INFO:teuthology.orchestra.run.trial103.stdout:Transaction test succeeded. 2026-02-17T10:02:08.708 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction 2026-02-17T10:02:08.759 INFO:teuthology.orchestra.run.trial103.stdout: Preparing : 1/1 2026-02-17T10:02:08.776 INFO:teuthology.orchestra.run.trial103.stdout: Installing : rbd-fuse-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:08.858 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:08.916 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : rbd-fuse-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:08.916 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:08.916 INFO:teuthology.orchestra.run.trial103.stdout:Installed: 2026-02-17T10:02:08.916 INFO:teuthology.orchestra.run.trial103.stdout: rbd-fuse-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:02:08.916 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:08.916 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:08.946 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install rbd-mirror 2026-02-17T10:02:09.261 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:44 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:09.344 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:09.344 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:09.344 INFO:teuthology.orchestra.run.trial103.stdout: Package Arch Version Repo Size 2026-02-17T10:02:09.345 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:09.345 INFO:teuthology.orchestra.run.trial103.stdout:Installing: 2026-02-17T10:02:09.345 INFO:teuthology.orchestra.run.trial103.stdout: rbd-mirror x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 2.6 M 2026-02-17T10:02:09.345 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:09.345 INFO:teuthology.orchestra.run.trial103.stdout:Transaction Summary 2026-02-17T10:02:09.345 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:09.345 INFO:teuthology.orchestra.run.trial103.stdout:Install 1 Package 2026-02-17T10:02:09.345 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:09.345 INFO:teuthology.orchestra.run.trial103.stdout:Total download size: 2.6 M 2026-02-17T10:02:09.345 INFO:teuthology.orchestra.run.trial103.stdout:Installed size: 11 M 2026-02-17T10:02:09.345 INFO:teuthology.orchestra.run.trial103.stdout:Downloading Packages: 2026-02-17T10:02:10.464 INFO:teuthology.orchestra.run.trial103.stdout:rbd-mirror-20.3.0-5345.gb58fab16.el9.x86_64.rpm 2.3 MB/s | 2.6 MB 00:01 2026-02-17T10:02:10.465 INFO:teuthology.orchestra.run.trial103.stdout:-------------------------------------------------------------------------------- 2026-02-17T10:02:10.465 INFO:teuthology.orchestra.run.trial103.stdout:Total 2.3 MB/s | 2.6 MB 00:01 2026-02-17T10:02:10.465 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction check 2026-02-17T10:02:10.470 INFO:teuthology.orchestra.run.trial103.stdout:Transaction check succeeded. 2026-02-17T10:02:10.470 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction test 2026-02-17T10:02:10.509 INFO:teuthology.orchestra.run.trial103.stdout:Transaction test succeeded. 2026-02-17T10:02:10.509 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction 2026-02-17T10:02:10.673 INFO:teuthology.orchestra.run.trial103.stdout: Preparing : 1/1 2026-02-17T10:02:10.675 INFO:teuthology.orchestra.run.trial103.stdout: Installing : rbd-mirror-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:10.698 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:10.698 INFO:teuthology.orchestra.run.trial103.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-17T10:02:10.698 INFO:teuthology.orchestra.run.trial103.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-17T10:02:10.698 INFO:teuthology.orchestra.run.trial103.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T10:02:10.698 INFO:teuthology.orchestra.run.trial103.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-17T10:02:10.698 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:11.013 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : rbd-mirror-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:11.013 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:11.014 INFO:teuthology.orchestra.run.trial103.stdout:Installed: 2026-02-17T10:02:11.014 INFO:teuthology.orchestra.run.trial103.stdout: rbd-mirror-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:02:11.014 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:11.014 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:11.042 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install rbd-nbd 2026-02-17T10:02:11.355 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:46 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:11.440 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:11.440 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:11.440 INFO:teuthology.orchestra.run.trial103.stdout: Package Architecture Version Repository Size 2026-02-17T10:02:11.440 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:11.440 INFO:teuthology.orchestra.run.trial103.stdout:Installing: 2026-02-17T10:02:11.440 INFO:teuthology.orchestra.run.trial103.stdout: rbd-nbd x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 167 k 2026-02-17T10:02:11.440 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:11.441 INFO:teuthology.orchestra.run.trial103.stdout:Transaction Summary 2026-02-17T10:02:11.441 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:11.441 INFO:teuthology.orchestra.run.trial103.stdout:Install 1 Package 2026-02-17T10:02:11.441 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:11.441 INFO:teuthology.orchestra.run.trial103.stdout:Total download size: 167 k 2026-02-17T10:02:11.441 INFO:teuthology.orchestra.run.trial103.stdout:Installed size: 496 k 2026-02-17T10:02:11.441 INFO:teuthology.orchestra.run.trial103.stdout:Downloading Packages: 2026-02-17T10:02:11.789 INFO:teuthology.orchestra.run.trial103.stdout:rbd-nbd-20.3.0-5345.gb58fab16.el9.x86_64.rpm 477 kB/s | 167 kB 00:00 2026-02-17T10:02:11.789 INFO:teuthology.orchestra.run.trial103.stdout:-------------------------------------------------------------------------------- 2026-02-17T10:02:11.789 INFO:teuthology.orchestra.run.trial103.stdout:Total 476 kB/s | 167 kB 00:00 2026-02-17T10:02:11.789 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction check 2026-02-17T10:02:11.794 INFO:teuthology.orchestra.run.trial103.stdout:Transaction check succeeded. 2026-02-17T10:02:11.794 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction test 2026-02-17T10:02:11.825 INFO:teuthology.orchestra.run.trial103.stdout:Transaction test succeeded. 2026-02-17T10:02:11.825 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction 2026-02-17T10:02:11.882 INFO:teuthology.orchestra.run.trial103.stdout: Preparing : 1/1 2026-02-17T10:02:11.899 INFO:teuthology.orchestra.run.trial103.stdout: Installing : rbd-nbd-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:11.982 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:12.039 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : rbd-nbd-2:20.3.0-5345.gb58fab16.el9.x86_64 1/1 2026-02-17T10:02:12.039 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:12.040 INFO:teuthology.orchestra.run.trial103.stdout:Installed: 2026-02-17T10:02:12.040 INFO:teuthology.orchestra.run.trial103.stdout: rbd-nbd-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:02:12.040 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:12.040 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:12.068 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-osd-crimson 2026-02-17T10:02:12.380 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:12.446 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-osd-crimson-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:02:12.464 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:12.465 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:02:12.465 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:12.487 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-17T10:02:12.797 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:01:47 ago on Tue 17 Feb 2026 10:00:25 AM UTC. 2026-02-17T10:02:12.881 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:02:12.881 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:12.881 INFO:teuthology.orchestra.run.trial103.stdout: Package Arch Version Repo Size 2026-02-17T10:02:12.881 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:12.881 INFO:teuthology.orchestra.run.trial103.stdout:Installing: 2026-02-17T10:02:12.881 INFO:teuthology.orchestra.run.trial103.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 425 M 2026-02-17T10:02:12.881 INFO:teuthology.orchestra.run.trial103.stdout:Installing weak dependencies: 2026-02-17T10:02:12.881 INFO:teuthology.orchestra.run.trial103.stdout: ceph-debugsource x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 12 M 2026-02-17T10:02:12.881 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:12.881 INFO:teuthology.orchestra.run.trial103.stdout:Transaction Summary 2026-02-17T10:02:12.882 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:02:12.882 INFO:teuthology.orchestra.run.trial103.stdout:Install 2 Packages 2026-02-17T10:02:12.882 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:12.882 INFO:teuthology.orchestra.run.trial103.stdout:Total download size: 437 M 2026-02-17T10:02:12.882 INFO:teuthology.orchestra.run.trial103.stdout:Installed size: 2.6 G 2026-02-17T10:02:12.882 INFO:teuthology.orchestra.run.trial103.stdout:Downloading Packages: 2026-02-17T10:02:16.487 INFO:teuthology.orchestra.run.trial103.stdout:(1/2): ceph-debugsource-20.3.0-5345.gb58fab16.e 3.3 MB/s | 12 MB 00:03 2026-02-17T10:02:29.407 INFO:teuthology.orchestra.run.trial103.stdout:(2/2): ceph-osd-crimson-debuginfo-20.3.0-5345.g 26 MB/s | 425 MB 00:16 2026-02-17T10:02:29.408 INFO:teuthology.orchestra.run.trial103.stdout:-------------------------------------------------------------------------------- 2026-02-17T10:02:29.408 INFO:teuthology.orchestra.run.trial103.stdout:Total 26 MB/s | 437 MB 00:16 2026-02-17T10:02:29.408 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction check 2026-02-17T10:02:29.413 INFO:teuthology.orchestra.run.trial103.stdout:Transaction check succeeded. 2026-02-17T10:02:29.413 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction test 2026-02-17T10:02:30.760 INFO:teuthology.orchestra.run.trial103.stdout:Transaction test succeeded. 2026-02-17T10:02:30.760 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction 2026-02-17T10:02:34.368 INFO:teuthology.orchestra.run.trial103.stdout: Preparing : 1/1 2026-02-17T10:02:56.539 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-debugsource-2:20.3.0-5345.gb58fab16.el9.x86_64 1/2 2026-02-17T10:02:56.557 INFO:teuthology.orchestra.run.trial103.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5345.gb58fab16.e 2/2 2026-02-17T10:02:58.481 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5345.gb58fab16.e 2/2 2026-02-17T10:02:58.481 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-debugsource-2:20.3.0-5345.gb58fab16.el9.x86_64 1/2 2026-02-17T10:02:58.540 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5345.gb58fab16.e 2/2 2026-02-17T10:02:58.540 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:58.540 INFO:teuthology.orchestra.run.trial103.stdout:Installed: 2026-02-17T10:02:58.540 INFO:teuthology.orchestra.run.trial103.stdout: ceph-debugsource-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:02:58.540 INFO:teuthology.orchestra.run.trial103.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:02:58.540 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:02:58.540 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:02:58.615 DEBUG:teuthology.parallel:result is None 2026-02-17T10:02:58.615 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=b58fab16a61897dbc214a1865300c7cef586ea65 2026-02-17T10:02:58.780 DEBUG:teuthology.orchestra.run.trial103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-17T10:02:58.801 INFO:teuthology.orchestra.run.trial103.stdout:20.3.0-5345.gb58fab16.el9 2026-02-17T10:02:58.802 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5345.gb58fab16.el9 2026-02-17T10:02:58.802 INFO:teuthology.task.install:The correct ceph version 20.3.0-5345.gb58fab16 is installed. 2026-02-17T10:02:58.804 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-17T10:02:58.804 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:02:58.804 DEBUG:teuthology.orchestra.run.trial103:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-17T10:02:58.869 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-17T10:02:58.869 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:02:58.869 DEBUG:teuthology.orchestra.run.trial103:> sudo dd of=/usr/bin/daemon-helper 2026-02-17T10:02:58.933 DEBUG:teuthology.orchestra.run.trial103:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-17T10:02:58.997 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-17T10:02:58.997 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:02:58.997 DEBUG:teuthology.orchestra.run.trial103:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-17T10:02:59.061 DEBUG:teuthology.orchestra.run.trial103:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-17T10:02:59.124 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-17T10:02:59.124 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:02:59.124 DEBUG:teuthology.orchestra.run.trial103:> sudo dd of=/usr/bin/stdin-killer 2026-02-17T10:02:59.189 DEBUG:teuthology.orchestra.run.trial103:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-17T10:02:59.252 INFO:teuthology.run_tasks:Running task pexec... 2026-02-17T10:02:59.265 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-17T10:02:59.265 DEBUG:teuthology.orchestra.run.trial103:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-17T10:02:59.293 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial103.front.sepia.ceph.com 2026-02-17T10:02:59.293 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-17T10:02:59.343 DEBUG:teuthology.parallel:result is None 2026-02-17T10:02:59.343 INFO:teuthology.run_tasks:Running task install... 2026-02-17T10:02:59.355 DEBUG:teuthology.task.install:project ceph 2026-02-17T10:02:59.355 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': 'b58fab16a61897dbc214a1865300c7cef586ea65'}} 2026-02-17T10:02:59.355 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': 'b58fab16a61897dbc214a1865300c7cef586ea65'} 2026-02-17T10:02:59.355 INFO:teuthology.task.install:Using flavor: debug 2026-02-17T10:02:59.357 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-17T10:02:59.357 INFO:teuthology.task.install:extra packages: [] 2026-02-17T10:02:59.357 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'b58fab16a61897dbc214a1865300c7cef586ea65', 'tag': None, 'wait_for_package': False} 2026-02-17T10:02:59.358 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=b58fab16a61897dbc214a1865300c7cef586ea65 2026-02-17T10:02:59.486 INFO:teuthology.task.install.rpm:Pulling from https://2.chacra.ceph.com/r/ceph/wip-sjust-seastore-2026-02-16-1771285632/b58fab16a61897dbc214a1865300c7cef586ea65/centos/9/flavors/debug/ 2026-02-17T10:02:59.486 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5345.gb58fab16 2026-02-17T10:02:59.738 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://2.chacra.ceph.com/r/ceph/wip-sjust-seastore-2026-02-16-1771285632/b58fab16a61897dbc214a1865300c7cef586ea65/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-sjust-seastore-2026-02-16-1771285632/b58fab16a61897dbc214a1865300c7cef586ea65/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://2.chacra.ceph.com/r/ceph/wip-sjust-seastore-2026-02-16-1771285632/b58fab16a61897dbc214a1865300c7cef586ea65/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-17T10:02:59.738 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:02:59.738 DEBUG:teuthology.orchestra.run.trial103:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-17T10:02:59.762 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-17T10:02:59.763 DEBUG:teuthology.orchestra.run.trial103:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/b58fab16a61897dbc214a1865300c7cef586ea65/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-17T10:02:59.829 DEBUG:teuthology.orchestra.run.trial103:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-17T10:02:59.894 DEBUG:teuthology.orchestra.run.trial103:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-17T10:02:59.949 INFO:teuthology.orchestra.run.trial103.stdout:check_obsoletes = 1 2026-02-17T10:02:59.959 DEBUG:teuthology.orchestra.run.trial103:> sudo yum clean all 2026-02-17T10:03:00.122 INFO:teuthology.orchestra.run.trial103.stdout:56 files removed 2026-02-17T10:03:00.137 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-radosgw 2026-02-17T10:03:00.773 INFO:teuthology.orchestra.run.trial103.stdout:ceph packages for x86_64 194 kB/s | 94 kB 00:00 2026-02-17T10:03:01.370 INFO:teuthology.orchestra.run.trial103.stdout:ceph noarch packages 43 kB/s | 24 kB 00:00 2026-02-17T10:03:01.881 INFO:teuthology.orchestra.run.trial103.stdout:ceph source packages 4.2 kB/s | 2.1 kB 00:00 2026-02-17T10:03:02.715 INFO:teuthology.orchestra.run.trial103.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-17T10:03:05.958 INFO:teuthology.orchestra.run.trial103.stdout:CentOS Stream 9 - AppStream 9.9 MB/s | 27 MB 00:02 2026-02-17T10:03:09.717 INFO:teuthology.orchestra.run.trial103.stdout:CentOS Stream 9 - CRB 5.7 MB/s | 7.8 MB 00:01 2026-02-17T10:03:10.659 INFO:teuthology.orchestra.run.trial103.stdout:CentOS Stream 9 - Extras packages 93 kB/s | 20 kB 00:00 2026-02-17T10:03:11.416 INFO:teuthology.orchestra.run.trial103.stdout:Extra Packages for Enterprise Linux 29 MB/s | 20 MB 00:00 2026-02-17T10:03:14.585 INFO:teuthology.orchestra.run.trial103.stdout:lab-extras 1.8 MB/s | 50 kB 00:00 2026-02-17T10:03:15.770 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-radosgw-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:15.788 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:15.788 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:15.788 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:15.812 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-test 2026-02-17T10:03:16.125 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:02 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:16.190 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-test-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:16.208 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:16.209 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:16.209 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:16.230 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph 2026-02-17T10:03:16.547 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:02 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:16.612 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:16.630 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:16.631 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:16.631 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:16.652 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-base 2026-02-17T10:03:16.965 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:02 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:17.030 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-base-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:17.048 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:17.049 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:17.049 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:17.070 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install cephadm 2026-02-17T10:03:17.383 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:03 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:17.447 INFO:teuthology.orchestra.run.trial103.stdout:Package cephadm-2:20.3.0-5345.gb58fab16.el9.noarch is already installed. 2026-02-17T10:03:17.465 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:17.466 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:17.466 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:17.487 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-immutable-object-cache 2026-02-17T10:03:17.794 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:03 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:17.859 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-immutable-object-cache-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:17.876 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:17.877 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:17.877 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:17.898 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-mgr 2026-02-17T10:03:18.215 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:04 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:18.280 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-mgr-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:18.298 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:18.299 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:18.299 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:18.321 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-mgr-dashboard 2026-02-17T10:03:18.631 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:04 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:18.695 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-mgr-dashboard-2:20.3.0-5345.gb58fab16.el9.noarch is already installed. 2026-02-17T10:03:18.713 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:18.713 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:18.713 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:18.735 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-17T10:03:19.050 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:05 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:19.116 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5345.gb58fab16.el9.noarch is already installed. 2026-02-17T10:03:19.134 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:19.135 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:19.135 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:19.156 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-mgr-rook 2026-02-17T10:03:19.466 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:05 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:19.532 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-mgr-rook-2:20.3.0-5345.gb58fab16.el9.noarch is already installed. 2026-02-17T10:03:19.550 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:19.551 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:19.551 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:19.572 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-mgr-cephadm 2026-02-17T10:03:19.887 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:05 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:19.952 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-mgr-cephadm-2:20.3.0-5345.gb58fab16.el9.noarch is already installed. 2026-02-17T10:03:19.970 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:19.971 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:19.971 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:19.992 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-osd 2026-02-17T10:03:20.309 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:06 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:20.373 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-osd-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:20.391 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:20.392 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:20.392 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:20.414 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-osd-classic 2026-02-17T10:03:20.728 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:06 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:20.795 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-osd-classic-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:20.812 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:20.813 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:20.813 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:20.834 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-fuse 2026-02-17T10:03:21.149 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:07 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:21.213 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-fuse-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:21.231 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:21.231 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:21.232 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:21.254 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install ceph-volume 2026-02-17T10:03:21.561 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:07 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:21.627 INFO:teuthology.orchestra.run.trial103.stdout:Package ceph-volume-2:20.3.0-5345.gb58fab16.el9.noarch is already installed. 2026-02-17T10:03:21.645 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:21.646 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:21.646 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:21.667 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install librados-devel 2026-02-17T10:03:21.982 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:07 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:22.048 INFO:teuthology.orchestra.run.trial103.stdout:Package librados-devel-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:22.065 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:22.066 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:22.067 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:22.088 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install libcephfs2 2026-02-17T10:03:22.403 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:08 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:22.466 INFO:teuthology.orchestra.run.trial103.stdout:Package libcephfs2-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:22.484 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:22.484 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:22.485 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:22.506 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install libcephfs-devel 2026-02-17T10:03:22.821 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:08 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:22.889 INFO:teuthology.orchestra.run.trial103.stdout:Package libcephfs-devel-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:22.907 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:22.908 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:22.908 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:22.930 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install librados2 2026-02-17T10:03:23.245 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:09 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:23.310 INFO:teuthology.orchestra.run.trial103.stdout:Package librados2-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:23.327 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:23.328 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:23.328 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:23.351 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install librbd1 2026-02-17T10:03:23.667 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:09 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:23.732 INFO:teuthology.orchestra.run.trial103.stdout:Package librbd1-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:23.750 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:23.751 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:23.751 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:23.773 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install python3-rados 2026-02-17T10:03:24.083 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:10 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:24.147 INFO:teuthology.orchestra.run.trial103.stdout:Package python3-rados-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:24.165 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:24.165 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:24.166 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:24.187 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install python3-rgw 2026-02-17T10:03:24.501 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:10 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:24.565 INFO:teuthology.orchestra.run.trial103.stdout:Package python3-rgw-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:24.583 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:24.584 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:24.584 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:24.606 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install python3-cephfs 2026-02-17T10:03:24.922 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:10 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:24.987 INFO:teuthology.orchestra.run.trial103.stdout:Package python3-cephfs-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:25.005 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:25.005 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:25.005 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:25.028 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install python3-rbd 2026-02-17T10:03:25.338 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:11 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:25.403 INFO:teuthology.orchestra.run.trial103.stdout:Package python3-rbd-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:25.420 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:25.421 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:25.421 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:25.443 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install rbd-fuse 2026-02-17T10:03:25.756 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:11 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:25.821 INFO:teuthology.orchestra.run.trial103.stdout:Package rbd-fuse-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:25.839 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:25.840 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:25.840 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:25.862 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install rbd-mirror 2026-02-17T10:03:26.177 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:12 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:26.242 INFO:teuthology.orchestra.run.trial103.stdout:Package rbd-mirror-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:26.260 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:26.260 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:26.261 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:26.283 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install rbd-nbd 2026-02-17T10:03:26.600 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:12 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:03:26.664 INFO:teuthology.orchestra.run.trial103.stdout:Package rbd-nbd-2:20.3.0-5345.gb58fab16.el9.x86_64 is already installed. 2026-02-17T10:03:26.683 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:03:26.683 INFO:teuthology.orchestra.run.trial103.stdout:Nothing to do. 2026-02-17T10:03:26.683 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:03:26.709 DEBUG:teuthology.parallel:result is None 2026-02-17T10:03:26.709 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=b58fab16a61897dbc214a1865300c7cef586ea65 2026-02-17T10:03:26.874 DEBUG:teuthology.orchestra.run.trial103:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-17T10:03:26.896 INFO:teuthology.orchestra.run.trial103.stdout:20.3.0-5345.gb58fab16.el9 2026-02-17T10:03:26.896 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5345.gb58fab16.el9 2026-02-17T10:03:26.896 INFO:teuthology.task.install:The correct ceph version 20.3.0-5345.gb58fab16 is installed. 2026-02-17T10:03:26.899 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-17T10:03:26.899 INFO:teuthology.run_tasks:Running task ceph... 2026-02-17T10:03:26.998 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-17T10:03:26.998 DEBUG:teuthology.orchestra.run.trial103:> sudo chmod 777 /var/log/ceph 2026-02-17T10:03:27.023 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-17T10:03:27.023 DEBUG:teuthology.orchestra.run.trial103:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-17T10:03:27.087 INFO:tasks.ceph:Creating extra log directories... 2026-02-17T10:03:27.087 DEBUG:teuthology.orchestra.run.trial103:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-17T10:03:27.163 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-17T10:03:27.163 INFO:tasks.ceph:config {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-17T10:03:27.163 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/matan-2026-02-17_08:51:22-crimson-rados-wip-sjust-seastore-2026-02-16-1771285632-distro-debug-trial/52100', 'branch': 'wip-sjust-seastore-2026-02-16-1771285632', 'description': 'crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} perf_ignorelist settings/optimized workloads/radosbench_4K_rand_read}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '52100', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'matan-2026-02-17_08:51:22-crimson-rados-wip-sjust-seastore-2026-02-16-1771285632-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'wip-sjust-seastore-2026-02-16-1771285632'}, 'ceph': {'conf': {'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': 'b58fab16a61897dbc214a1865300c7cef586ea65'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'global': {'osd crush chooseleaf type': 0, 'osd pool default pg num': 128, 'osd pool default pgp num': 128, 'osd pool default size': 2}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': 'b58fab16a61897dbc214a1865300c7cef586ea65'}}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'ebc89147fc1a1731b84ef7e7003e19f704ba15c0'}}, 'owner': 'scheduled_matan@soko04.front.sepia.ceph.com', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'mgr.x', 'osd.0', 'osd.1', 'osd.2', 'client.0']], 'seed': 2314, 'sha1': 'b58fab16a61897dbc214a1865300c7cef586ea65', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_ebc89147fc1a1731b84ef7e7003e19f704ba15c0/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': 'ebc89147fc1a1731b84ef7e7003e19f704ba15c0', 'targets': {'trial103.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCcwH+06X39oKcH6RUHgOLQ7XUqt/xbjS5DGxwrZ+qCsHpgSxFnPJQ9Fcw3+GPzTyIQNOZIkxDmFVn1EEahoK6U='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': 'b58fab16a61897dbc214a1865300c7cef586ea65'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'global': {'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'auth supported': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': '0/0', 'debug ms': '0/0', 'debug paxos': '0/0', 'osd pool default crimson': True}}, 'wait-for-scrub': False, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)', '\\(PG_', '\\(OSD_', '\\(OBJECT_', 'overall HEALTH'], 'sha1': 'b58fab16a61897dbc214a1865300c7cef586ea65', 'cluster': 'ceph'}}, {'ssh_keys': None}, {'cbt': {'benchmarks': {'radosbench': {'concurrent_ops': 4, 'concurrent_procs': 2, 'op_size': [4096], 'pool_monitoring_list': ['collectl'], 'pool_profile': 'replicated', 'readmode': 'rand', 'run_monitoring_list': ['collectl'], 'time': 60, 'write_only': False}}, 'cluster': {'iterations': 1, 'osds_per_node': 3, 'pool_profiles': {'replicated': {'pg_size': 256, 'pgp_size': 256, 'replication': 'replicated'}}, 'user': 'ubuntu'}, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-17_08:51:22', 'tube': 'trial', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.3682'} 2026-02-17T10:03:27.164 DEBUG:teuthology.orchestra.run.trial103:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-17T10:03:27.208 DEBUG:teuthology.orchestra.run.trial103:> sudo install -d -m0777 -- /var/run/ceph 2026-02-17T10:03:27.275 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:03:27.275 DEBUG:teuthology.orchestra.run.trial103:> dd if=/scratch_devs of=/dev/stdout 2026-02-17T10:03:27.332 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-17T10:03:27.332 DEBUG:teuthology.orchestra.run.trial103:> stat /dev/vg_nvme/lv_1 2026-02-17T10:03:27.390 INFO:teuthology.orchestra.run.trial103.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-17T10:03:27.390 INFO:teuthology.orchestra.run.trial103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T10:03:27.390 INFO:teuthology.orchestra.run.trial103.stdout:Device: 6h/6d Inode: 1305 Links: 1 2026-02-17T10:03:27.390 INFO:teuthology.orchestra.run.trial103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T10:03:27.390 INFO:teuthology.orchestra.run.trial103.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T10:03:27.390 INFO:teuthology.orchestra.run.trial103.stdout:Access: 2026-02-17 10:02:10.837334315 +0000 2026-02-17T10:03:27.390 INFO:teuthology.orchestra.run.trial103.stdout:Modify: 2026-02-17 10:00:31.137891440 +0000 2026-02-17T10:03:27.390 INFO:teuthology.orchestra.run.trial103.stdout:Change: 2026-02-17 10:00:31.137891440 +0000 2026-02-17T10:03:27.390 INFO:teuthology.orchestra.run.trial103.stdout: Birth: 2026-02-17 10:00:31.137891440 +0000 2026-02-17T10:03:27.391 DEBUG:teuthology.orchestra.run.trial103:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-17T10:03:27.452 INFO:teuthology.orchestra.run.trial103.stderr:1+0 records in 2026-02-17T10:03:27.453 INFO:teuthology.orchestra.run.trial103.stderr:1+0 records out 2026-02-17T10:03:27.453 INFO:teuthology.orchestra.run.trial103.stderr:512 bytes copied, 0.000160067 s, 3.2 MB/s 2026-02-17T10:03:27.454 DEBUG:teuthology.orchestra.run.trial103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-17T10:03:27.510 DEBUG:teuthology.orchestra.run.trial103:> stat /dev/vg_nvme/lv_2 2026-02-17T10:03:27.565 INFO:teuthology.orchestra.run.trial103.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-17T10:03:27.566 INFO:teuthology.orchestra.run.trial103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T10:03:27.566 INFO:teuthology.orchestra.run.trial103.stdout:Device: 6h/6d Inode: 1313 Links: 1 2026-02-17T10:03:27.566 INFO:teuthology.orchestra.run.trial103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T10:03:27.566 INFO:teuthology.orchestra.run.trial103.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T10:03:27.566 INFO:teuthology.orchestra.run.trial103.stdout:Access: 2026-02-17 10:02:10.837334315 +0000 2026-02-17T10:03:27.566 INFO:teuthology.orchestra.run.trial103.stdout:Modify: 2026-02-17 10:00:31.137891440 +0000 2026-02-17T10:03:27.566 INFO:teuthology.orchestra.run.trial103.stdout:Change: 2026-02-17 10:00:31.137891440 +0000 2026-02-17T10:03:27.566 INFO:teuthology.orchestra.run.trial103.stdout: Birth: 2026-02-17 10:00:31.137891440 +0000 2026-02-17T10:03:27.567 DEBUG:teuthology.orchestra.run.trial103:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-17T10:03:27.628 INFO:teuthology.orchestra.run.trial103.stderr:1+0 records in 2026-02-17T10:03:27.629 INFO:teuthology.orchestra.run.trial103.stderr:1+0 records out 2026-02-17T10:03:27.629 INFO:teuthology.orchestra.run.trial103.stderr:512 bytes copied, 0.000193379 s, 2.6 MB/s 2026-02-17T10:03:27.630 DEBUG:teuthology.orchestra.run.trial103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-17T10:03:27.687 DEBUG:teuthology.orchestra.run.trial103:> stat /dev/vg_nvme/lv_3 2026-02-17T10:03:27.743 INFO:teuthology.orchestra.run.trial103.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-17T10:03:27.743 INFO:teuthology.orchestra.run.trial103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T10:03:27.743 INFO:teuthology.orchestra.run.trial103.stdout:Device: 6h/6d Inode: 1299 Links: 1 2026-02-17T10:03:27.744 INFO:teuthology.orchestra.run.trial103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T10:03:27.744 INFO:teuthology.orchestra.run.trial103.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T10:03:27.744 INFO:teuthology.orchestra.run.trial103.stdout:Access: 2026-02-17 10:02:10.837334315 +0000 2026-02-17T10:03:27.744 INFO:teuthology.orchestra.run.trial103.stdout:Modify: 2026-02-17 10:00:31.136891455 +0000 2026-02-17T10:03:27.744 INFO:teuthology.orchestra.run.trial103.stdout:Change: 2026-02-17 10:00:31.136891455 +0000 2026-02-17T10:03:27.744 INFO:teuthology.orchestra.run.trial103.stdout: Birth: 2026-02-17 10:00:31.136891455 +0000 2026-02-17T10:03:27.744 DEBUG:teuthology.orchestra.run.trial103:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-17T10:03:27.806 INFO:teuthology.orchestra.run.trial103.stderr:1+0 records in 2026-02-17T10:03:27.806 INFO:teuthology.orchestra.run.trial103.stderr:1+0 records out 2026-02-17T10:03:27.806 INFO:teuthology.orchestra.run.trial103.stderr:512 bytes copied, 0.000144157 s, 3.6 MB/s 2026-02-17T10:03:27.808 DEBUG:teuthology.orchestra.run.trial103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-17T10:03:27.866 DEBUG:teuthology.orchestra.run.trial103:> stat /dev/vg_nvme/lv_4 2026-02-17T10:03:27.922 INFO:teuthology.orchestra.run.trial103.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-17T10:03:27.922 INFO:teuthology.orchestra.run.trial103.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-17T10:03:27.923 INFO:teuthology.orchestra.run.trial103.stdout:Device: 6h/6d Inode: 1291 Links: 1 2026-02-17T10:03:27.923 INFO:teuthology.orchestra.run.trial103.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-17T10:03:27.923 INFO:teuthology.orchestra.run.trial103.stdout:Context: system_u:object_r:device_t:s0 2026-02-17T10:03:27.923 INFO:teuthology.orchestra.run.trial103.stdout:Access: 2026-02-17 10:02:10.838334299 +0000 2026-02-17T10:03:27.923 INFO:teuthology.orchestra.run.trial103.stdout:Modify: 2026-02-17 10:00:31.136891455 +0000 2026-02-17T10:03:27.923 INFO:teuthology.orchestra.run.trial103.stdout:Change: 2026-02-17 10:00:31.136891455 +0000 2026-02-17T10:03:27.923 INFO:teuthology.orchestra.run.trial103.stdout: Birth: 2026-02-17 10:00:31.136891455 +0000 2026-02-17T10:03:27.923 DEBUG:teuthology.orchestra.run.trial103:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-17T10:03:27.987 INFO:teuthology.orchestra.run.trial103.stderr:1+0 records in 2026-02-17T10:03:27.987 INFO:teuthology.orchestra.run.trial103.stderr:1+0 records out 2026-02-17T10:03:27.988 INFO:teuthology.orchestra.run.trial103.stderr:512 bytes copied, 0.00012964 s, 3.9 MB/s 2026-02-17T10:03:27.989 DEBUG:teuthology.orchestra.run.trial103:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-17T10:03:28.046 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-17T10:03:28.046 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial103.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-17T10:03:28.046 INFO:tasks.ceph:Generating config... 2026-02-17T10:03:28.049 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-17T10:03:28.049 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-17T10:03:28.049 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-17T10:03:28.049 INFO:tasks.ceph:[osd] debug filestore = 0/0 2026-02-17T10:03:28.049 INFO:tasks.ceph:[osd] debug journal = 0/0 2026-02-17T10:03:28.049 INFO:tasks.ceph:[osd] debug ms = 0/0 2026-02-17T10:03:28.049 INFO:tasks.ceph:[osd] debug osd = 0/0 2026-02-17T10:03:28.049 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-17T10:03:28.049 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-17T10:03:28.049 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-17T10:03:28.050 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-17T10:03:28.050 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-17T10:03:28.050 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-17T10:03:28.050 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-17T10:03:28.050 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-17T10:03:28.050 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-17T10:03:28.050 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-17T10:03:28.050 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-17T10:03:28.050 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-17T10:03:28.050 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-17T10:03:28.050 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-17T10:03:28.050 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-17T10:03:28.050 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-17T10:03:28.051 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-17T10:03:28.051 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-17T10:03:28.051 INFO:tasks.ceph:[osd] seastore main device type = RANDOM_BLOCK_SSD 2026-02-17T10:03:28.051 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-17T10:03:28.051 INFO:tasks.ceph:[global] auth client required = none 2026-02-17T10:03:28.051 INFO:tasks.ceph:[global] auth cluster required = none 2026-02-17T10:03:28.051 INFO:tasks.ceph:[global] auth service required = none 2026-02-17T10:03:28.051 INFO:tasks.ceph:[global] auth supported = none 2026-02-17T10:03:28.051 INFO:tasks.ceph:[global] debug asok = 0/0 2026-02-17T10:03:28.051 INFO:tasks.ceph:[global] debug auth = 0/0 2026-02-17T10:03:28.051 INFO:tasks.ceph:[global] debug bdev = 0/0 2026-02-17T10:03:28.051 INFO:tasks.ceph:[global] debug bluefs = 0/0 2026-02-17T10:03:28.052 INFO:tasks.ceph:[global] debug bluestore = 0/0 2026-02-17T10:03:28.052 INFO:tasks.ceph:[global] debug buffer = 0/0 2026-02-17T10:03:28.052 INFO:tasks.ceph:[global] debug civetweb = 0/0 2026-02-17T10:03:28.052 INFO:tasks.ceph:[global] debug client = 0/0 2026-02-17T10:03:28.052 INFO:tasks.ceph:[global] debug compressor = 0/0 2026-02-17T10:03:28.052 INFO:tasks.ceph:[global] debug context = 0/0 2026-02-17T10:03:28.052 INFO:tasks.ceph:[global] debug crush = 0/0 2026-02-17T10:03:28.052 INFO:tasks.ceph:[global] debug crypto = 0/0 2026-02-17T10:03:28.052 INFO:tasks.ceph:[global] debug dpdk = 0/0 2026-02-17T10:03:28.052 INFO:tasks.ceph:[global] debug eventtrace = 0/0 2026-02-17T10:03:28.052 INFO:tasks.ceph:[global] debug filer = 0/0 2026-02-17T10:03:28.052 INFO:tasks.ceph:[global] debug filestore = 0/0 2026-02-17T10:03:28.052 INFO:tasks.ceph:[global] debug finisher = 0/0 2026-02-17T10:03:28.053 INFO:tasks.ceph:[global] debug fuse = 0/0 2026-02-17T10:03:28.053 INFO:tasks.ceph:[global] debug heartbeatmap = 0/0 2026-02-17T10:03:28.053 INFO:tasks.ceph:[global] debug javaclient = 0/0 2026-02-17T10:03:28.053 INFO:tasks.ceph:[global] debug journal = 0/0 2026-02-17T10:03:28.053 INFO:tasks.ceph:[global] debug journaler = 0/0 2026-02-17T10:03:28.053 INFO:tasks.ceph:[global] debug kstore = 0/0 2026-02-17T10:03:28.053 INFO:tasks.ceph:[global] debug leveldb = 0/0 2026-02-17T10:03:28.053 INFO:tasks.ceph:[global] debug lockdep = 0/0 2026-02-17T10:03:28.053 INFO:tasks.ceph:[global] debug mds = 0/0 2026-02-17T10:03:28.053 INFO:tasks.ceph:[global] debug mds balancer = 0/0 2026-02-17T10:03:28.053 INFO:tasks.ceph:[global] debug mds locker = 0/0 2026-02-17T10:03:28.053 INFO:tasks.ceph:[global] debug mds log = 0/0 2026-02-17T10:03:28.053 INFO:tasks.ceph:[global] debug mds log expire = 0/0 2026-02-17T10:03:28.054 INFO:tasks.ceph:[global] debug mds migrator = 0/0 2026-02-17T10:03:28.054 INFO:tasks.ceph:[global] debug memdb = 0/0 2026-02-17T10:03:28.054 INFO:tasks.ceph:[global] debug mgr = 0/0 2026-02-17T10:03:28.054 INFO:tasks.ceph:[global] debug mgrc = 0/0 2026-02-17T10:03:28.054 INFO:tasks.ceph:[global] debug mon = 0/0 2026-02-17T10:03:28.054 INFO:tasks.ceph:[global] debug monc = 0/0 2026-02-17T10:03:28.054 INFO:tasks.ceph:[global] debug ms = 0/0 2026-02-17T10:03:28.054 INFO:tasks.ceph:[global] debug objclass = 0/0 2026-02-17T10:03:28.054 INFO:tasks.ceph:[global] debug objectcacher = 0/0 2026-02-17T10:03:28.054 INFO:tasks.ceph:[global] debug objecter = 0/0 2026-02-17T10:03:28.054 INFO:tasks.ceph:[global] debug optracker = 0/0 2026-02-17T10:03:28.054 INFO:tasks.ceph:[global] debug osd = 0/0 2026-02-17T10:03:28.054 INFO:tasks.ceph:[global] debug paxos = 0/0 2026-02-17T10:03:28.054 INFO:tasks.ceph:[global] debug perfcounter = 0/0 2026-02-17T10:03:28.055 INFO:tasks.ceph:[global] debug rados = 0/0 2026-02-17T10:03:28.055 INFO:tasks.ceph:[global] debug rbd = 0/0 2026-02-17T10:03:28.055 INFO:tasks.ceph:[global] debug rbd mirror = 0/0 2026-02-17T10:03:28.055 INFO:tasks.ceph:[global] debug rbd replay = 0/0 2026-02-17T10:03:28.055 INFO:tasks.ceph:[global] debug refs = 0/0 2026-02-17T10:03:28.055 INFO:tasks.ceph:[global] debug rgw = 0/0 2026-02-17T10:03:28.055 INFO:tasks.ceph:[global] debug rgw lifecycle = 0/0 2026-02-17T10:03:28.055 INFO:tasks.ceph:[global] debug rgw notification = 0/0 2026-02-17T10:03:28.055 INFO:tasks.ceph:[global] debug rgw sync = 0/0 2026-02-17T10:03:28.055 INFO:tasks.ceph:[global] debug rocksdb = 0/0 2026-02-17T10:03:28.055 INFO:tasks.ceph:[global] debug striper = 0/0 2026-02-17T10:03:28.055 INFO:tasks.ceph:[global] debug throttle = 0/0 2026-02-17T10:03:28.055 INFO:tasks.ceph:[global] debug timer = 0/0 2026-02-17T10:03:28.056 INFO:tasks.ceph:[global] debug tp = 0/0 2026-02-17T10:03:28.056 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-17T10:03:28.056 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-17T10:03:28.056 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-17T10:03:28.056 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-17T10:03:28.056 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-17T10:03:28.056 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-17T10:03:28.056 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-17T10:03:28.056 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-17T10:03:28.056 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-17T10:03:28.056 INFO:tasks.ceph:[mon] debug mon = 0/0 2026-02-17T10:03:28.056 INFO:tasks.ceph:[mon] debug ms = 0/0 2026-02-17T10:03:28.056 INFO:tasks.ceph:[mon] debug paxos = 0/0 2026-02-17T10:03:28.057 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-17T10:03:28.057 INFO:tasks.ceph:Setting up mon.a... 2026-02-17T10:03:28.057 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-17T10:03:28.127 INFO:teuthology.orchestra.run.trial103.stdout:creating /etc/ceph/ceph.keyring 2026-02-17T10:03:28.130 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-17T10:03:28.215 DEBUG:teuthology.orchestra.run.trial103:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-17T10:03:28.241 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.103')] 2026-02-17T10:03:28.241 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'none', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.103', 'auth client required': 'none', 'auth cluster required': 'none', 'auth service required': 'none', 'debug asok': '0/0', 'debug auth': '0/0', 'debug bdev': '0/0', 'debug bluefs': '0/0', 'debug bluestore': '0/0', 'debug buffer': '0/0', 'debug civetweb': '0/0', 'debug client': '0/0', 'debug compressor': '0/0', 'debug context': '0/0', 'debug crush': '0/0', 'debug crypto': '0/0', 'debug dpdk': '0/0', 'debug eventtrace': '0/0', 'debug filer': '0/0', 'debug filestore': '0/0', 'debug finisher': '0/0', 'debug fuse': '0/0', 'debug heartbeatmap': '0/0', 'debug javaclient': '0/0', 'debug journal': '0/0', 'debug journaler': '0/0', 'debug kstore': '0/0', 'debug leveldb': '0/0', 'debug lockdep': '0/0', 'debug mds': '0/0', 'debug mds balancer': '0/0', 'debug mds locker': '0/0', 'debug mds log': '0/0', 'debug mds log expire': '0/0', 'debug mds migrator': '0/0', 'debug memdb': '0/0', 'debug mgr': '0/0', 'debug mgrc': '0/0', 'debug mon': '0/0', 'debug monc': '0/0', 'debug ms': '0/0', 'debug objclass': '0/0', 'debug objectcacher': '0/0', 'debug objecter': '0/0', 'debug optracker': '0/0', 'debug osd': '0/0', 'debug paxos': '0/0', 'debug perfcounter': '0/0', 'debug rados': '0/0', 'debug rbd': '0/0', 'debug rbd mirror': '0/0', 'debug rbd replay': '0/0', 'debug refs': '0/0', 'debug rgw': '0/0', 'debug rgw lifecycle': '0/0', 'debug rgw notification': '0/0', 'debug rgw sync': '0/0', 'debug rocksdb': '0/0', 'debug striper': '0/0', 'debug throttle': '0/0', 'debug timer': '0/0', 'debug tp': '0/0', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug filestore': '0/0', 'debug journal': '0/0', 'debug ms': '0/0', 'debug osd': '0/0', 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore main device type': 'RANDOM_BLOCK_SSD', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': '0/0', 'debug mon': '0/0', 'debug paxos': '0/0', 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-17T10:03:28.243 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:03:28.243 DEBUG:teuthology.orchestra.run.trial103:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-17T10:03:28.299 DEBUG:teuthology.orchestra.run.trial103:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.103 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-17T10:03:28.381 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:28.380+0000 7f2b7a092f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:28.382 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:28.380+0000 7f2b7a092f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:28.382 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:28.380+0000 7f2b7a092f40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:28.383 INFO:teuthology.orchestra.run.trial103.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-17T10:03:28.383 INFO:teuthology.orchestra.run.trial103.stdout:monmaptool: generated fsid 9bb44552-44bc-4583-a00f-cc1b54452651 2026-02-17T10:03:28.383 INFO:teuthology.orchestra.run.trial103.stdout:setting min_mon_release = tentacle 2026-02-17T10:03:28.383 INFO:teuthology.orchestra.run.trial103.stdout:epoch 0 2026-02-17T10:03:28.383 INFO:teuthology.orchestra.run.trial103.stdout:fsid 9bb44552-44bc-4583-a00f-cc1b54452651 2026-02-17T10:03:28.384 INFO:teuthology.orchestra.run.trial103.stdout:last_changed 2026-02-17T10:03:28.381206+0000 2026-02-17T10:03:28.384 INFO:teuthology.orchestra.run.trial103.stdout:created 2026-02-17T10:03:28.381206+0000 2026-02-17T10:03:28.384 INFO:teuthology.orchestra.run.trial103.stdout:min_mon_release 20 (tentacle) 2026-02-17T10:03:28.384 INFO:teuthology.orchestra.run.trial103.stdout:election_strategy: 1 2026-02-17T10:03:28.384 INFO:teuthology.orchestra.run.trial103.stdout:0: [v2:10.20.193.103:3300/0,v1:10.20.193.103:6789/0] mon.a 2026-02-17T10:03:28.384 INFO:teuthology.orchestra.run.trial103.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-17T10:03:28.385 DEBUG:teuthology.orchestra.run.trial103:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-17T10:03:28.439 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID 9bb44552-44bc-4583-a00f-cc1b54452651... 2026-02-17T10:03:28.441 DEBUG:teuthology.orchestra.run.trial103:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-17T10:03:28.520 INFO:teuthology.orchestra.run.trial103.stdout:[global] 2026-02-17T10:03:28.520 INFO:teuthology.orchestra.run.trial103.stdout: chdir = "" 2026-02-17T10:03:28.520 INFO:teuthology.orchestra.run.trial103.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-17T10:03:28.520 INFO:teuthology.orchestra.run.trial103.stdout: auth supported = none 2026-02-17T10:03:28.520 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.520 INFO:teuthology.orchestra.run.trial103.stdout: filestore xattr use omap = true 2026-02-17T10:03:28.520 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.521 INFO:teuthology.orchestra.run.trial103.stdout: mon clock drift allowed = 1.000 2026-02-17T10:03:28.521 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.521 INFO:teuthology.orchestra.run.trial103.stdout: osd crush chooseleaf type = 0 2026-02-17T10:03:28.521 INFO:teuthology.orchestra.run.trial103.stdout: auth debug = true 2026-02-17T10:03:28.521 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.521 INFO:teuthology.orchestra.run.trial103.stdout: ms die on old message = true 2026-02-17T10:03:28.521 INFO:teuthology.orchestra.run.trial103.stdout: ms die on bug = true 2026-02-17T10:03:28.521 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.521 INFO:teuthology.orchestra.run.trial103.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-17T10:03:28.521 INFO:teuthology.orchestra.run.trial103.stdout: mon pg warn max object skew = 0 2026-02-17T10:03:28.521 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.521 INFO:teuthology.orchestra.run.trial103.stdout: # disable pg_autoscaler by default for new pools 2026-02-17T10:03:28.522 INFO:teuthology.orchestra.run.trial103.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-17T10:03:28.522 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.522 INFO:teuthology.orchestra.run.trial103.stdout: osd pool default size = 2 2026-02-17T10:03:28.522 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.522 INFO:teuthology.orchestra.run.trial103.stdout: mon osd allow primary affinity = true 2026-02-17T10:03:28.522 INFO:teuthology.orchestra.run.trial103.stdout: mon osd allow pg remap = true 2026-02-17T10:03:28.522 INFO:teuthology.orchestra.run.trial103.stdout: mon warn on legacy crush tunables = false 2026-02-17T10:03:28.522 INFO:teuthology.orchestra.run.trial103.stdout: mon warn on crush straw calc version zero = false 2026-02-17T10:03:28.522 INFO:teuthology.orchestra.run.trial103.stdout: mon warn on no sortbitwise = false 2026-02-17T10:03:28.522 INFO:teuthology.orchestra.run.trial103.stdout: mon warn on osd down out interval zero = false 2026-02-17T10:03:28.522 INFO:teuthology.orchestra.run.trial103.stdout: mon warn on too few osds = false 2026-02-17T10:03:28.522 INFO:teuthology.orchestra.run.trial103.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-17T10:03:28.523 INFO:teuthology.orchestra.run.trial103.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-17T10:03:28.523 INFO:teuthology.orchestra.run.trial103.stdout: mon_allow_pool_size_one = true 2026-02-17T10:03:28.523 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.523 INFO:teuthology.orchestra.run.trial103.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-17T10:03:28.523 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.523 INFO:teuthology.orchestra.run.trial103.stdout: osd default data pool replay window = 5 2026-02-17T10:03:28.523 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.523 INFO:teuthology.orchestra.run.trial103.stdout: mon allow pool delete = true 2026-02-17T10:03:28.523 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.523 INFO:teuthology.orchestra.run.trial103.stdout: mon cluster log file level = debug 2026-02-17T10:03:28.523 INFO:teuthology.orchestra.run.trial103.stdout: debug asserts on shutdown = true 2026-02-17T10:03:28.523 INFO:teuthology.orchestra.run.trial103.stdout: mon health detail to clog = false 2026-02-17T10:03:28.523 INFO:teuthology.orchestra.run.trial103.stdout: mon host = 10.20.193.103 2026-02-17T10:03:28.524 INFO:teuthology.orchestra.run.trial103.stdout: auth client required = none 2026-02-17T10:03:28.524 INFO:teuthology.orchestra.run.trial103.stdout: auth cluster required = none 2026-02-17T10:03:28.524 INFO:teuthology.orchestra.run.trial103.stdout: auth service required = none 2026-02-17T10:03:28.524 INFO:teuthology.orchestra.run.trial103.stdout: debug asok = 0/0 2026-02-17T10:03:28.524 INFO:teuthology.orchestra.run.trial103.stdout: debug auth = 0/0 2026-02-17T10:03:28.524 INFO:teuthology.orchestra.run.trial103.stdout: debug bdev = 0/0 2026-02-17T10:03:28.524 INFO:teuthology.orchestra.run.trial103.stdout: debug bluefs = 0/0 2026-02-17T10:03:28.524 INFO:teuthology.orchestra.run.trial103.stdout: debug bluestore = 0/0 2026-02-17T10:03:28.524 INFO:teuthology.orchestra.run.trial103.stdout: debug buffer = 0/0 2026-02-17T10:03:28.524 INFO:teuthology.orchestra.run.trial103.stdout: debug civetweb = 0/0 2026-02-17T10:03:28.524 INFO:teuthology.orchestra.run.trial103.stdout: debug client = 0/0 2026-02-17T10:03:28.524 INFO:teuthology.orchestra.run.trial103.stdout: debug compressor = 0/0 2026-02-17T10:03:28.524 INFO:teuthology.orchestra.run.trial103.stdout: debug context = 0/0 2026-02-17T10:03:28.525 INFO:teuthology.orchestra.run.trial103.stdout: debug crush = 0/0 2026-02-17T10:03:28.525 INFO:teuthology.orchestra.run.trial103.stdout: debug crypto = 0/0 2026-02-17T10:03:28.525 INFO:teuthology.orchestra.run.trial103.stdout: debug dpdk = 0/0 2026-02-17T10:03:28.525 INFO:teuthology.orchestra.run.trial103.stdout: debug eventtrace = 0/0 2026-02-17T10:03:28.525 INFO:teuthology.orchestra.run.trial103.stdout: debug filer = 0/0 2026-02-17T10:03:28.525 INFO:teuthology.orchestra.run.trial103.stdout: debug filestore = 0/0 2026-02-17T10:03:28.525 INFO:teuthology.orchestra.run.trial103.stdout: debug finisher = 0/0 2026-02-17T10:03:28.525 INFO:teuthology.orchestra.run.trial103.stdout: debug fuse = 0/0 2026-02-17T10:03:28.525 INFO:teuthology.orchestra.run.trial103.stdout: debug heartbeatmap = 0/0 2026-02-17T10:03:28.525 INFO:teuthology.orchestra.run.trial103.stdout: debug javaclient = 0/0 2026-02-17T10:03:28.525 INFO:teuthology.orchestra.run.trial103.stdout: debug journal = 0/0 2026-02-17T10:03:28.525 INFO:teuthology.orchestra.run.trial103.stdout: debug journaler = 0/0 2026-02-17T10:03:28.525 INFO:teuthology.orchestra.run.trial103.stdout: debug kstore = 0/0 2026-02-17T10:03:28.526 INFO:teuthology.orchestra.run.trial103.stdout: debug leveldb = 0/0 2026-02-17T10:03:28.526 INFO:teuthology.orchestra.run.trial103.stdout: debug lockdep = 0/0 2026-02-17T10:03:28.526 INFO:teuthology.orchestra.run.trial103.stdout: debug mds = 0/0 2026-02-17T10:03:28.526 INFO:teuthology.orchestra.run.trial103.stdout: debug mds balancer = 0/0 2026-02-17T10:03:28.526 INFO:teuthology.orchestra.run.trial103.stdout: debug mds locker = 0/0 2026-02-17T10:03:28.526 INFO:teuthology.orchestra.run.trial103.stdout: debug mds log = 0/0 2026-02-17T10:03:28.526 INFO:teuthology.orchestra.run.trial103.stdout: debug mds log expire = 0/0 2026-02-17T10:03:28.526 INFO:teuthology.orchestra.run.trial103.stdout: debug mds migrator = 0/0 2026-02-17T10:03:28.526 INFO:teuthology.orchestra.run.trial103.stdout: debug memdb = 0/0 2026-02-17T10:03:28.526 INFO:teuthology.orchestra.run.trial103.stdout: debug mgr = 0/0 2026-02-17T10:03:28.526 INFO:teuthology.orchestra.run.trial103.stdout: debug mgrc = 0/0 2026-02-17T10:03:28.526 INFO:teuthology.orchestra.run.trial103.stdout: debug mon = 0/0 2026-02-17T10:03:28.526 INFO:teuthology.orchestra.run.trial103.stdout: debug monc = 0/0 2026-02-17T10:03:28.527 INFO:teuthology.orchestra.run.trial103.stdout: debug ms = 0/0 2026-02-17T10:03:28.527 INFO:teuthology.orchestra.run.trial103.stdout: debug objclass = 0/0 2026-02-17T10:03:28.527 INFO:teuthology.orchestra.run.trial103.stdout: debug objectcacher = 0/0 2026-02-17T10:03:28.527 INFO:teuthology.orchestra.run.trial103.stdout: debug objecter = 0/0 2026-02-17T10:03:28.527 INFO:teuthology.orchestra.run.trial103.stdout: debug optracker = 0/0 2026-02-17T10:03:28.527 INFO:teuthology.orchestra.run.trial103.stdout: debug osd = 0/0 2026-02-17T10:03:28.527 INFO:teuthology.orchestra.run.trial103.stdout: debug paxos = 0/0 2026-02-17T10:03:28.527 INFO:teuthology.orchestra.run.trial103.stdout: debug perfcounter = 0/0 2026-02-17T10:03:28.527 INFO:teuthology.orchestra.run.trial103.stdout: debug rados = 0/0 2026-02-17T10:03:28.527 INFO:teuthology.orchestra.run.trial103.stdout: debug rbd = 0/0 2026-02-17T10:03:28.527 INFO:teuthology.orchestra.run.trial103.stdout: debug rbd mirror = 0/0 2026-02-17T10:03:28.527 INFO:teuthology.orchestra.run.trial103.stdout: debug rbd replay = 0/0 2026-02-17T10:03:28.527 INFO:teuthology.orchestra.run.trial103.stdout: debug refs = 0/0 2026-02-17T10:03:28.527 INFO:teuthology.orchestra.run.trial103.stdout: debug rgw = 0/0 2026-02-17T10:03:28.528 INFO:teuthology.orchestra.run.trial103.stdout: debug rgw lifecycle = 0/0 2026-02-17T10:03:28.528 INFO:teuthology.orchestra.run.trial103.stdout: debug rgw notification = 0/0 2026-02-17T10:03:28.528 INFO:teuthology.orchestra.run.trial103.stdout: debug rgw sync = 0/0 2026-02-17T10:03:28.528 INFO:teuthology.orchestra.run.trial103.stdout: debug rocksdb = 0/0 2026-02-17T10:03:28.528 INFO:teuthology.orchestra.run.trial103.stdout: debug striper = 0/0 2026-02-17T10:03:28.528 INFO:teuthology.orchestra.run.trial103.stdout: debug throttle = 0/0 2026-02-17T10:03:28.528 INFO:teuthology.orchestra.run.trial103.stdout: debug timer = 0/0 2026-02-17T10:03:28.528 INFO:teuthology.orchestra.run.trial103.stdout: debug tp = 0/0 2026-02-17T10:03:28.528 INFO:teuthology.orchestra.run.trial103.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-17T10:03:28.528 INFO:teuthology.orchestra.run.trial103.stdout: ms client mode = crc 2026-02-17T10:03:28.528 INFO:teuthology.orchestra.run.trial103.stdout: ms cluster mode = crc 2026-02-17T10:03:28.528 INFO:teuthology.orchestra.run.trial103.stdout: ms mon client mode = crc 2026-02-17T10:03:28.528 INFO:teuthology.orchestra.run.trial103.stdout: ms mon cluster mode = crc 2026-02-17T10:03:28.529 INFO:teuthology.orchestra.run.trial103.stdout: ms mon service mode = crc 2026-02-17T10:03:28.529 INFO:teuthology.orchestra.run.trial103.stdout: ms service mode = crc 2026-02-17T10:03:28.529 INFO:teuthology.orchestra.run.trial103.stdout: fsid = 9bb44552-44bc-4583-a00f-cc1b54452651 2026-02-17T10:03:28.529 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.529 INFO:teuthology.orchestra.run.trial103.stdout:[osd] 2026-02-17T10:03:28.529 INFO:teuthology.orchestra.run.trial103.stdout: osd journal size = 100 2026-02-17T10:03:28.529 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.529 INFO:teuthology.orchestra.run.trial103.stdout: osd scrub load threshold = 5.0 2026-02-17T10:03:28.529 INFO:teuthology.orchestra.run.trial103.stdout: osd scrub max interval = 600 2026-02-17T10:03:28.529 INFO:teuthology.orchestra.run.trial103.stdout: osd mclock profile = high_recovery_ops 2026-02-17T10:03:28.529 INFO:teuthology.orchestra.run.trial103.stdout: osd mclock skip benchmark = true 2026-02-17T10:03:28.529 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.529 INFO:teuthology.orchestra.run.trial103.stdout: osd recover clone overlap = true 2026-02-17T10:03:28.530 INFO:teuthology.orchestra.run.trial103.stdout: osd recovery max chunk = 1048576 2026-02-17T10:03:28.530 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.530 INFO:teuthology.orchestra.run.trial103.stdout: osd debug shutdown = true 2026-02-17T10:03:28.530 INFO:teuthology.orchestra.run.trial103.stdout: osd debug op order = true 2026-02-17T10:03:28.530 INFO:teuthology.orchestra.run.trial103.stdout: osd debug verify stray on activate = true 2026-02-17T10:03:28.530 INFO:teuthology.orchestra.run.trial103.stdout: osd debug trim objects = true 2026-02-17T10:03:28.530 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.530 INFO:teuthology.orchestra.run.trial103.stdout: osd open classes on start = true 2026-02-17T10:03:28.530 INFO:teuthology.orchestra.run.trial103.stdout: osd debug pg log writeout = true 2026-02-17T10:03:28.530 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.530 INFO:teuthology.orchestra.run.trial103.stdout: osd deep scrub update digest min age = 30 2026-02-17T10:03:28.530 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.530 INFO:teuthology.orchestra.run.trial103.stdout: osd map max advance = 10 2026-02-17T10:03:28.531 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.531 INFO:teuthology.orchestra.run.trial103.stdout: journal zero on create = true 2026-02-17T10:03:28.531 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.531 INFO:teuthology.orchestra.run.trial103.stdout: filestore ondisk finisher threads = 3 2026-02-17T10:03:28.531 INFO:teuthology.orchestra.run.trial103.stdout: filestore apply finisher threads = 3 2026-02-17T10:03:28.531 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.531 INFO:teuthology.orchestra.run.trial103.stdout: bdev debug aio = true 2026-02-17T10:03:28.531 INFO:teuthology.orchestra.run.trial103.stdout: osd debug misdirected ops = true 2026-02-17T10:03:28.531 INFO:teuthology.orchestra.run.trial103.stdout: debug monc = 20 2026-02-17T10:03:28.531 INFO:teuthology.orchestra.run.trial103.stdout: crimson cpu num = 4 2026-02-17T10:03:28.531 INFO:teuthology.orchestra.run.trial103.stdout: crimson osd obc lru size = 10 2026-02-17T10:03:28.531 INFO:teuthology.orchestra.run.trial103.stdout: debug filestore = 0/0 2026-02-17T10:03:28.531 INFO:teuthology.orchestra.run.trial103.stdout: debug journal = 0/0 2026-02-17T10:03:28.531 INFO:teuthology.orchestra.run.trial103.stdout: debug ms = 0/0 2026-02-17T10:03:28.532 INFO:teuthology.orchestra.run.trial103.stdout: debug osd = 0/0 2026-02-17T10:03:28.532 INFO:teuthology.orchestra.run.trial103.stdout: debug seastore = 20 2026-02-17T10:03:28.532 INFO:teuthology.orchestra.run.trial103.stdout: debug seastore backref = 20 2026-02-17T10:03:28.532 INFO:teuthology.orchestra.run.trial103.stdout: debug seastore cache = 20 2026-02-17T10:03:28.532 INFO:teuthology.orchestra.run.trial103.stdout: debug seastore cleaner = 20 2026-02-17T10:03:28.532 INFO:teuthology.orchestra.run.trial103.stdout: debug seastore device = 20 2026-02-17T10:03:28.532 INFO:teuthology.orchestra.run.trial103.stdout: debug seastore epm = 20 2026-02-17T10:03:28.532 INFO:teuthology.orchestra.run.trial103.stdout: debug seastore fixedkv tree = 20 2026-02-17T10:03:28.532 INFO:teuthology.orchestra.run.trial103.stdout: debug seastore journal = 20 2026-02-17T10:03:28.532 INFO:teuthology.orchestra.run.trial103.stdout: debug seastore lba = 20 2026-02-17T10:03:28.532 INFO:teuthology.orchestra.run.trial103.stdout: debug seastore odata = 20 2026-02-17T10:03:28.532 INFO:teuthology.orchestra.run.trial103.stdout: debug seastore omap = 20 2026-02-17T10:03:28.532 INFO:teuthology.orchestra.run.trial103.stdout: debug seastore onode = 20 2026-02-17T10:03:28.533 INFO:teuthology.orchestra.run.trial103.stdout: debug seastore t = 20 2026-02-17T10:03:28.533 INFO:teuthology.orchestra.run.trial103.stdout: debug seastore tm = 20 2026-02-17T10:03:28.533 INFO:teuthology.orchestra.run.trial103.stdout: osd objectstore = seastore 2026-02-17T10:03:28.533 INFO:teuthology.orchestra.run.trial103.stdout: osd shutdown pgref assert = True 2026-02-17T10:03:28.533 INFO:teuthology.orchestra.run.trial103.stdout: seastore cachepin size pershard = 64M 2026-02-17T10:03:28.533 INFO:teuthology.orchestra.run.trial103.stdout: seastore cachepin type = 2Q 2026-02-17T10:03:28.533 INFO:teuthology.orchestra.run.trial103.stdout: seastore main device type = RANDOM_BLOCK_SSD 2026-02-17T10:03:28.533 INFO:teuthology.orchestra.run.trial103.stdout: seastore max concurrent transactions = 8 2026-02-17T10:03:28.533 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.533 INFO:teuthology.orchestra.run.trial103.stdout:[mgr] 2026-02-17T10:03:28.533 INFO:teuthology.orchestra.run.trial103.stdout: debug ms = 1 2026-02-17T10:03:28.533 INFO:teuthology.orchestra.run.trial103.stdout: debug mgr = 20 2026-02-17T10:03:28.533 INFO:teuthology.orchestra.run.trial103.stdout: debug mon = 20 2026-02-17T10:03:28.533 INFO:teuthology.orchestra.run.trial103.stdout: debug auth = 20 2026-02-17T10:03:28.534 INFO:teuthology.orchestra.run.trial103.stdout: mon reweight min pgs per osd = 4 2026-02-17T10:03:28.534 INFO:teuthology.orchestra.run.trial103.stdout: mon reweight min bytes per osd = 10 2026-02-17T10:03:28.534 INFO:teuthology.orchestra.run.trial103.stdout: mgr/telemetry/nag = false 2026-02-17T10:03:28.534 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.534 INFO:teuthology.orchestra.run.trial103.stdout:[mon] 2026-02-17T10:03:28.534 INFO:teuthology.orchestra.run.trial103.stdout: debug ms = 0/0 2026-02-17T10:03:28.534 INFO:teuthology.orchestra.run.trial103.stdout: debug mon = 0/0 2026-02-17T10:03:28.534 INFO:teuthology.orchestra.run.trial103.stdout: debug paxos = 0/0 2026-02-17T10:03:28.534 INFO:teuthology.orchestra.run.trial103.stdout: debug auth = 20 2026-02-17T10:03:28.534 INFO:teuthology.orchestra.run.trial103.stdout: mon data avail warn = 5 2026-02-17T10:03:28.534 INFO:teuthology.orchestra.run.trial103.stdout: mon mgr mkfs grace = 240 2026-02-17T10:03:28.534 INFO:teuthology.orchestra.run.trial103.stdout: mon reweight min pgs per osd = 4 2026-02-17T10:03:28.534 INFO:teuthology.orchestra.run.trial103.stdout: mon osd reporter subtree level = osd 2026-02-17T10:03:28.535 INFO:teuthology.orchestra.run.trial103.stdout: mon osd prime pg temp = true 2026-02-17T10:03:28.535 INFO:teuthology.orchestra.run.trial103.stdout: mon reweight min bytes per osd = 10 2026-02-17T10:03:28.535 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.535 INFO:teuthology.orchestra.run.trial103.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-17T10:03:28.535 INFO:teuthology.orchestra.run.trial103.stdout: auth mon ticket ttl = 660 # 11m 2026-02-17T10:03:28.535 INFO:teuthology.orchestra.run.trial103.stdout: auth service ticket ttl = 240 # 4m 2026-02-17T10:03:28.535 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.535 INFO:teuthology.orchestra.run.trial103.stdout: # don't complain about insecure global_id in the test suite 2026-02-17T10:03:28.535 INFO:teuthology.orchestra.run.trial103.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-17T10:03:28.535 INFO:teuthology.orchestra.run.trial103.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-17T10:03:28.535 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.535 INFO:teuthology.orchestra.run.trial103.stdout: # 1m isn't quite enough 2026-02-17T10:03:28.535 INFO:teuthology.orchestra.run.trial103.stdout: mon_down_mkfs_grace = 2m 2026-02-17T10:03:28.536 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.536 INFO:teuthology.orchestra.run.trial103.stdout: mon_warn_on_filestore_osds = false 2026-02-17T10:03:28.536 INFO:teuthology.orchestra.run.trial103.stdout: osd pool default crimson = True 2026-02-17T10:03:28.536 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:28.536 INFO:teuthology.orchestra.run.trial103.stdout:[client] 2026-02-17T10:03:28.536 INFO:teuthology.orchestra.run.trial103.stdout: rgw cache enabled = true 2026-02-17T10:03:28.536 INFO:teuthology.orchestra.run.trial103.stdout: rgw enable ops log = true 2026-02-17T10:03:28.536 INFO:teuthology.orchestra.run.trial103.stdout: rgw enable usage log = true 2026-02-17T10:03:28.536 INFO:teuthology.orchestra.run.trial103.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-17T10:03:28.536 INFO:teuthology.orchestra.run.trial103.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-17T10:03:28.536 INFO:teuthology.orchestra.run.trial103.stdout:[mon.a] 2026-02-17T10:03:28.537 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-17T10:03:28.538 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-17T10:03:28.630 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-17T10:03:28.630 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:03:28.630 DEBUG:teuthology.orchestra.run.trial103:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-17T10:03:28.646 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:03:28.647 DEBUG:teuthology.orchestra.run.trial103:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-17T10:03:28.703 INFO:tasks.ceph:Sending monmap to node ubuntu@trial103.front.sepia.ceph.com 2026-02-17T10:03:28.704 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:03:28.704 DEBUG:teuthology.orchestra.run.trial103:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-17T10:03:28.704 DEBUG:teuthology.orchestra.run.trial103:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-17T10:03:28.777 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:03:28.777 DEBUG:teuthology.orchestra.run.trial103:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-17T10:03:28.836 INFO:tasks.ceph:Setting up mon nodes... 2026-02-17T10:03:28.836 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-17T10:03:28.836 DEBUG:teuthology.orchestra.run.trial103:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-17T10:03:28.926 INFO:teuthology.orchestra.run.trial103.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-17T10:03:28.929 INFO:tasks.ceph:Setting up mds nodes... 2026-02-17T10:03:28.929 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-17T10:03:28.929 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-17T10:03:28.972 INFO:teuthology.orchestra.run.trial103.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-17T10:03:28.983 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-17T10:03:28.983 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial103.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'}} 2026-02-17T10:03:28.984 DEBUG:teuthology.orchestra.run.trial103:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-17T10:03:29.048 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-17T10:03:29.048 INFO:tasks.ceph:role: osd.0 2026-02-17T10:03:29.049 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial103.front.sepia.ceph.com 2026-02-17T10:03:29.049 DEBUG:teuthology.orchestra.run.trial103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-17T10:03:29.114 INFO:teuthology.orchestra.run.trial103.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T10:03:29.114 INFO:teuthology.orchestra.run.trial103.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T10:03:29.114 INFO:teuthology.orchestra.run.trial103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T10:03:29.114 INFO:teuthology.orchestra.run.trial103.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T10:03:29.114 INFO:teuthology.orchestra.run.trial103.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T10:03:29.115 INFO:teuthology.orchestra.run.trial103.stdout: = sunit=0 swidth=0 blks 2026-02-17T10:03:29.115 INFO:teuthology.orchestra.run.trial103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T10:03:29.115 INFO:teuthology.orchestra.run.trial103.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T10:03:29.115 INFO:teuthology.orchestra.run.trial103.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T10:03:29.115 INFO:teuthology.orchestra.run.trial103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T10:03:29.691 INFO:teuthology.orchestra.run.trial103.stdout:Discarding blocks...Done. 2026-02-17T10:03:29.693 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial103.front.sepia.ceph.com -o noatime 2026-02-17T10:03:29.693 DEBUG:teuthology.orchestra.run.trial103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-17T10:03:29.764 DEBUG:teuthology.orchestra.run.trial103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-17T10:03:29.831 DEBUG:teuthology.orchestra.run.trial103:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-17T10:03:29.895 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-17T10:03:29.895 INFO:tasks.ceph:role: osd.1 2026-02-17T10:03:29.895 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial103.front.sepia.ceph.com 2026-02-17T10:03:29.895 DEBUG:teuthology.orchestra.run.trial103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-17T10:03:29.962 INFO:teuthology.orchestra.run.trial103.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T10:03:29.962 INFO:teuthology.orchestra.run.trial103.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T10:03:29.963 INFO:teuthology.orchestra.run.trial103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T10:03:29.963 INFO:teuthology.orchestra.run.trial103.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T10:03:29.963 INFO:teuthology.orchestra.run.trial103.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T10:03:29.963 INFO:teuthology.orchestra.run.trial103.stdout: = sunit=0 swidth=0 blks 2026-02-17T10:03:29.963 INFO:teuthology.orchestra.run.trial103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T10:03:29.963 INFO:teuthology.orchestra.run.trial103.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T10:03:29.963 INFO:teuthology.orchestra.run.trial103.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T10:03:29.963 INFO:teuthology.orchestra.run.trial103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T10:03:30.521 INFO:teuthology.orchestra.run.trial103.stdout:Discarding blocks...Done. 2026-02-17T10:03:30.523 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial103.front.sepia.ceph.com -o noatime 2026-02-17T10:03:30.523 DEBUG:teuthology.orchestra.run.trial103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-17T10:03:30.594 DEBUG:teuthology.orchestra.run.trial103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-17T10:03:30.663 DEBUG:teuthology.orchestra.run.trial103:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-17T10:03:30.729 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2', 'osd.2': '/dev/vg_nvme/lv_3'} 2026-02-17T10:03:30.729 INFO:tasks.ceph:role: osd.2 2026-02-17T10:03:30.729 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_3 on ubuntu@trial103.front.sepia.ceph.com 2026-02-17T10:03:30.729 DEBUG:teuthology.orchestra.run.trial103:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_3 2026-02-17T10:03:30.795 INFO:teuthology.orchestra.run.trial103.stdout:meta-data=/dev/vg_nvme/lv_3 isize=2048 agcount=4, agsize=45875200 blks 2026-02-17T10:03:30.795 INFO:teuthology.orchestra.run.trial103.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-17T10:03:30.796 INFO:teuthology.orchestra.run.trial103.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-17T10:03:30.796 INFO:teuthology.orchestra.run.trial103.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-17T10:03:30.796 INFO:teuthology.orchestra.run.trial103.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-17T10:03:30.796 INFO:teuthology.orchestra.run.trial103.stdout: = sunit=0 swidth=0 blks 2026-02-17T10:03:30.796 INFO:teuthology.orchestra.run.trial103.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-17T10:03:30.796 INFO:teuthology.orchestra.run.trial103.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-17T10:03:30.796 INFO:teuthology.orchestra.run.trial103.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-17T10:03:30.796 INFO:teuthology.orchestra.run.trial103.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-17T10:03:31.360 INFO:teuthology.orchestra.run.trial103.stdout:Discarding blocks...Done. 2026-02-17T10:03:31.362 INFO:tasks.ceph:mount /dev/vg_nvme/lv_3 on ubuntu@trial103.front.sepia.ceph.com -o noatime 2026-02-17T10:03:31.362 DEBUG:teuthology.orchestra.run.trial103:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_3 /var/lib/ceph/osd/ceph-2 2026-02-17T10:03:31.429 DEBUG:teuthology.orchestra.run.trial103:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-17T10:03:31.498 DEBUG:teuthology.orchestra.run.trial103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T10:03:31.651 INFO:teuthology.orchestra.run.trial103.stderr:INFO  osd - OSD::main: parsing early config 2026-02-17T10:03:31.702 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:31,701 seastar - Reactor backend: linux-aio 2026-02-17T10:03:31.720 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:31,719 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-17T10:03:31.720 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:31,719 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-17T10:03:31.722 INFO:teuthology.orchestra.run.trial103.stderr:==39299==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-17T10:03:31.782 INFO:teuthology.orchestra.run.trial103.stderr:DEBUG 2026-02-17 10:03:31,781 [shard 0:main] seastar - reactor::drain 2026-02-17T10:03:33.821 INFO:teuthology.orchestra.run.trial103.stderr:----------------------------------------------------- 2026-02-17T10:03:33.821 INFO:teuthology.orchestra.run.trial103.stderr:Suppressions used: 2026-02-17T10:03:33.821 INFO:teuthology.orchestra.run.trial103.stderr: count bytes template 2026-02-17T10:03:33.821 INFO:teuthology.orchestra.run.trial103.stderr: 1 8 MallocExtension::Register 2026-02-17T10:03:33.821 INFO:teuthology.orchestra.run.trial103.stderr:----------------------------------------------------- 2026-02-17T10:03:33.821 INFO:teuthology.orchestra.run.trial103.stderr: 2026-02-17T10:03:33.840 INFO:teuthology.orchestra.run.trial103.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-17T10:03:33.840 INFO:teuthology.orchestra.run.trial103.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-17T10:03:33.885 INFO:teuthology.orchestra.run.trial103.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-17T10:03:33.885 INFO:teuthology.orchestra.run.trial103.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-17T10:03:33.892 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:33,891 seastar - Reactor backend: linux-aio 2026-02-17T10:03:33.903 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:33,902 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-17T10:03:33.903 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:33,902 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-17T10:03:33.907 INFO:teuthology.orchestra.run.trial103.stderr:==39297==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-17T10:03:33.907 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:33,907 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-17T10:03:34.046 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x157f2 0x1022b6b 0x10dd80a 0x994019a 0x9940ad9 0x9a580ad 0x9a62793 0x786fa4b 0x7b608a9 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f6fdb6 0x6f73bb7 0x6f79761 0x73e3839 0x7412d74 0x7256dc1 0x7256eb9 0x732d51e 0x732d5d7 0x732d78e 0x7412df5 0x7256f36 0x7256fbb 0x72819ab 0x73ec44d 0x73ec4d2 0x73ec5b0 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:34.047 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f 0x3135bf0 0x3135dc6 0x313607d 0x3136172 0x6f3fb56 0x9a45f9a 0x9a461b2 0x9a46484 0x7b60fb5 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f6fdb6 0x6f73bb7 0x6f79761 0x73e3839 0x7412d74 0x7256dc1 0x7256eb9 0x732d51e 0x732d5d7 0x732d78e 0x7412df5 0x7256f36 0x7256fbb 0x72819ab 0x73ec44d 0x73ec4d2 0x73ec5b0 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:34.047 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x104e62 /lib64/libasan.so.8+0xea9a /lib64/libasan.so.8+0xec776 0xc80a85 0x6f3fbe2 0x9a45f9a 0x9a461b2 0x9a46484 0x7b60fb5 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f6fdb6 0x6f73bb7 0x6f79761 0x73e3839 0x7412d74 0x7256dc1 0x7256eb9 0x732d51e 0x732d5d7 0x732d78e 0x7412df5 0x7256f36 0x7256fbb 0x72819ab 0x73ec44d 0x73ec4d2 0x73ec5b0 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:34.194 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f 0xc809e1 0x740aaa2 0x72c8fb8 0x712c4a8 0x7b61055 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f0b812 0x6f0db7a 0x73e34d8 0x740d434 0x7255508 0x7255600 0x732cdbe 0x732ce77 0x732d02e 0x740d4b5 0x725567d 0x7255702 0x727ca0b 0x73eae9c 0x73eaf21 0x73eafff 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:34.194 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f 0x993d510 0x993dbac 0x9942cd1 0x99433b1 0x9a5c1c4 0x9a6086f 0x99c4c35 0x99c3c35 0x7b60f77 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f0b812 0x6f0db7a 0x73e34d8 0x740d434 0x7255508 0x7255600 0x732cdbe 0x732ce77 0x732d02e 0x740d4b5 0x725567d 0x7255702 0x727ca0b 0x73eae9c 0x73eaf21 0x73eafff 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:34.195 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f 0xd91640 0x98c11d0 0x98c1314 0x98c13f9 0x98c1655 0x98c1748 0x98c1824 0x98c1ee0 0x98c2047 0x98c244b 0x98c2708 0x9a5ea3d 0x99c4c35 0x99c3c35 0x7b60f77 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f0b812 0x6f0db7a 0x73e34d8 0x740d434 0x7255508 0x7255600 0x732cdbe 0x732ce77 0x732d02e 0x740d4b5 0x725567d 0x7255702 0x727ca0b 0x73eae9c 0x73eaf21 0x73eafff 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:36.358 INFO:teuthology.orchestra.run.trial103.stderr:----------------------------------------------------- 2026-02-17T10:03:36.358 INFO:teuthology.orchestra.run.trial103.stderr:Suppressions used: 2026-02-17T10:03:36.358 INFO:teuthology.orchestra.run.trial103.stderr: count bytes template 2026-02-17T10:03:36.358 INFO:teuthology.orchestra.run.trial103.stderr: 1 8 MallocExtension::Register 2026-02-17T10:03:36.358 INFO:teuthology.orchestra.run.trial103.stderr:----------------------------------------------------- 2026-02-17T10:03:36.359 INFO:teuthology.orchestra.run.trial103.stderr: 2026-02-17T10:03:36.390 DEBUG:teuthology.orchestra.run.trial103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-17T10:03:36.415 DEBUG:teuthology.orchestra.run.trial103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T10:03:36.569 INFO:teuthology.orchestra.run.trial103.stderr:INFO  osd - OSD::main: parsing early config 2026-02-17T10:03:36.621 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:36,621 seastar - Reactor backend: linux-aio 2026-02-17T10:03:36.636 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:36,635 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-17T10:03:36.636 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:36,636 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-17T10:03:36.638 INFO:teuthology.orchestra.run.trial103.stderr:==39365==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-17T10:03:36.698 INFO:teuthology.orchestra.run.trial103.stderr:DEBUG 2026-02-17 10:03:36,698 [shard 0:main] seastar - reactor::drain 2026-02-17T10:03:38.735 INFO:teuthology.orchestra.run.trial103.stderr:----------------------------------------------------- 2026-02-17T10:03:38.735 INFO:teuthology.orchestra.run.trial103.stderr:Suppressions used: 2026-02-17T10:03:38.735 INFO:teuthology.orchestra.run.trial103.stderr: count bytes template 2026-02-17T10:03:38.735 INFO:teuthology.orchestra.run.trial103.stderr: 1 8 MallocExtension::Register 2026-02-17T10:03:38.735 INFO:teuthology.orchestra.run.trial103.stderr:----------------------------------------------------- 2026-02-17T10:03:38.736 INFO:teuthology.orchestra.run.trial103.stderr: 2026-02-17T10:03:38.755 INFO:teuthology.orchestra.run.trial103.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-17T10:03:38.755 INFO:teuthology.orchestra.run.trial103.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-17T10:03:38.796 INFO:teuthology.orchestra.run.trial103.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-17T10:03:38.796 INFO:teuthology.orchestra.run.trial103.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-17T10:03:38.803 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:38,802 seastar - Reactor backend: linux-aio 2026-02-17T10:03:38.818 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:38,818 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-17T10:03:38.819 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:38,818 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-17T10:03:38.823 INFO:teuthology.orchestra.run.trial103.stderr:==39363==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-17T10:03:38.823 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:38,822 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-17T10:03:38.965 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f 0xc80bcf 0x99fb735 0x9a4439a 0x9a5bb87 0x9a6086f 0x99c4c35 0x99c3c35 0x7b60f77 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f6fdb6 0x6f73bb7 0x6f79761 0x73e3839 0x7412d74 0x7256dc1 0x7256eb9 0x732d51e 0x732d5d7 0x732d78e 0x7412df5 0x7256f36 0x7256fbb 0x72819ab 0x73ec44d 0x73ec4d2 0x73ec5b0 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:38.965 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb1437 0xb3a067 0x1022af4 0x10dd80a 0x993cdc4 0x993d0c1 0x993d334 0x993d5ea 0x993e099 0x993e597 0x993ebc7 0x993ed31 0x9a4bfe1 0x9a5ede8 0x99c4c35 0x99c3c35 0x7b60f77 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f6fdb6 0x6f73bb7 0x6f79761 0x73e3839 0x7412d74 0x7256dc1 0x7256eb9 0x732d51e 0x732d5d7 0x732d78e 0x7412df5 0x7256f36 0x7256fbb 0x72819ab 0x73ec44d 0x73ec4d2 0x73ec5b0 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:38.967 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 34 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xeacf /lib64/libasan.so.8+0xec776 0xc80a85 0x998d777 0x9a60b40 0x99c4c35 0x99c3c35 0x7b60f77 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f6fdb6 0x6f73bb7 0x6f79761 0x73e3839 0x7412d74 0x7256dc1 0x7256eb9 0x732d51e 0x732d5d7 0x732d78e 0x7412df5 0x7256f36 0x7256fbb 0x72819ab 0x73ec44d 0x73ec4d2 0x73ec5b0 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:39.112 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x16bd7 0x9a52ac0 0x9a57c88 0x9a582c6 0x9a62793 0x786fa4b 0x7b61ccd 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f0b812 0x6f0db7a 0x73e34d8 0x740d434 0x7255508 0x7255600 0x732cdbe 0x732ce77 0x732d02e 0x740d4b5 0x725567d 0x7255702 0x727ca0b 0x73eae9c 0x73eaf21 0x73eafff 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:39.112 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1294d2 /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x993cc60 0x993d0c1 0x993d334 0x993d5ea 0x993dbac 0x9942cd1 0x99433b1 0x9a5c1c4 0x9a6086f 0x99c4c35 0x99c3c35 0x7b60f77 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f0b812 0x6f0db7a 0x73e34d8 0x740d434 0x7255508 0x7255600 0x732cdbe 0x732ce77 0x732d02e 0x740d4b5 0x725567d 0x7255702 0x727ca0b 0x73eae9c 0x73eaf21 0x73eafff 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:39.112 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xfcaf3 /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x993cc60 0x993d0c1 0x993d334 0x993e334 0x9943035 0x99433b1 0x9a5c1c4 0x9a6086f 0x99c4c35 0x99c3c35 0x7b60f77 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f0b812 0x6f0db7a 0x73e34d8 0x740d434 0x7255508 0x7255600 0x732cdbe 0x732ce77 0x732d02e 0x740d4b5 0x725567d 0x7255702 0x727ca0b 0x73eae9c 0x73eaf21 0x73eafff 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:41.239 INFO:teuthology.orchestra.run.trial103.stderr:----------------------------------------------------- 2026-02-17T10:03:41.239 INFO:teuthology.orchestra.run.trial103.stderr:Suppressions used: 2026-02-17T10:03:41.239 INFO:teuthology.orchestra.run.trial103.stderr: count bytes template 2026-02-17T10:03:41.239 INFO:teuthology.orchestra.run.trial103.stderr: 1 8 MallocExtension::Register 2026-02-17T10:03:41.239 INFO:teuthology.orchestra.run.trial103.stderr:----------------------------------------------------- 2026-02-17T10:03:41.239 INFO:teuthology.orchestra.run.trial103.stderr: 2026-02-17T10:03:41.269 DEBUG:teuthology.orchestra.run.trial103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-17T10:03:41.293 DEBUG:teuthology.orchestra.run.trial103:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-17T10:03:41.444 INFO:teuthology.orchestra.run.trial103.stderr:INFO  osd - OSD::main: parsing early config 2026-02-17T10:03:41.497 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:41,497 seastar - Reactor backend: linux-aio 2026-02-17T10:03:41.512 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:41,512 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-17T10:03:41.512 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:41,512 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-17T10:03:41.514 INFO:teuthology.orchestra.run.trial103.stderr:==39429==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-17T10:03:41.574 INFO:teuthology.orchestra.run.trial103.stderr:DEBUG 2026-02-17 10:03:41,573 [shard 0:main] seastar - reactor::drain 2026-02-17T10:03:43.623 INFO:teuthology.orchestra.run.trial103.stderr:----------------------------------------------------- 2026-02-17T10:03:43.623 INFO:teuthology.orchestra.run.trial103.stderr:Suppressions used: 2026-02-17T10:03:43.623 INFO:teuthology.orchestra.run.trial103.stderr: count bytes template 2026-02-17T10:03:43.623 INFO:teuthology.orchestra.run.trial103.stderr: 1 8 MallocExtension::Register 2026-02-17T10:03:43.623 INFO:teuthology.orchestra.run.trial103.stderr:----------------------------------------------------- 2026-02-17T10:03:43.623 INFO:teuthology.orchestra.run.trial103.stderr: 2026-02-17T10:03:43.643 INFO:teuthology.orchestra.run.trial103.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-17T10:03:43.643 INFO:teuthology.orchestra.run.trial103.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-17T10:03:43.684 INFO:teuthology.orchestra.run.trial103.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-17T10:03:43.684 INFO:teuthology.orchestra.run.trial103.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-17T10:03:43.691 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:43,690 seastar - Reactor backend: linux-aio 2026-02-17T10:03:43.702 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:43,701 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-17T10:03:43.702 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:43,701 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-17T10:03:43.706 INFO:teuthology.orchestra.run.trial103.stderr:==39427==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-17T10:03:43.706 INFO:teuthology.orchestra.run.trial103.stderr:INFO 2026-02-17 10:03:43,705 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-17T10:03:43.845 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f 0xd7b249 0x10dd70d 0x10dd7ff 0x993cbf7 0x993d0c1 0x993d334 0x993f05b 0x9a51ef5 0x9a52164 0x9a5c3f6 0x9a6086f 0x99c4c35 0x99c3c35 0x7b60f77 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f6fdb6 0x6f73bb7 0x6f79761 0x73e3839 0x7412d74 0x7256dc1 0x7256eb9 0x732d51e 0x732d5d7 0x732d78e 0x7412df5 0x7256f36 0x7256fbb 0x72819ab 0x73ec44d 0x73ec4d2 0x73ec5b0 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:43.845 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151c1 0x993cc60 0x993d0c1 0x993d334 0x993e334 0x9943035 0x99433b1 0x9a5c1c4 0x9a6086f 0x99c4c35 0x99c3c35 0x7b60f77 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f6fdb6 0x6f73bb7 0x6f79761 0x73e3839 0x7412d74 0x7256dc1 0x7256eb9 0x732d51e 0x732d5d7 0x732d78e 0x7412df5 0x7256f36 0x7256fbb 0x72819ab 0x73ec44d 0x73ec4d2 0x73ec5b0 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:43.845 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f 0x1022ca8 0x10dd80a 0x993cbf7 0x993d0c1 0x993d334 0x993f05b 0x9a4f972 0x9a515c4 0x9a51c1e 0x9a52164 0x9a5c3f6 0x9a6086f 0x99c4c35 0x99c3c35 0x7b60f77 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f6fdb6 0x6f73bb7 0x6f79761 0x73e3839 0x7412d74 0x7256dc1 0x7256eb9 0x732d51e 0x732d5d7 0x732d78e 0x7412df5 0x7256f36 0x7256fbb 0x72819ab 0x73ec44d 0x73ec4d2 0x73ec5b0 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:43.993 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15ef5 0x9a4d0a9 0x9a4d2b1 0x9a5d40c 0x9a6086f 0x99c4c35 0x99c3c35 0x7b60f77 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f0b812 0x6f0db7a 0x73e34d8 0x740d434 0x7255508 0x7255600 0x732cdbe 0x732ce77 0x732d02e 0x740d4b5 0x725567d 0x7255702 0x727ca0b 0x73eae9c 0x73eaf21 0x73eafff 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:43.993 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc25 /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x10dd764 0x10dd7ff 0x993cbf7 0x993d0c1 0x993d334 0x993f05b 0x993f5a8 0x993f7a7 0x9a51a09 0x9a52164 0x9a5c3f6 0x9a6086f 0x99c4c35 0x99c3c35 0x7b61d78 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f0b812 0x6f0db7a 0x73e34d8 0x740d434 0x7255508 0x7255600 0x732cdbe 0x732ce77 0x732d02e 0x740d4b5 0x725567d 0x7255702 0x727ca0b 0x73eae9c 0x73eaf21 0x73eafff 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:43.993 INFO:teuthology.orchestra.run.trial103.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9c8e6a6 0x9ac12ed 0x9afc253 0x9aff80c 0x9affa7a 0x9affbc2 0x9b00081 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc657f /lib64/libasan.so.8+0x157c7 0x9a52170 0x9a5c3f6 0x9a6086f 0x99c4c35 0x99c3c35 0x7b60f77 0x7b6aae6 0x7a0f4cf 0x7a2845f 0x70e97ab 0x6f0b812 0x6f0db7a 0x73e34d8 0x740d434 0x7255508 0x7255600 0x732cdbe 0x732ce77 0x732d02e 0x740d4b5 0x725567d 0x7255702 0x727ca0b 0x73eae9c 0x73eaf21 0x73eafff 0x740d634 0x7388718 0x75533a1 0x73f4234 0x73f42af 0x74a88b2 0x759bcbc 0x9bc371d 0x9ac988c 0x9b822d9 0x9bd2877 0x9d17362 0x9d179de 0x265f883 0xa352190 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-17T10:03:46.149 INFO:teuthology.orchestra.run.trial103.stderr:----------------------------------------------------- 2026-02-17T10:03:46.149 INFO:teuthology.orchestra.run.trial103.stderr:Suppressions used: 2026-02-17T10:03:46.149 INFO:teuthology.orchestra.run.trial103.stderr: count bytes template 2026-02-17T10:03:46.149 INFO:teuthology.orchestra.run.trial103.stderr: 1 8 MallocExtension::Register 2026-02-17T10:03:46.149 INFO:teuthology.orchestra.run.trial103.stderr:----------------------------------------------------- 2026-02-17T10:03:46.150 INFO:teuthology.orchestra.run.trial103.stderr: 2026-02-17T10:03:46.179 DEBUG:teuthology.orchestra.run.trial103:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-17T10:03:46.204 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-17T10:03:46.205 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:03:46.205 DEBUG:teuthology.orchestra.run.trial103:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-17T10:03:46.270 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:03:46.270 DEBUG:teuthology.orchestra.run.trial103:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-17T10:03:46.333 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:03:46.333 DEBUG:teuthology.orchestra.run.trial103:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-17T10:03:46.396 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:03:46.396 DEBUG:teuthology.orchestra.run.trial103:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-17T10:03:46.460 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:03:46.460 DEBUG:teuthology.orchestra.run.trial103:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-17T10:03:46.516 INFO:tasks.ceph:Adding keys to all mons... 2026-02-17T10:03:46.517 DEBUG:teuthology.orchestra.run.trial103:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-17T10:03:46.579 INFO:teuthology.orchestra.run.trial103.stdout:[mgr.x] 2026-02-17T10:03:46.579 INFO:teuthology.orchestra.run.trial103.stdout: key = AQDwPJRp9/MsNxAARxqkbtL6BsylDhNXuviILQ== 2026-02-17T10:03:46.579 INFO:teuthology.orchestra.run.trial103.stdout:[osd.0] 2026-02-17T10:03:46.579 INFO:teuthology.orchestra.run.trial103.stdout: key = AQD1PJRpaQjvORAAXVRoZfx0+2sIj7sy1g25wQ== 2026-02-17T10:03:46.579 INFO:teuthology.orchestra.run.trial103.stdout:[osd.1] 2026-02-17T10:03:46.579 INFO:teuthology.orchestra.run.trial103.stdout: key = AQD6PJRpJJcaNRAAeSXNSefjAf6bq2XksBf8RQ== 2026-02-17T10:03:46.579 INFO:teuthology.orchestra.run.trial103.stdout:[osd.2] 2026-02-17T10:03:46.580 INFO:teuthology.orchestra.run.trial103.stdout: key = AQD/PJRpXArdLRAArUqRUd9xTQbJi6Ft4k/E4g== 2026-02-17T10:03:46.580 INFO:teuthology.orchestra.run.trial103.stdout:[client.0] 2026-02-17T10:03:46.580 INFO:teuthology.orchestra.run.trial103.stdout: key = AQDwPJRpv/jnORAAQqFIaqnwPmCMs3WYG/Rybw== 2026-02-17T10:03:46.581 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-17T10:03:46.669 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T10:03:46.718 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T10:03:46.767 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-17T10:03:46.815 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-17T10:03:46.864 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-17T10:03:46.864 DEBUG:teuthology.orchestra.run.trial103:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-17T10:03:46.889 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-17T10:03:46.975 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:46.973+0000 7f1eb4680c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:46.975 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:46.973+0000 7f1eb4680c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:46.982 DEBUG:teuthology.orchestra.run.trial103:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-17T10:03:47.007 DEBUG:teuthology.orchestra.run.trial103:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-17T10:03:47.063 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-17T10:03:47.063 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-17T10:03:47.063 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-17T10:03:47.105 INFO:tasks.ceph.mon.a:Started 2026-02-17T10:03:47.105 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-17T10:03:47.106 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-17T10:03:47.106 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-17T10:03:47.108 INFO:tasks.ceph.mgr.x:Started 2026-02-17T10:03:47.109 DEBUG:tasks.ceph:set 0 configs 2026-02-17T10:03:47.109 DEBUG:teuthology.orchestra.run.trial103:> sudo ceph --cluster ceph config dump 2026-02-17T10:03:47.169 INFO:tasks.ceph.mon.a.trial103.stderr:2026-02-17T10:03:47.166+0000 7fa02a239c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:47.169 INFO:tasks.ceph.mon.a.trial103.stderr:2026-02-17T10:03:47.166+0000 7fa02a239c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:47.169 INFO:tasks.ceph.mon.a.trial103.stderr:2026-02-17T10:03:47.167+0000 7fa02a239c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:47.173 INFO:tasks.ceph.mgr.x.trial103.stderr:2026-02-17T10:03:47.171+0000 7f7ddbbc2fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:47.185 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:47.181+0000 7fddbd45a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:47.186 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:47.184+0000 7fddbd45a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:47.217 INFO:teuthology.orchestra.run.trial103.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-17T10:03:47.225 INFO:tasks.ceph:Setting crush tunables to default 2026-02-17T10:03:47.226 DEBUG:teuthology.orchestra.run.trial103:> sudo ceph --cluster ceph osd crush tunables default 2026-02-17T10:03:47.302 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:47.297+0000 7f6523211640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:47.303 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:47.301+0000 7f6523211640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:47.338 INFO:teuthology.orchestra.run.trial103.stderr:adjusted tunables profile to default 2026-02-17T10:03:47.346 INFO:tasks.ceph:check_enable_crimson: True 2026-02-17T10:03:47.346 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-17T10:03:47.346 DEBUG:teuthology.orchestra.run.trial103:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-17T10:03:47.376 INFO:tasks.ceph.mgr.x.trial103.stderr:2026-02-17T10:03:47.374+0000 7f7ddbbc2fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:47.376 INFO:tasks.ceph.mgr.x.trial103.stderr:2026-02-17T10:03:47.374+0000 7f7ddbbc2fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:47.420 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:47.416+0000 7fd672d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:47.421 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:47.420+0000 7fd672d0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:47.459 INFO:tasks.ceph.mon.a.trial103.stderr:2026-02-17T10:03:47.457+0000 7fa0215f1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-17T10:03:47.460 INFO:tasks.ceph.mon.a.trial103.stderr:Please be aware that this feature is experimental, untested, 2026-02-17T10:03:47.460 INFO:tasks.ceph.mon.a.trial103.stderr:unsupported, and may result in data corruption, data loss, 2026-02-17T10:03:47.460 INFO:tasks.ceph.mon.a.trial103.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-17T10:03:47.460 INFO:tasks.ceph.mon.a.trial103.stderr:feature with important data. 2026-02-17T10:03:47.460 INFO:tasks.ceph.mon.a.trial103.stderr: 2026-02-17T10:03:48.213 INFO:tasks.ceph.mgr.x.trial103.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-17T10:03:48.213 INFO:tasks.ceph.mgr.x.trial103.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-17T10:03:48.213 INFO:tasks.ceph.mgr.x.trial103.stderr: from numpy import show_config as show_numpy_config 2026-02-17T10:03:48.349 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-17T10:03:48.350 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:03:48.350 DEBUG:teuthology.orchestra.run.trial103:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-17T10:03:48.415 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:03:48.415 DEBUG:teuthology.orchestra.run.trial103:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-17T10:03:48.479 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:03:48.479 DEBUG:teuthology.orchestra.run.trial103:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-17T10:03:48.542 DEBUG:teuthology.orchestra.run.trial103:> sudo ceph --cluster ceph osd new a5c43c4b-74d1-4464-b6c5-dbc357010d54 0 2026-02-17T10:03:48.655 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:48.650+0000 7f2e80dc1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:48.656 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:48.655+0000 7f2e80dc1640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:48.693 INFO:teuthology.orchestra.run.trial103.stdout:0 2026-02-17T10:03:48.702 DEBUG:teuthology.orchestra.run.trial103:> sudo ceph --cluster ceph osd new 508cf48d-890d-4371-9143-2ab10f86e4e9 1 2026-02-17T10:03:48.777 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:48.772+0000 7f158efd3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:48.779 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:48.777+0000 7f158efd3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:48.815 INFO:teuthology.orchestra.run.trial103.stdout:1 2026-02-17T10:03:48.822 DEBUG:teuthology.orchestra.run.trial103:> sudo ceph --cluster ceph osd new dae20159-5b3f-4d66-ae2c-4a69d2d950e7 2 2026-02-17T10:03:48.896 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:48.891+0000 7f0fc985a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:48.898 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:48.896+0000 7f0fc985a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:48.935 INFO:teuthology.orchestra.run.trial103.stdout:2 2026-02-17T10:03:48.943 DEBUG:teuthology.orchestra.run.trial103:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-17T10:03:48.970 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-17T10:03:48.971 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-17T10:03:49.012 INFO:tasks.ceph.osd.0:Started 2026-02-17T10:03:49.012 DEBUG:teuthology.orchestra.run.trial103:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-17T10:03:49.040 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-17T10:03:49.040 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-17T10:03:49.083 INFO:tasks.ceph.osd.1:Started 2026-02-17T10:03:49.084 DEBUG:teuthology.orchestra.run.trial103:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-17T10:03:49.111 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-17T10:03:49.111 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-17T10:03:49.153 INFO:tasks.ceph.osd.2:Started 2026-02-17T10:03:49.154 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T10:03:49.231 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:49.227+0000 7f5b93ec2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:49.232 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:03:49.230+0000 7f5b93ec2640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:49.266 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:03:49.266 INFO:teuthology.orchestra.run.trial103.stdout:{"epoch":6,"fsid":"9bb44552-44bc-4583-a00f-cc1b54452651","created":"2026-02-17T10:03:47.178918+0000","modified":"2026-02-17T10:03:48.933548+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T10:03:48.933548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"a5c43c4b-74d1-4464-b6c5-dbc357010d54","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"508cf48d-890d-4371-9143-2ab10f86e4e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"dae20159-5b3f-4d66-ae2c-4a69d2d950e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T10:03:49.274 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-17T10:03:49.274 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-17T10:03:49.576 DEBUG:teuthology.orchestra.run.trial103:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T10:03:49.643 INFO:teuthology.misc.health.trial103.stderr:2026-02-17T10:03:49.640+0000 7f757d1dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:49.644 INFO:teuthology.misc.health.trial103.stderr:2026-02-17T10:03:49.642+0000 7f757d1dc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:49.676 INFO:teuthology.misc.health.trial103.stdout: 2026-02-17T10:03:49.676 INFO:teuthology.misc.health.trial103.stdout:{"epoch":6,"fsid":"9bb44552-44bc-4583-a00f-cc1b54452651","created":"2026-02-17T10:03:47.178918+0000","modified":"2026-02-17T10:03:48.933548+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T10:03:48.933548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"a5c43c4b-74d1-4464-b6c5-dbc357010d54","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"508cf48d-890d-4371-9143-2ab10f86e4e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"dae20159-5b3f-4d66-ae2c-4a69d2d950e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T10:03:49.683 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-17T10:03:50.619 INFO:tasks.ceph.mgr.x.trial103.stderr:2026-02-17T10:03:50.617+0000 7f7cf6c4b640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-17T10:03:55.985 DEBUG:teuthology.orchestra.run.trial103:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T10:03:56.057 INFO:teuthology.misc.health.trial103.stderr:2026-02-17T10:03:56.053+0000 7f5c1808b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:56.058 INFO:teuthology.misc.health.trial103.stderr:2026-02-17T10:03:56.056+0000 7f5c1808b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:03:56.169 INFO:teuthology.misc.health.trial103.stdout: 2026-02-17T10:03:56.169 INFO:teuthology.misc.health.trial103.stdout:{"epoch":7,"fsid":"9bb44552-44bc-4583-a00f-cc1b54452651","created":"2026-02-17T10:03:47.178918+0000","modified":"2026-02-17T10:03:55.213047+0000","last_up_change":"0.000000","last_in_change":"2026-02-17T10:03:48.933548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":3,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"a5c43c4b-74d1-4464-b6c5-dbc357010d54","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"508cf48d-890d-4371-9143-2ab10f86e4e9","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"dae20159-5b3f-4d66-ae2c-4a69d2d950e7","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T10:03:56.177 DEBUG:teuthology.misc:0 of 3 OSDs are up 2026-02-17T10:04:02.479 DEBUG:teuthology.orchestra.run.trial103:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-17T10:04:02.548 INFO:teuthology.misc.health.trial103.stderr:2026-02-17T10:04:02.545+0000 7ff86f0cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:02.549 INFO:teuthology.misc.health.trial103.stderr:2026-02-17T10:04:02.548+0000 7ff86f0cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:02.662 INFO:teuthology.misc.health.trial103.stdout: 2026-02-17T10:04:02.662 INFO:teuthology.misc.health.trial103.stdout:{"epoch":13,"fsid":"9bb44552-44bc-4583-a00f-cc1b54452651","created":"2026-02-17T10:03:47.178918+0000","modified":"2026-02-17T10:04:02.226487+0000","last_up_change":"2026-02-17T10:03:58.215093+0000","last_in_change":"2026-02-17T10:03:48.933548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":1,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-17T10:03:59.629185+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5c43c4b-74d1-4464-b6c5-dbc357010d54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6806","nonce":444342425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6807","nonce":444342425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6810","nonce":444342425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6809","nonce":444342425}]},"public_addr":"10.20.193.103:6806/444342425","cluster_addr":"10.20.193.103:6807/444342425","heartbeat_back_addr":"10.20.193.103:6810/444342425","heartbeat_front_addr":"10.20.193.103:6809/444342425","state":["exists","up"]},{"osd":1,"uuid":"508cf48d-890d-4371-9143-2ab10f86e4e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":11,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6803","nonce":3758725814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6802","nonce":3758725814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6813","nonce":3758725814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6812","nonce":3758725814}]},"public_addr":"10.20.193.103:6803/3758725814","cluster_addr":"10.20.193.103:6802/3758725814","heartbeat_back_addr":"10.20.193.103:6813/3758725814","heartbeat_front_addr":"10.20.193.103:6812/3758725814","state":["exists","up"]},{"osd":2,"uuid":"dae20159-5b3f-4d66-ae2c-4a69d2d950e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6805","nonce":832153609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6804","nonce":832153609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6811","nonce":832153609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6808","nonce":832153609}]},"public_addr":"10.20.193.103:6805/832153609","cluster_addr":"10.20.193.103:6804/832153609","heartbeat_back_addr":"10.20.193.103:6811/832153609","heartbeat_front_addr":"10.20.193.103:6808/832153609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T10:04:02.670 DEBUG:teuthology.misc:3 of 3 OSDs are up 2026-02-17T10:04:02.670 INFO:tasks.ceph:Creating RBD pool 2026-02-17T10:04:02.671 DEBUG:teuthology.orchestra.run.trial103:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-17T10:04:02.747 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:02.741+0000 7f34f9a8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:02.748 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:02.746+0000 7f34f9a8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:03.563 INFO:teuthology.orchestra.run.trial103.stderr:pool 'rbd' created 2026-02-17T10:04:03.580 DEBUG:teuthology.orchestra.run.trial103:> rbd --cluster ceph pool init rbd 2026-02-17T10:04:03.622 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:03.619+0000 7f398c253240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:03.622 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:03.620+0000 7f398c253240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:03.623 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:03.621+0000 7f398c253240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:06.600 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-17T10:04:06.601 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-17T10:04:06.601 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-17T10:04:06.717 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:06.712+0000 7fb5025f5640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:06.718 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:06.716+0000 7fb50136c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:06.832 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:04:06.841 INFO:teuthology.orchestra.run.trial103.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-17T10:04:06.842 INFO:tasks.ceph_manager:config epoch is 1 2026-02-17T10:04:06.842 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-17T10:04:06.842 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-17T10:04:06.842 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-17T10:04:06.917 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:06.913+0000 7f8bef21a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:06.918 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:06.916+0000 7f8bef21a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.059 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:04:07.068 INFO:teuthology.orchestra.run.trial103.stdout:{"epoch":5,"flags":0,"active_gid":4103,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6800","nonce":340030769},{"type":"v1","addr":"10.20.193.103:6801","nonce":340030769}]},"active_addr":"10.20.193.103:6801/340030769","active_change":"2026-02-17T10:03:49.613250+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.6","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":true,"error_string":"","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"k8sevents","can_run":true,"error_string":"","module_options":{"ceph_event_retention_days":{"name":"ceph_event_retention_days","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"Days to hold ceph event information within local cache","long_desc":"","tags":[],"see_also":[]},"config_check_secs":{"name":"config_check_secs","type":"int","level":"advanced","flags":0,"default_value":"10","min":"10","max":"","enum_allowed":[],"desc":"interval (secs) to check for cluster configuration changes","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.103:0","nonce":3435239647}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.103:0","nonce":2243770481}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.103:0","nonce":2778916003}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.103:0","nonce":1884479946}]}]} 2026-02-17T10:04:07.071 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-17T10:04:07.072 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-17T10:04:07.072 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T10:04:07.148 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.144+0000 7f258fc46640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.149 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.147+0000 7f258fc46640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.260 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:04:07.260 INFO:teuthology.orchestra.run.trial103.stdout:{"epoch":17,"fsid":"9bb44552-44bc-4583-a00f-cc1b54452651","created":"2026-02-17T10:03:47.178918+0000","modified":"2026-02-17T10:04:06.572418+0000","last_up_change":"2026-02-17T10:03:58.215093+0000","last_in_change":"2026-02-17T10:03:48.933548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-17T10:03:59.629185+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-17T10:04:02.861455+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5c43c4b-74d1-4464-b6c5-dbc357010d54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6806","nonce":444342425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6807","nonce":444342425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6810","nonce":444342425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6809","nonce":444342425}]},"public_addr":"10.20.193.103:6806/444342425","cluster_addr":"10.20.193.103:6807/444342425","heartbeat_back_addr":"10.20.193.103:6810/444342425","heartbeat_front_addr":"10.20.193.103:6809/444342425","state":["exists","up"]},{"osd":1,"uuid":"508cf48d-890d-4371-9143-2ab10f86e4e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6803","nonce":3758725814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6802","nonce":3758725814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6813","nonce":3758725814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6812","nonce":3758725814}]},"public_addr":"10.20.193.103:6803/3758725814","cluster_addr":"10.20.193.103:6802/3758725814","heartbeat_back_addr":"10.20.193.103:6813/3758725814","heartbeat_front_addr":"10.20.193.103:6812/3758725814","state":["exists","up"]},{"osd":2,"uuid":"dae20159-5b3f-4d66-ae2c-4a69d2d950e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6805","nonce":832153609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6804","nonce":832153609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6811","nonce":832153609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6808","nonce":832153609}]},"public_addr":"10.20.193.103:6805/832153609","cluster_addr":"10.20.193.103:6804/832153609","heartbeat_back_addr":"10.20.193.103:6811/832153609","heartbeat_front_addr":"10.20.193.103:6808/832153609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T10:04:07.270 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-17T10:04:07.270 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-17T10:04:07.346 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.342+0000 7feecd65a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.347 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.345+0000 7feecd65a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.455 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:04:07.456 INFO:teuthology.orchestra.run.trial103.stdout:{"epoch":17,"fsid":"9bb44552-44bc-4583-a00f-cc1b54452651","created":"2026-02-17T10:03:47.178918+0000","modified":"2026-02-17T10:04:06.572418+0000","last_up_change":"2026-02-17T10:03:58.215093+0000","last_in_change":"2026-02-17T10:03:48.933548+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":3,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":".mgr","create_time":"2026-02-17T10:03:59.629185+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"13","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2.9900000095367432,"score_stable":2.9900000095367432,"optimal_score":0.67000001668930054,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":"rbd","create_time":"2026-02-17T10:04:02.861455+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"17","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":17,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":1.8799999952316284,"score_stable":1.8799999952316284,"optimal_score":1,"raw_score_acting":1.8799999952316284,"raw_score_stable":1.8799999952316284,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"a5c43c4b-74d1-4464-b6c5-dbc357010d54","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":9,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6806","nonce":444342425}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6807","nonce":444342425}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6810","nonce":444342425}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6809","nonce":444342425}]},"public_addr":"10.20.193.103:6806/444342425","cluster_addr":"10.20.193.103:6807/444342425","heartbeat_back_addr":"10.20.193.103:6810/444342425","heartbeat_front_addr":"10.20.193.103:6809/444342425","state":["exists","up"]},{"osd":1,"uuid":"508cf48d-890d-4371-9143-2ab10f86e4e9","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6803","nonce":3758725814}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6802","nonce":3758725814}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6813","nonce":3758725814}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6812","nonce":3758725814}]},"public_addr":"10.20.193.103:6803/3758725814","cluster_addr":"10.20.193.103:6802/3758725814","heartbeat_back_addr":"10.20.193.103:6813/3758725814","heartbeat_front_addr":"10.20.193.103:6812/3758725814","state":["exists","up"]},{"osd":2,"uuid":"dae20159-5b3f-4d66-ae2c-4a69d2d950e7","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":14,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6805","nonce":832153609}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6804","nonce":832153609}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6811","nonce":832153609}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.103:6808","nonce":832153609}]},"public_addr":"10.20.193.103:6805/832153609","cluster_addr":"10.20.193.103:6804/832153609","heartbeat_back_addr":"10.20.193.103:6811/832153609","heartbeat_front_addr":"10.20.193.103:6808/832153609","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":2,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-17T10:04:07.465 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-17T10:04:07.466 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-17T10:04:07.466 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-17T10:04:07.549 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.544+0000 7fc3e5764640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.549 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.544+0000 7fba79406640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.549 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.544+0000 7f3456a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.550 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.548+0000 7fc3dffff640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.550 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.548+0000 7fba79406640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.551 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.548+0000 7f3456a5a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.565 INFO:teuthology.orchestra.run.trial103.stdout:42949672969 2026-02-17T10:04:07.566 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-17T10:04:07.567 INFO:teuthology.orchestra.run.trial103.stdout:38654705674 2026-02-17T10:04:07.567 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-17T10:04:07.568 INFO:teuthology.orchestra.run.trial103.stdout:42949672969 2026-02-17T10:04:07.569 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-17T10:04:07.687 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.683+0000 7f874161a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.688 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.683+0000 7f9fb805a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.688 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.687+0000 7f874161a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.689 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.687+0000 7f9fb805a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.689 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.684+0000 7f96123cc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.690 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.689+0000 7f96123cc640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.801 INFO:teuthology.orchestra.run.trial103.stdout:42949672969 2026-02-17T10:04:07.803 INFO:teuthology.orchestra.run.trial103.stdout:42949672969 2026-02-17T10:04:07.806 INFO:teuthology.orchestra.run.trial103.stdout:38654705674 2026-02-17T10:04:07.811 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672969 for osd.2 2026-02-17T10:04:07.811 DEBUG:teuthology.parallel:result is None 2026-02-17T10:04:07.815 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672969 got 42949672969 for osd.1 2026-02-17T10:04:07.815 DEBUG:teuthology.parallel:result is None 2026-02-17T10:04:07.819 INFO:tasks.ceph.ceph_manager.ceph:need seq 38654705674 got 38654705674 for osd.0 2026-02-17T10:04:07.820 DEBUG:teuthology.parallel:result is None 2026-02-17T10:04:07.820 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-17T10:04:07.820 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T10:04:07.936 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.932+0000 7f4afef9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:07.937 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:07.935+0000 7f4afef9c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:08.045 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:04:08.045 INFO:teuthology.orchestra.run.trial103.stderr:dumped all 2026-02-17T10:04:08.053 INFO:teuthology.orchestra.run.trial103.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-17T10:04:07.616970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62917052,"kb_used_data":62917052,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94369300,"statfs":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.058265"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.584331+0000","last_change":"2026-02-17T10:04:06.586877+0000","last_active":"2026-02-17T10:04:06.584331+0000","last_peered":"2026-02-17T10:04:06.584331+0000","last_clean":"2026-02-17T10:04:06.584331+0000","last_became_active":"2026-02-17T10:04:04.598110+0000","last_became_peered":"2026-02-17T10:04:04.598110+0000","last_unstale":"2026-02-17T10:04:06.584331+0000","last_undegraded":"2026-02-17T10:04:06.584331+0000","last_fullsized":"2026-02-17T10:04:06.584331+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_clean_scrub_stamp":"2026-02-17T10:04:03.561603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.585790+0000","last_change":"2026-02-17T10:04:06.588753+0000","last_active":"2026-02-17T10:04:06.585790+0000","last_peered":"2026-02-17T10:04:06.585790+0000","last_clean":"2026-02-17T10:04:06.585790+0000","last_became_active":"2026-02-17T10:04:04.600103+0000","last_became_peered":"2026-02-17T10:04:04.600103+0000","last_unstale":"2026-02-17T10:04:06.585790+0000","last_undegraded":"2026-02-17T10:04:06.585790+0000","last_fullsized":"2026-02-17T10:04:06.585790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_clean_scrub_stamp":"2026-02-17T10:04:03.561603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.585476+0000","last_change":"2026-02-17T10:04:06.588859+0000","last_active":"2026-02-17T10:04:06.585476+0000","last_peered":"2026-02-17T10:04:06.585476+0000","last_clean":"2026-02-17T10:04:06.585476+0000","last_became_active":"2026-02-17T10:04:04.601701+0000","last_became_peered":"2026-02-17T10:04:04.601701+0000","last_unstale":"2026-02-17T10:04:06.585476+0000","last_undegraded":"2026-02-17T10:04:06.585476+0000","last_fullsized":"2026-02-17T10:04:06.585476+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_clean_scrub_stamp":"2026-02-17T10:04:03.561603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.585665+0000","last_change":"2026-02-17T10:04:06.587616+0000","last_active":"2026-02-17T10:04:06.585665+0000","last_peered":"2026-02-17T10:04:06.585665+0000","last_clean":"2026-02-17T10:04:06.585665+0000","last_became_active":"2026-02-17T10:04:04.605096+0000","last_became_peered":"2026-02-17T10:04:04.605096+0000","last_unstale":"2026-02-17T10:04:06.585665+0000","last_undegraded":"2026-02-17T10:04:06.585665+0000","last_fullsized":"2026-02-17T10:04:06.585665+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_clean_scrub_stamp":"2026-02-17T10:04:03.561603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.585937+0000","last_change":"2026-02-17T10:04:06.587209+0000","last_active":"2026-02-17T10:04:06.585937+0000","last_peered":"2026-02-17T10:04:06.585937+0000","last_clean":"2026-02-17T10:04:06.585937+0000","last_became_active":"2026-02-17T10:04:04.606710+0000","last_became_peered":"2026-02-17T10:04:04.606710+0000","last_unstale":"2026-02-17T10:04:06.585937+0000","last_undegraded":"2026-02-17T10:04:06.585937+0000","last_fullsized":"2026-02-17T10:04:06.585937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_clean_scrub_stamp":"2026-02-17T10:04:03.561603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.584910+0000","last_change":"2026-02-17T10:04:06.586604+0000","last_active":"2026-02-17T10:04:06.584910+0000","last_peered":"2026-02-17T10:04:06.584910+0000","last_clean":"2026-02-17T10:04:06.584910+0000","last_became_active":"2026-02-17T10:04:04.610819+0000","last_became_peered":"2026-02-17T10:04:04.610819+0000","last_unstale":"2026-02-17T10:04:06.584910+0000","last_undegraded":"2026-02-17T10:04:06.584910+0000","last_fullsized":"2026-02-17T10:04:06.584910+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_clean_scrub_stamp":"2026-02-17T10:04:03.561603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.582635+0000","last_change":"2026-02-17T10:04:06.585218+0000","last_active":"2026-02-17T10:04:06.582635+0000","last_peered":"2026-02-17T10:04:06.582635+0000","last_clean":"2026-02-17T10:04:06.582635+0000","last_became_active":"2026-02-17T10:04:04.604565+0000","last_became_peered":"2026-02-17T10:04:04.604565+0000","last_unstale":"2026-02-17T10:04:06.582635+0000","last_undegraded":"2026-02-17T10:04:06.582635+0000","last_fullsized":"2026-02-17T10:04:06.582635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_clean_scrub_stamp":"2026-02-17T10:04:03.561603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.586185+0000","last_change":"2026-02-17T10:04:06.588873+0000","last_active":"2026-02-17T10:04:06.586185+0000","last_peered":"2026-02-17T10:04:06.586185+0000","last_clean":"2026-02-17T10:04:06.586185+0000","last_became_active":"2026-02-17T10:04:04.602725+0000","last_became_peered":"2026-02-17T10:04:04.602725+0000","last_unstale":"2026-02-17T10:04:06.586185+0000","last_undegraded":"2026-02-17T10:04:06.586185+0000","last_fullsized":"2026-02-17T10:04:06.586185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_clean_scrub_stamp":"2026-02-17T10:04:03.561603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.584356+0000","last_change":"2026-02-17T10:04:01.254423+0000","last_active":"2026-02-17T10:04:06.584356+0000","last_peered":"2026-02-17T10:04:06.584356+0000","last_clean":"2026-02-17T10:04:06.584356+0000","last_became_active":"2026-02-17T10:04:01.247382+0000","last_became_peered":"2026-02-17T10:04:01.247382+0000","last_unstale":"2026-02-17T10:04:06.584356+0000","last_undegraded":"2026-02-17T10:04:06.584356+0000","last_fullsized":"2026-02-17T10:04:06.584356+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:00.218263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:00.218263+0000","last_clean_scrub_stamp":"2026-02-17T10:04:00.218263+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971820,"kb_used_data":20971820,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456964,"statfs":{"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972652,"kb_used_data":20972652,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456132,"statfs":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972580,"kb_used_data":20972580,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456204,"statfs":{"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T10:04:08.055 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-17T10:04:08.133 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:08.129+0000 7f005c0cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:08.134 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:08.132+0000 7f005c0cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:08.242 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:04:08.242 INFO:teuthology.orchestra.run.trial103.stderr:dumped all 2026-02-17T10:04:08.253 INFO:teuthology.orchestra.run.trial103.stdout:{"pg_ready":true,"pg_map":{"version":15,"stamp":"2026-02-17T10:04:07.616970+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":3,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":157286352,"kb_used":62917052,"kb_used_data":62917052,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":94369300,"statfs":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.058265"},"pg_stats":[{"pgid":"2.7","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.584331+0000","last_change":"2026-02-17T10:04:06.586877+0000","last_active":"2026-02-17T10:04:06.584331+0000","last_peered":"2026-02-17T10:04:06.584331+0000","last_clean":"2026-02-17T10:04:06.584331+0000","last_became_active":"2026-02-17T10:04:04.598110+0000","last_became_peered":"2026-02-17T10:04:04.598110+0000","last_unstale":"2026-02-17T10:04:06.584331+0000","last_undegraded":"2026-02-17T10:04:06.584331+0000","last_fullsized":"2026-02-17T10:04:06.584331+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_clean_scrub_stamp":"2026-02-17T10:04:03.561603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.6","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.585790+0000","last_change":"2026-02-17T10:04:06.588753+0000","last_active":"2026-02-17T10:04:06.585790+0000","last_peered":"2026-02-17T10:04:06.585790+0000","last_clean":"2026-02-17T10:04:06.585790+0000","last_became_active":"2026-02-17T10:04:04.600103+0000","last_became_peered":"2026-02-17T10:04:04.600103+0000","last_unstale":"2026-02-17T10:04:06.585790+0000","last_undegraded":"2026-02-17T10:04:06.585790+0000","last_fullsized":"2026-02-17T10:04:06.585790+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_clean_scrub_stamp":"2026-02-17T10:04:03.561603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.5","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.585476+0000","last_change":"2026-02-17T10:04:06.588859+0000","last_active":"2026-02-17T10:04:06.585476+0000","last_peered":"2026-02-17T10:04:06.585476+0000","last_clean":"2026-02-17T10:04:06.585476+0000","last_became_active":"2026-02-17T10:04:04.601701+0000","last_became_peered":"2026-02-17T10:04:04.601701+0000","last_unstale":"2026-02-17T10:04:06.585476+0000","last_undegraded":"2026-02-17T10:04:06.585476+0000","last_fullsized":"2026-02-17T10:04:06.585476+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_clean_scrub_stamp":"2026-02-17T10:04:03.561603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.4","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.585665+0000","last_change":"2026-02-17T10:04:06.587616+0000","last_active":"2026-02-17T10:04:06.585665+0000","last_peered":"2026-02-17T10:04:06.585665+0000","last_clean":"2026-02-17T10:04:06.585665+0000","last_became_active":"2026-02-17T10:04:04.605096+0000","last_became_peered":"2026-02-17T10:04:04.605096+0000","last_unstale":"2026-02-17T10:04:06.585665+0000","last_undegraded":"2026-02-17T10:04:06.585665+0000","last_fullsized":"2026-02-17T10:04:06.585665+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_clean_scrub_stamp":"2026-02-17T10:04:03.561603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"2.2","version":"17'2","reported_seq":17,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.585937+0000","last_change":"2026-02-17T10:04:06.587209+0000","last_active":"2026-02-17T10:04:06.585937+0000","last_peered":"2026-02-17T10:04:06.585937+0000","last_clean":"2026-02-17T10:04:06.585937+0000","last_became_active":"2026-02-17T10:04:04.606710+0000","last_became_peered":"2026-02-17T10:04:04.606710+0000","last_unstale":"2026-02-17T10:04:06.585937+0000","last_undegraded":"2026-02-17T10:04:06.585937+0000","last_fullsized":"2026-02-17T10:04:06.585937+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_clean_scrub_stamp":"2026-02-17T10:04:03.561603+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,1],"acting":[0,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"2.1","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.584910+0000","last_change":"2026-02-17T10:04:06.586604+0000","last_active":"2026-02-17T10:04:06.584910+0000","last_peered":"2026-02-17T10:04:06.584910+0000","last_clean":"2026-02-17T10:04:06.584910+0000","last_became_active":"2026-02-17T10:04:04.610819+0000","last_became_peered":"2026-02-17T10:04:04.610819+0000","last_unstale":"2026-02-17T10:04:06.584910+0000","last_undegraded":"2026-02-17T10:04:06.584910+0000","last_fullsized":"2026-02-17T10:04:06.584910+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_clean_scrub_stamp":"2026-02-17T10:04:03.561603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.0","version":"0'0","reported_seq":15,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.582635+0000","last_change":"2026-02-17T10:04:06.585218+0000","last_active":"2026-02-17T10:04:06.582635+0000","last_peered":"2026-02-17T10:04:06.582635+0000","last_clean":"2026-02-17T10:04:06.582635+0000","last_became_active":"2026-02-17T10:04:04.604565+0000","last_became_peered":"2026-02-17T10:04:04.604565+0000","last_unstale":"2026-02-17T10:04:06.582635+0000","last_undegraded":"2026-02-17T10:04:06.582635+0000","last_fullsized":"2026-02-17T10:04:06.582635+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_clean_scrub_stamp":"2026-02-17T10:04:03.561603+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[2,1],"acting":[2,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":2,"acting_primary":2,"purged_snaps":[]},{"pgid":"2.3","version":"15'1","reported_seq":16,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.586185+0000","last_change":"2026-02-17T10:04:06.588873+0000","last_active":"2026-02-17T10:04:06.586185+0000","last_peered":"2026-02-17T10:04:06.586185+0000","last_clean":"2026-02-17T10:04:06.586185+0000","last_became_active":"2026-02-17T10:04:04.602725+0000","last_became_peered":"2026-02-17T10:04:04.602725+0000","last_unstale":"2026-02-17T10:04:06.586185+0000","last_undegraded":"2026-02-17T10:04:06.586185+0000","last_fullsized":"2026-02-17T10:04:06.586185+0000","mapping_epoch":14,"log_start":"0'0","ondisk_log_start":"0'0","created":14,"last_epoch_clean":15,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:03.561603+0000","last_clean_scrub_stamp":"2026-02-17T10:04:03.561603+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.0","version":"12'52","reported_seq":90,"reported_epoch":17,"state":"active+clean","last_fresh":"2026-02-17T10:04:06.584356+0000","last_change":"2026-02-17T10:04:01.254423+0000","last_active":"2026-02-17T10:04:06.584356+0000","last_peered":"2026-02-17T10:04:06.584356+0000","last_clean":"2026-02-17T10:04:06.584356+0000","last_became_active":"2026-02-17T10:04:01.247382+0000","last_became_peered":"2026-02-17T10:04:01.247382+0000","last_unstale":"2026-02-17T10:04:06.584356+0000","last_undegraded":"2026-02-17T10:04:06.584356+0000","last_fullsized":"2026-02-17T10:04:06.584356+0000","mapping_epoch":11,"log_start":"0'0","ondisk_log_start":"0'0","created":11,"last_epoch_clean":12,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-17T10:04:00.218263+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-17T10:04:00.218263+0000","last_clean_scrub_stamp":"2026-02-17T10:04:00.218263+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":161061224448,"available":96634163200,"internally_reserved":0,"allocated":64427061248,"data_stored":64427061248,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3},{"poolid":1,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1}],"osd_stats":[{"osd":2,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20971820,"kb_used_data":20971820,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456964,"statfs":{"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":10,"seq":42949672969,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972652,"kb_used_data":20972652,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456132,"statfs":{"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":9,"seq":38654705674,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52428784,"kb_used":20972580,"kb_used_data":20972580,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":31456204,"statfs":{"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53687074816,"available":32211152896,"internally_reserved":0,"allocated":21475921920,"data_stored":21475921920,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":1,"total":53687074816,"available":32211079168,"internally_reserved":0,"allocated":21475995648,"data_stored":21475995648,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":2,"total":53687074816,"available":32211931136,"internally_reserved":0,"allocated":21475143680,"data_stored":21475143680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-17T10:04:08.254 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-17T10:04:08.255 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-17T10:04:08.255 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-17T10:04:08.255 DEBUG:teuthology.orchestra.run.trial103:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-17T10:04:08.331 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:08.328+0000 7f92b525a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:08.332 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:04:08.330+0000 7f92b525a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:04:08.469 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:04:08.469 INFO:teuthology.orchestra.run.trial103.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-17T10:04:08.478 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-17T10:04:08.478 INFO:teuthology.run_tasks:Running task ssh_keys... 2026-02-17T10:04:08.556 DEBUG:teuthology.orchestra.run.trial103:> echo 'StrictHostKeyChecking no 2026-02-17T10:04:08.556 DEBUG:teuthology.orchestra.run.trial103:> ' > /home/ubuntu/.ssh/config && echo 'UserKnownHostsFile ' /dev/null >> /home/ubuntu/.ssh/config && chmod 600 /home/ubuntu/.ssh/config 2026-02-17T10:04:08.573 INFO:teuthology.task.ssh_keys:generated public key AAAAB3NzaC1yc2EAAAADAQABAAABAQDQiowXxiPyyvhZvjjItKRYJ3nIBzd5nn2vsKiIDdpdBHQxO/0vZh19FCZDQq5yOueukzVZRhvwwhd1fDTzjFz+OABbqrFJd0w33W4LDUuK0evr6c9OJ2rX/yF/3VQSVNFmj4sPDm3iWyqZND0kh9JIrrrPhPyUsML7tTXfhSSVi/z8/G8wFyEuK/ObSw/JmreJ13vGB4XfVAvzN9uUosasLcoQTfT1umyyTSd4gLKKRClIi2BPSOZKuEZ26nIRV4KJKgti3sPtiI+M5zlXeMrAgcjs+V20yrGpEw6v03euMu0UZywA0BiU3WUExQ2YibJdA0wpYZO8FUeQhmZ5gv+r 2026-02-17T10:04:08.573 INFO:teuthology.task.ssh_keys:pushing keys to trial103.front.sepia.ceph.com for ubuntu 2026-02-17T10:04:08.573 DEBUG:teuthology.orchestra.run.trial103:> rm -f -- /home/ubuntu/.ssh/id_rsa 2026-02-17T10:04:08.629 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:04:08.629 DEBUG:teuthology.orchestra.run.trial103:> dd of=/home/ubuntu/.ssh/id_rsa 2026-02-17T10:04:08.630 DEBUG:teuthology.orchestra.run.trial103:> chmod 0500 /home/ubuntu/.ssh/id_rsa 2026-02-17T10:04:08.687 DEBUG:teuthology.orchestra.run.trial103:> rm -f -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-17T10:04:08.743 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:04:08.743 DEBUG:teuthology.orchestra.run.trial103:> dd of=/home/ubuntu/.ssh/id_rsa.pub 2026-02-17T10:04:08.799 DEBUG:teuthology.orchestra.run.trial103:> cp -v -a /home/ubuntu/.ssh/authorized_keys /home/ubuntu/.ssh/authorized_keys_2026-02-17_10:04:08:799201 2026-02-17T10:04:08.856 INFO:teuthology.orchestra.run.trial103.stdout:'/home/ubuntu/.ssh/authorized_keys' -> '/home/ubuntu/.ssh/authorized_keys_2026-02-17_10:04:08:799201' 2026-02-17T10:04:08.856 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:04:08.856 DEBUG:teuthology.orchestra.run.trial103:> dd of=/home/ubuntu/.ssh/authorized_keys conv=notrunc oflag=append 2026-02-17T10:04:08.912 INFO:teuthology.run_tasks:Running task cbt... 2026-02-17T10:04:08.937 INFO:tasks.cbt:cbt configuration is {'cluster': {'user': 'ubuntu', 'head': 'ubuntu@trial103.front.sepia.ceph.com', 'osds': ['ubuntu@trial103.front.sepia.ceph.com'], 'mons': ['ubuntu@trial103.front.sepia.ceph.com'], 'clients': ['ubuntu@trial103.front.sepia.ceph.com'], 'rgws': {'ubuntu@trial103.front.sepia.ceph.com': None}, 'osds_per_node': 3, 'rebuild_every_test': False, 'use_existing': True, 'is_teuthology': True, 'iterations': 1, 'tmp_dir': '/tmp/cbt', 'pool_profiles': {'replicated': {'pg_size': 256, 'pgp_size': 256, 'replication': 'replicated'}}, 'pid_dir': '/var/run/ceph'}, 'benchmarks': {'radosbench': {'concurrent_ops': 4, 'concurrent_procs': 2, 'op_size': [4096], 'pool_monitoring_list': ['collectl'], 'pool_profile': 'replicated', 'readmode': 'rand', 'run_monitoring_list': ['collectl'], 'time': 60, 'write_only': False}}, 'client_endpoints': None, 'monitoring_profiles': {'perf': {'args': 'stat -p {pid} -o {perf_dir}/perf_stat.{pid}', 'nodes': ['osds'], 'perf_cmd': 'perf', 'pid_glob': 'crimson-osd.*.pid'}}} 2026-02-17T10:04:08.937 DEBUG:teuthology.orchestra.run.trial103:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/cbt 2026-02-17T10:04:08.974 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:04:08.974 DEBUG:teuthology.orchestra.run.trial103:> dd of=/home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2026-02-17T10:04:09.026 DEBUG:teuthology.orchestra.run.trial103:> git clone --depth 1 -b master https://github.com/ceph/cbt.git /home/ubuntu/cephtest/cbt 2026-02-17T10:04:09.089 INFO:teuthology.orchestra.run.trial103.stderr:Cloning into '/home/ubuntu/cephtest/cbt'... 2026-02-17T10:04:09.316 INFO:tasks.cbt:Installing collectl 2026-02-17T10:04:09.316 DEBUG:teuthology.orchestra.run.trial103:> sudo mkdir collectl && cd collectl && sudo wget https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download -O collectl.tar.gz && sudo tar -xvf collectl.tar.gz && cd collectl-4.3.1 && sudo ./INSTALL 2026-02-17T10:04:09.390 INFO:teuthology.orchestra.run.trial103.stderr:--2026-02-17 10:04:09-- https://sourceforge.net/projects/collectl/files/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz/download 2026-02-17T10:04:09.452 INFO:teuthology.orchestra.run.trial103.stderr:Resolving sourceforge.net (sourceforge.net)... 104.18.13.149, 104.18.12.149, 2606:4700::6812:c95, ... 2026-02-17T10:04:09.455 INFO:teuthology.orchestra.run.trial103.stderr:Connecting to sourceforge.net (sourceforge.net)|104.18.13.149|:443... connected. 2026-02-17T10:04:09.754 INFO:teuthology.orchestra.run.trial103.stderr:HTTP request sent, awaiting response... 302 Found 2026-02-17T10:04:09.755 INFO:teuthology.orchestra.run.trial103.stderr:Location: https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABplD0ZqlD2-SGe3PFdeto4VbR-Nfqr5iADEXHLV4FaI3OPldfjfFYDkBgb7yXcAh8H-89BR2VuGbON-qYiMpnXOOw32A%3D%3D&use_mirror=cfhcable&r= [following] 2026-02-17T10:04:09.755 INFO:teuthology.orchestra.run.trial103.stderr:--2026-02-17 10:04:09-- https://downloads.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?ts=gAAAAABplD0ZqlD2-SGe3PFdeto4VbR-Nfqr5iADEXHLV4FaI3OPldfjfFYDkBgb7yXcAh8H-89BR2VuGbON-qYiMpnXOOw32A%3D%3D&use_mirror=cfhcable&r= 2026-02-17T10:04:09.773 INFO:teuthology.orchestra.run.trial103.stderr:Resolving downloads.sourceforge.net (downloads.sourceforge.net)... 104.18.12.149, 104.18.13.149, 2606:4700::6812:d95, ... 2026-02-17T10:04:09.777 INFO:teuthology.orchestra.run.trial103.stderr:Connecting to downloads.sourceforge.net (downloads.sourceforge.net)|104.18.12.149|:443... connected. 2026-02-17T10:04:09.953 INFO:teuthology.orchestra.run.trial103.stderr:HTTP request sent, awaiting response... 302 Found 2026-02-17T10:04:09.953 INFO:teuthology.orchestra.run.trial103.stderr:Location: https://cfhcable.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 [following] 2026-02-17T10:04:09.953 INFO:teuthology.orchestra.run.trial103.stderr:--2026-02-17 10:04:09-- https://cfhcable.dl.sourceforge.net/project/collectl/collectl/collectl-4.3.1/collectl-4.3.1.src.tar.gz?viasf=1 2026-02-17T10:04:09.962 INFO:teuthology.orchestra.run.trial103.stderr:Resolving cfhcable.dl.sourceforge.net (cfhcable.dl.sourceforge.net)... 146.71.73.6 2026-02-17T10:04:10.003 INFO:teuthology.orchestra.run.trial103.stderr:Connecting to cfhcable.dl.sourceforge.net (cfhcable.dl.sourceforge.net)|146.71.73.6|:443... connected. 2026-02-17T10:04:10.355 INFO:teuthology.orchestra.run.trial103.stderr:HTTP request sent, awaiting response... 200 OK 2026-02-17T10:04:10.356 INFO:teuthology.orchestra.run.trial103.stderr:Length: 620693 (606K) [application/x-gzip] 2026-02-17T10:04:10.356 INFO:teuthology.orchestra.run.trial103.stderr:Saving to: ‘collectl.tar.gz’ 2026-02-17T10:04:10.356 INFO:teuthology.orchestra.run.trial103.stderr: 2026-02-17T10:04:10.397 INFO:teuthology.orchestra.run.trial103.stderr: 0K .......... .......... .......... .......... .......... 8% 1.19M 0s 2026-02-17T10:04:10.438 INFO:teuthology.orchestra.run.trial103.stderr: 50K .......... .......... .......... .......... .......... 16% 1.20M 0s 2026-02-17T10:04:10.438 INFO:teuthology.orchestra.run.trial103.stderr: 100K .......... .......... .......... .......... .......... 24% 266M 0s 2026-02-17T10:04:10.480 INFO:teuthology.orchestra.run.trial103.stderr: 150K .......... .......... .......... .......... .......... 32% 1.20M 0s 2026-02-17T10:04:10.480 INFO:teuthology.orchestra.run.trial103.stderr: 200K .......... .......... .......... .......... .......... 41% 433M 0s 2026-02-17T10:04:10.481 INFO:teuthology.orchestra.run.trial103.stderr: 250K .......... .......... .......... .......... .......... 49% 343M 0s 2026-02-17T10:04:10.525 INFO:teuthology.orchestra.run.trial103.stderr: 300K .......... .......... .......... .......... .......... 57% 1.21M 0s 2026-02-17T10:04:10.525 INFO:teuthology.orchestra.run.trial103.stderr: 350K .......... .......... .......... .......... .......... 65% 334M 0s 2026-02-17T10:04:10.525 INFO:teuthology.orchestra.run.trial103.stderr: 400K .......... .......... .......... .......... .......... 74% 340M 0s 2026-02-17T10:04:10.525 INFO:teuthology.orchestra.run.trial103.stderr: 450K .......... .......... .......... .......... .......... 82% 404M 0s 2026-02-17T10:04:10.525 INFO:teuthology.orchestra.run.trial103.stderr: 500K .......... .......... .......... .......... .......... 90% 406M 0s 2026-02-17T10:04:10.525 INFO:teuthology.orchestra.run.trial103.stderr: 550K .......... .......... .......... .......... .......... 98% 367M 0s 2026-02-17T10:04:10.525 INFO:teuthology.orchestra.run.trial103.stderr: 600K ...... 100% 11.4T=0.2s 2026-02-17T10:04:10.525 INFO:teuthology.orchestra.run.trial103.stderr: 2026-02-17T10:04:10.526 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17 10:04:10 (3.61 MB/s) - ‘collectl.tar.gz’ saved [620693/620693] 2026-02-17T10:04:10.526 INFO:teuthology.orchestra.run.trial103.stderr: 2026-02-17T10:04:10.542 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/ 2026-02-17T10:04:10.542 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/collectl.conf 2026-02-17T10:04:10.542 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/ 2026-02-17T10:04:10.542 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/SlowProc.html 2026-02-17T10:04:10.542 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/BuddyInfo.html 2026-02-17T10:04:10.542 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/WhySummary.html 2026-02-17T10:04:10.543 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Socket.html 2026-02-17T10:04:10.543 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/colmux.html 2026-02-17T10:04:10.543 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Tutorial.html 2026-02-17T10:04:10.543 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Gexpr.html 2026-02-17T10:04:10.543 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Import.html 2026-02-17T10:04:10.543 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/style.css 2026-02-17T10:04:10.543 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Ganglia3.jpg 2026-02-17T10:04:10.543 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Ganglia2.jpg 2026-02-17T10:04:10.543 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/FileNaming.html 2026-02-17T10:04:10.543 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/TheMath.html 2026-02-17T10:04:10.543 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Environmental.html 2026-02-17T10:04:10.543 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Process.html 2026-02-17T10:04:10.544 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/OutputFiles.html 2026-02-17T10:04:10.544 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/HiResTime.html 2026-02-17T10:04:10.544 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/OutputFormats.html 2026-02-17T10:04:10.544 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/slub.jpg 2026-02-17T10:04:10.544 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Lustre.html 2026-02-17T10:04:10.544 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Documentation.html 2026-02-17T10:04:10.544 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Inodes.html 2026-02-17T10:04:10.544 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Graphite.html 2026-02-17T10:04:10.544 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/InputFiles.html 2026-02-17T10:04:10.544 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Export.html 2026-02-17T10:04:10.544 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Interrupts.html 2026-02-17T10:04:10.544 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Startup.html 2026-02-17T10:04:10.544 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Performance.html 2026-02-17T10:04:10.545 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Messages.html 2026-02-17T10:04:10.545 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Logging.html 2026-02-17T10:04:10.545 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Data-detail.html 2026-02-17T10:04:10.545 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/CPUs.html 2026-02-17T10:04:10.545 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Matrix.html 2026-02-17T10:04:10.545 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Data-brief.html 2026-02-17T10:04:10.545 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/SlabInfo.html 2026-02-17T10:04:10.545 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Misc.html 2026-02-17T10:04:10.545 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Examples.html 2026-02-17T10:04:10.545 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/OpenStack.html 2026-02-17T10:04:10.545 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Plotfiles.html 2026-02-17T10:04:10.545 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/ColmuxCPU.jpg 2026-02-17T10:04:10.545 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Tutorial-Lustre.html 2026-02-17T10:04:10.546 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Data-verbose.html 2026-02-17T10:04:10.546 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Data.html 2026-02-17T10:04:10.546 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Exceptions.html 2026-02-17T10:04:10.546 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/RunningAsAService.html 2026-02-17T10:04:10.546 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/FAQ-collectl.html 2026-02-17T10:04:10.546 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Ganglia.jpg 2026-02-17T10:04:10.546 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/OperationalModes.html 2026-02-17T10:04:10.546 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Playback.html 2026-02-17T10:04:10.546 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Hello.html 2026-02-17T10:04:10.546 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Sockets.html 2026-02-17T10:04:10.546 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Network.html 2026-02-17T10:04:10.546 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Infiniband.html 2026-02-17T10:04:10.546 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Memory.html 2026-02-17T10:04:10.546 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/ColmuxLustre.jpg 2026-02-17T10:04:10.547 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/Disks.html 2026-02-17T10:04:10.547 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/docs/NfsInfo.html 2026-02-17T10:04:10.547 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/service/ 2026-02-17T10:04:10.547 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/service/collectl.service 2026-02-17T10:04:10.547 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/initd/ 2026-02-17T10:04:10.547 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/initd/collectl-suse 2026-02-17T10:04:10.547 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/initd/collectl-debian 2026-02-17T10:04:10.547 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/initd/collectl-generic 2026-02-17T10:04:10.547 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/initd/collectl 2026-02-17T10:04:10.547 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/misc.ph 2026-02-17T10:04:10.547 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/gexpr.ph 2026-02-17T10:04:10.547 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/statsd.ph 2026-02-17T10:04:10.547 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/envrules.std 2026-02-17T10:04:10.548 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/hello.ph 2026-02-17T10:04:10.548 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/vmsum.ph 2026-02-17T10:04:10.548 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/proctree.ph 2026-02-17T10:04:10.548 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/README-WINDOWS 2026-02-17T10:04:10.548 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/formatit.ph 2026-02-17T10:04:10.548 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/colmux 2026-02-17T10:04:10.548 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/INSTALL 2026-02-17T10:04:10.548 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/graphite.ph 2026-02-17T10:04:10.548 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/RELEASE-collectl 2026-02-17T10:04:10.548 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/vmstat.ph 2026-02-17T10:04:10.548 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/README 2026-02-17T10:04:10.548 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/man1/ 2026-02-17T10:04:10.548 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/man1/collectl.1 2026-02-17T10:04:10.548 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/man1/colmux.1 2026-02-17T10:04:10.549 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/lexpr.ph 2026-02-17T10:04:10.549 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/client.pl 2026-02-17T10:04:10.549 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/collectl 2026-02-17T10:04:10.549 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/COPYING 2026-02-17T10:04:10.549 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/vnet.ph 2026-02-17T10:04:10.549 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/GPL 2026-02-17T10:04:10.549 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/UNINSTALL 2026-02-17T10:04:10.549 INFO:teuthology.orchestra.run.trial103.stdout:collectl-4.3.1/ARTISTIC 2026-02-17T10:04:10.584 INFO:teuthology.orchestra.run.trial103.stderr:./INSTALL: line 90: chkconfig: command not found 2026-02-17T10:04:10.586 DEBUG:teuthology.orchestra.run.trial103:> sudo yum -y install librbd-devel pdsh pdsh-rcmd-ssh perf 2026-02-17T10:04:10.945 INFO:teuthology.orchestra.run.trial103.stdout:Last metadata expiration check: 0:00:56 ago on Tue 17 Feb 2026 10:03:14 AM UTC. 2026-02-17T10:04:11.030 INFO:teuthology.orchestra.run.trial103.stdout:Dependencies resolved. 2026-02-17T10:04:11.031 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:04:11.031 INFO:teuthology.orchestra.run.trial103.stdout: Package Arch Version Repository Size 2026-02-17T10:04:11.032 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:04:11.032 INFO:teuthology.orchestra.run.trial103.stdout:Installing: 2026-02-17T10:04:11.032 INFO:teuthology.orchestra.run.trial103.stdout: librbd-devel x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 24 k 2026-02-17T10:04:11.032 INFO:teuthology.orchestra.run.trial103.stdout: pdsh x86_64 2.34-7.el9 epel 118 k 2026-02-17T10:04:11.032 INFO:teuthology.orchestra.run.trial103.stdout: pdsh-rcmd-ssh x86_64 2.34-7.el9 epel 13 k 2026-02-17T10:04:11.032 INFO:teuthology.orchestra.run.trial103.stdout: perf x86_64 5.14.0-681.el9 appstream 3.2 M 2026-02-17T10:04:11.032 INFO:teuthology.orchestra.run.trial103.stdout:Installing dependencies: 2026-02-17T10:04:11.032 INFO:teuthology.orchestra.run.trial103.stdout: libradospp-devel x86_64 2:20.3.0-5345.gb58fab16.el9 ceph 31 k 2026-02-17T10:04:11.032 INFO:teuthology.orchestra.run.trial103.stdout: libtraceevent x86_64 1.8.4-2.el9 baseos 290 k 2026-02-17T10:04:11.032 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:04:11.032 INFO:teuthology.orchestra.run.trial103.stdout:Transaction Summary 2026-02-17T10:04:11.033 INFO:teuthology.orchestra.run.trial103.stdout:================================================================================ 2026-02-17T10:04:11.033 INFO:teuthology.orchestra.run.trial103.stdout:Install 6 Packages 2026-02-17T10:04:11.033 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:04:11.033 INFO:teuthology.orchestra.run.trial103.stdout:Total download size: 3.7 M 2026-02-17T10:04:11.033 INFO:teuthology.orchestra.run.trial103.stdout:Installed size: 14 M 2026-02-17T10:04:11.033 INFO:teuthology.orchestra.run.trial103.stdout:Downloading Packages: 2026-02-17T10:04:11.615 INFO:teuthology.orchestra.run.trial103.stdout:(1/6): libradospp-devel-20.3.0-5345.gb58fab16.e 119 kB/s | 31 kB 00:00 2026-02-17T10:04:11.618 INFO:teuthology.orchestra.run.trial103.stdout:(2/6): librbd-devel-20.3.0-5345.gb58fab16.el9.x 89 kB/s | 24 kB 00:00 2026-02-17T10:04:11.691 INFO:teuthology.orchestra.run.trial103.stdout:(3/6): perf-5.14.0-681.el9.x86_64.rpm 42 MB/s | 3.2 MB 00:00 2026-02-17T10:04:11.757 INFO:teuthology.orchestra.run.trial103.stdout:(4/6): pdsh-rcmd-ssh-2.34-7.el9.x86_64.rpm 200 kB/s | 13 kB 00:00 2026-02-17T10:04:11.770 INFO:teuthology.orchestra.run.trial103.stdout:(5/6): pdsh-2.34-7.el9.x86_64.rpm 778 kB/s | 118 kB 00:00 2026-02-17T10:04:12.258 INFO:teuthology.orchestra.run.trial103.stdout:(6/6): libtraceevent-1.8.4-2.el9.x86_64.rpm 320 kB/s | 290 kB 00:00 2026-02-17T10:04:12.264 INFO:teuthology.orchestra.run.trial103.stdout:-------------------------------------------------------------------------------- 2026-02-17T10:04:12.264 INFO:teuthology.orchestra.run.trial103.stdout:Total 3.0 MB/s | 3.7 MB 00:01 2026-02-17T10:04:12.288 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction check 2026-02-17T10:04:12.298 INFO:teuthology.orchestra.run.trial103.stdout:Transaction check succeeded. 2026-02-17T10:04:12.298 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction test 2026-02-17T10:04:12.355 INFO:teuthology.orchestra.run.trial103.stdout:Transaction test succeeded. 2026-02-17T10:04:12.355 INFO:teuthology.orchestra.run.trial103.stdout:Running transaction 2026-02-17T10:04:12.428 INFO:teuthology.orchestra.run.trial103.stdout: Preparing : 1/1 2026-02-17T10:04:12.433 INFO:teuthology.orchestra.run.trial103.stdout: Installing : pdsh-rcmd-ssh-2.34-7.el9.x86_64 1/6 2026-02-17T10:04:12.473 INFO:teuthology.orchestra.run.trial103.stdout: Installing : pdsh-2.34-7.el9.x86_64 2/6 2026-02-17T10:04:12.477 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libtraceevent-1.8.4-2.el9.x86_64 3/6 2026-02-17T10:04:12.480 INFO:teuthology.orchestra.run.trial103.stdout: Installing : libradospp-devel-2:20.3.0-5345.gb58fab16.el9.x86_64 4/6 2026-02-17T10:04:12.644 INFO:teuthology.orchestra.run.trial103.stdout: Installing : librbd-devel-2:20.3.0-5345.gb58fab16.el9.x86_64 5/6 2026-02-17T10:04:12.666 INFO:teuthology.orchestra.run.trial103.stdout: Installing : perf-5.14.0-681.el9.x86_64 6/6 2026-02-17T10:04:12.774 INFO:teuthology.orchestra.run.trial103.stdout: Running scriptlet: perf-5.14.0-681.el9.x86_64 6/6 2026-02-17T10:04:12.774 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libradospp-devel-2:20.3.0-5345.gb58fab16.el9.x86_64 1/6 2026-02-17T10:04:12.774 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : librbd-devel-2:20.3.0-5345.gb58fab16.el9.x86_64 2/6 2026-02-17T10:04:12.774 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : libtraceevent-1.8.4-2.el9.x86_64 3/6 2026-02-17T10:04:12.774 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : perf-5.14.0-681.el9.x86_64 4/6 2026-02-17T10:04:12.774 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : pdsh-2.34-7.el9.x86_64 5/6 2026-02-17T10:04:12.834 INFO:teuthology.orchestra.run.trial103.stdout: Verifying : pdsh-rcmd-ssh-2.34-7.el9.x86_64 6/6 2026-02-17T10:04:12.834 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:04:12.834 INFO:teuthology.orchestra.run.trial103.stdout:Installed: 2026-02-17T10:04:12.834 INFO:teuthology.orchestra.run.trial103.stdout: libradospp-devel-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:04:12.834 INFO:teuthology.orchestra.run.trial103.stdout: librbd-devel-2:20.3.0-5345.gb58fab16.el9.x86_64 2026-02-17T10:04:12.835 INFO:teuthology.orchestra.run.trial103.stdout: libtraceevent-1.8.4-2.el9.x86_64 2026-02-17T10:04:12.835 INFO:teuthology.orchestra.run.trial103.stdout: pdsh-2.34-7.el9.x86_64 2026-02-17T10:04:12.835 INFO:teuthology.orchestra.run.trial103.stdout: pdsh-rcmd-ssh-2.34-7.el9.x86_64 2026-02-17T10:04:12.835 INFO:teuthology.orchestra.run.trial103.stdout: perf-5.14.0-681.el9.x86_64 2026-02-17T10:04:12.835 INFO:teuthology.orchestra.run.trial103.stdout: 2026-02-17T10:04:12.835 INFO:teuthology.orchestra.run.trial103.stdout:Complete! 2026-02-17T10:04:12.865 DEBUG:teuthology.orchestra.run.trial103:> python3 -m pip install -r /home/ubuntu/cephtest/cbt/requirements.txt 2026-02-17T10:04:13.006 INFO:teuthology.orchestra.run.trial103.stdout:Defaulting to user installation because normal site-packages is not writeable 2026-02-17T10:04:13.018 INFO:teuthology.orchestra.run.trial103.stdout:Requirement already satisfied: pyyaml in /usr/lib64/python3.9/site-packages (from -r /home/ubuntu/cephtest/cbt/requirements.txt (line 1)) (5.4.1) 2026-02-17T10:04:13.018 INFO:teuthology.orchestra.run.trial103.stdout:Requirement already satisfied: lxml in /usr/lib64/python3.9/site-packages (from -r /home/ubuntu/cephtest/cbt/requirements.txt (line 2)) (4.6.5) 2026-02-17T10:04:13.341 INFO:teuthology.orchestra.run.trial103.stdout:Collecting matplotlib 2026-02-17T10:04:13.371 INFO:teuthology.orchestra.run.trial103.stdout: Downloading matplotlib-3.9.4-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (8.3 MB) 2026-02-17T10:04:13.745 INFO:teuthology.orchestra.run.trial103.stdout:Collecting fonttools>=4.22.0 2026-02-17T10:04:13.751 INFO:teuthology.orchestra.run.trial103.stdout: Downloading fonttools-4.60.2-cp39-cp39-manylinux2014_x86_64.manylinux_2_17_x86_64.whl (4.8 MB) 2026-02-17T10:04:13.775 INFO:teuthology.orchestra.run.trial103.stdout:Requirement already satisfied: python-dateutil>=2.7 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (2.9.0.post0) 2026-02-17T10:04:13.787 INFO:teuthology.orchestra.run.trial103.stdout:Collecting cycler>=0.10 2026-02-17T10:04:13.791 INFO:teuthology.orchestra.run.trial103.stdout: Downloading cycler-0.12.1-py3-none-any.whl (8.3 kB) 2026-02-17T10:04:14.172 INFO:teuthology.orchestra.run.trial103.stdout:Collecting pillow>=8 2026-02-17T10:04:14.178 INFO:teuthology.orchestra.run.trial103.stdout: Downloading pillow-11.3.0-cp39-cp39-manylinux_2_27_x86_64.manylinux_2_28_x86_64.whl (6.6 MB) 2026-02-17T10:04:14.209 INFO:teuthology.orchestra.run.trial103.stdout:Requirement already satisfied: pyparsing>=2.3.1 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (2.4.7) 2026-02-17T10:04:14.243 INFO:teuthology.orchestra.run.trial103.stdout:Collecting importlib-resources>=3.2.0 2026-02-17T10:04:14.247 INFO:teuthology.orchestra.run.trial103.stdout: Downloading importlib_resources-6.5.2-py3-none-any.whl (37 kB) 2026-02-17T10:04:14.249 INFO:teuthology.orchestra.run.trial103.stdout:Requirement already satisfied: numpy>=1.23 in /usr/lib64/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (1.23.5) 2026-02-17T10:04:14.385 INFO:teuthology.orchestra.run.trial103.stdout:Collecting contourpy>=1.0.1 2026-02-17T10:04:14.389 INFO:teuthology.orchestra.run.trial103.stdout: Downloading contourpy-1.3.0-cp39-cp39-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (321 kB) 2026-02-17T10:04:14.538 INFO:teuthology.orchestra.run.trial103.stdout:Collecting kiwisolver>=1.3.1 2026-02-17T10:04:14.544 INFO:teuthology.orchestra.run.trial103.stdout: Downloading kiwisolver-1.4.7-cp39-cp39-manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.6 MB) 2026-02-17T10:04:14.552 INFO:teuthology.orchestra.run.trial103.stdout:Requirement already satisfied: packaging>=20.0 in /usr/lib/python3.9/site-packages (from matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (20.9) 2026-02-17T10:04:14.601 INFO:teuthology.orchestra.run.trial103.stdout:Collecting zipp>=3.1.0 2026-02-17T10:04:14.605 INFO:teuthology.orchestra.run.trial103.stdout: Downloading zipp-3.23.0-py3-none-any.whl (10 kB) 2026-02-17T10:04:14.616 INFO:teuthology.orchestra.run.trial103.stdout:Requirement already satisfied: six>=1.5 in /usr/lib/python3.9/site-packages (from python-dateutil>=2.7->matplotlib->-r /home/ubuntu/cephtest/cbt/requirements.txt (line 3)) (1.15.0) 2026-02-17T10:04:14.748 INFO:teuthology.orchestra.run.trial103.stdout:Installing collected packages: zipp, pillow, kiwisolver, importlib-resources, fonttools, cycler, contourpy, matplotlib 2026-02-17T10:04:15.791 INFO:teuthology.orchestra.run.trial103.stdout:Successfully installed contourpy-1.3.0 cycler-0.12.1 fonttools-4.60.2 importlib-resources-6.5.2 kiwisolver-1.4.7 matplotlib-3.9.4 pillow-11.3.0 zipp-3.23.0 2026-02-17T10:04:15.902 INFO:tasks.cbt:benchmark: radosbench 2026-02-17T10:04:15.902 DEBUG:teuthology.orchestra.run.trial103:> sudo /sbin/sysctl -q -w kernel.perf_event_paranoid=0 2026-02-17T10:04:15.929 DEBUG:teuthology.orchestra.run.trial103:> /home/ubuntu/cephtest/cbt/cbt.py -a /home/ubuntu/cephtest/archive/cbt /home/ubuntu/cephtest/archive/cbt/cbt_config.yaml 2026-02-17T10:04:16.042 INFO:teuthology.orchestra.run.trial103.stderr:10:04:16 - DEBUG - cbt - Settings.cluster: 2026-02-17T10:04:16.042 INFO:teuthology.orchestra.run.trial103.stderr: {'archive_dir': '/home/ubuntu/cephtest/archive/cbt', 2026-02-17T10:04:16.042 INFO:teuthology.orchestra.run.trial103.stderr: 'clients': ['ubuntu@trial103.front.sepia.ceph.com'], 2026-02-17T10:04:16.042 INFO:teuthology.orchestra.run.trial103.stderr: 'conf_file': '/etc/ceph/ceph.conf', 2026-02-17T10:04:16.043 INFO:teuthology.orchestra.run.trial103.stderr: 'head': 'ubuntu@trial103.front.sepia.ceph.com', 2026-02-17T10:04:16.043 INFO:teuthology.orchestra.run.trial103.stderr: 'is_teuthology': True, 2026-02-17T10:04:16.043 INFO:teuthology.orchestra.run.trial103.stderr: 'iterations': 1, 2026-02-17T10:04:16.043 INFO:teuthology.orchestra.run.trial103.stderr: 'mons': ['ubuntu@trial103.front.sepia.ceph.com'], 2026-02-17T10:04:16.043 INFO:teuthology.orchestra.run.trial103.stderr: 'osds': ['ubuntu@trial103.front.sepia.ceph.com'], 2026-02-17T10:04:16.043 INFO:teuthology.orchestra.run.trial103.stderr: 'osds_per_node': 3, 2026-02-17T10:04:16.043 INFO:teuthology.orchestra.run.trial103.stderr: 'pid_dir': '/var/run/ceph', 2026-02-17T10:04:16.043 INFO:teuthology.orchestra.run.trial103.stderr: 'pool_profiles': {'replicated': {'pg_size': 256, 2026-02-17T10:04:16.043 INFO:teuthology.orchestra.run.trial103.stderr: 'pgp_size': 256, 2026-02-17T10:04:16.043 INFO:teuthology.orchestra.run.trial103.stderr: 'replication': 'replicated'}}, 2026-02-17T10:04:16.043 INFO:teuthology.orchestra.run.trial103.stderr: 'rebuild_every_test': False, 2026-02-17T10:04:16.044 INFO:teuthology.orchestra.run.trial103.stderr: 'rgws': {'ubuntu@trial103.front.sepia.ceph.com': None}, 2026-02-17T10:04:16.044 INFO:teuthology.orchestra.run.trial103.stderr: 'tmp_dir': '/tmp/cbt', 2026-02-17T10:04:16.044 INFO:teuthology.orchestra.run.trial103.stderr: 'use_existing': True, 2026-02-17T10:04:16.044 INFO:teuthology.orchestra.run.trial103.stderr: 'user': 'ubuntu'} 2026-02-17T10:04:16.044 INFO:teuthology.orchestra.run.trial103.stderr:10:04:16 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3 2026-02-17T10:04:16.044 INFO:teuthology.orchestra.run.trial103.stderr:10:04:16 - INFO - cbt - Pausing for 60s for idle monitoring. 2026-02-17T10:04:16.044 INFO:teuthology.orchestra.run.trial103.stderr:10:04:16 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/collectl 2026-02-17T10:04:16.209 INFO:teuthology.orchestra.run.trial103.stderr:10:04:16 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-17T10:04:16.211 INFO:teuthology.orchestra.run.trial103.stderr:10:04:16 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/perf 2026-02-17T10:04:16.380 INFO:teuthology.orchestra.run.trial103.stderr:10:04:16 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-17T10:04:16.381 INFO:teuthology.orchestra.run.trial103.stderr:10:04:16 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/perf/perf_stat.${pid} & done 2026-02-17T10:05:16.408 INFO:teuthology.orchestra.run.trial103.stderr:10:05:16 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-17T10:05:16.595 INFO:teuthology.orchestra.run.trial103.stderr:10:05:16 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-17T10:05:16.781 INFO:teuthology.orchestra.run.trial103.stderr:10:05:16 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/perf/perf.data 2026-02-17T10:05:16.783 INFO:teuthology.orchestra.run.trial103.stderr:10:05:16 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/idle_monitoring/perf/perf_stat.* 2026-02-17T10:05:16.785 INFO:teuthology.orchestra.run.trial103.stderr:10:05:16 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/* 2026-02-17T10:05:16.970 INFO:teuthology.orchestra.run.trial103.stderr:10:05:16 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial103.front.sepia.ceph.com -r /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3 2026-02-17T10:05:17.133 INFO:teuthology.orchestra.run.trial103.stderr:10:05:17 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo killall -9 rados 2026-02-17T10:05:17.312 INFO:teuthology.orchestra.run.trial103.stderr:10:05:17 - INFO - cbt - Results dir: /home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3 2026-02-17T10:05:17.312 INFO:teuthology.orchestra.run.trial103.stderr:10:05:17 - INFO - cbt - Skipping existing test in /home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3. 2026-02-17T10:05:17.313 INFO:teuthology.orchestra.run.trial103.stderr:10:05:17 - INFO - cbt - Running benchmark /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004 2026-02-17T10:05:17.313 INFO:teuthology.orchestra.run.trial103.stderr:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3 2026-02-17T10:05:17.313 INFO:teuthology.orchestra.run.trial103.stderr:{'concurrent_ops': 4, 'concurrent_procs': 2, 'pool_profile': 'replicated', 'readmode': 'rand', 'time': 60, 'write_only': False, 'op_size': 4096, 'pool_monitoring_list': 'collectl', 'run_monitoring_list': 'collectl', 'benchmark': 'radosbench', 'iteration': 0} == iteration 0 == 2026-02-17T10:05:17.313 INFO:teuthology.orchestra.run.trial103.stderr:10:05:17 - INFO - cbt - Setting OSD Read Ahead to: 0 2026-02-17T10:05:17.313 INFO:teuthology.orchestra.run.trial103.stderr:10:05:17 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com find /dev/disk/by-partlabel/osd-device-*data -exec readlink {} \; | cut -d"/" -f 3 | sed "s/[0-9]$//" | xargs -I{} sudo sh -c "echo 0 > /sys/block/'{}'/queue/read_ahead_kb" 2026-02-17T10:05:17.314 INFO:teuthology.orchestra.run.trial103.stderr:10:05:17 - DEBUG - cbt - Cleaning existing temporary run directory: /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004 2026-02-17T10:05:17.314 INFO:teuthology.orchestra.run.trial103.stderr:10:05:17 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo rm -rf /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004 2026-02-17T10:05:17.495 INFO:teuthology.orchestra.run.trial103.stderr:10:05:17 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/collectl 2026-02-17T10:05:17.656 INFO:teuthology.orchestra.run.trial103.stderr:10:05:17 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-17T10:05:17.658 INFO:teuthology.orchestra.run.trial103.stderr:10:05:17 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/perf 2026-02-17T10:05:17.823 INFO:teuthology.orchestra.run.trial103.stderr:10:05:17 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-17T10:05:17.824 INFO:teuthology.orchestra.run.trial103.stderr:10:05:17 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/perf/perf_stat.${pid} & done 2026-02-17T10:05:17.824 INFO:teuthology.orchestra.run.trial103.stderr:10:05:17 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool delete rados-bench-cbt rados-bench-cbt --yes-i-really-really-mean-it 2026-02-17T10:05:18.174 INFO:teuthology.orchestra.run.trial103.stderr:10:05:18 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool create rados-bench-cbt 256 256 2026-02-17T10:05:18.724 INFO:teuthology.orchestra.run.trial103.stderr:10:05:18 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf osd pool application enable rados-bench-cbt radosbench 2026-02-17T10:05:19.728 INFO:teuthology.orchestra.run.trial103.stderr:10:05:19 - INFO - cbt - Checking Health after pool creation. 2026-02-17T10:05:24.734 INFO:teuthology.orchestra.run.trial103.stderr:10:05:24 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com /usr/bin/ceph -c /etc/ceph/ceph.conf health 2026-02-17T10:05:25.089 INFO:teuthology.orchestra.run.trial103.stderr:10:05:25 - INFO - cbt - Final Pool Health Check. 2026-02-17T10:05:30.095 INFO:teuthology.orchestra.run.trial103.stderr:10:05:30 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com /usr/bin/ceph -c /etc/ceph/ceph.conf health 2026-02-17T10:05:30.466 INFO:teuthology.orchestra.run.trial103.stderr:10:05:30 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-17T10:05:30.645 INFO:teuthology.orchestra.run.trial103.stderr:10:05:30 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-17T10:05:30.835 INFO:teuthology.orchestra.run.trial103.stderr:10:05:30 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/perf/perf.data 2026-02-17T10:05:30.837 INFO:teuthology.orchestra.run.trial103.stderr:10:05:30 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/pool_monitoring/perf/perf_stat.* 2026-02-17T10:05:30.839 INFO:teuthology.orchestra.run.trial103.stderr:10:05:30 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sync 2026-02-17T10:05:31.028 INFO:teuthology.orchestra.run.trial103.stderr:10:05:31 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com echo 3 | sudo tee /proc/sys/vm/drop_caches 2026-02-17T10:05:32.037 INFO:teuthology.orchestra.run.trial103.stderr:10:05:32 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -v 2026-02-17T10:05:32.244 INFO:teuthology.orchestra.run.trial103.stderr:10:05:32 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write 2026-02-17T10:05:32.410 INFO:teuthology.orchestra.run.trial103.stderr:10:05:32 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf daemon osd.0 config show > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/ceph_settings.out 2026-02-17T10:05:32.691 INFO:teuthology.orchestra.run.trial103.stderr:10:05:32 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/collectl 2026-02-17T10:05:32.854 INFO:teuthology.orchestra.run.trial103.stderr:10:05:32 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-17T10:05:32.855 INFO:teuthology.orchestra.run.trial103.stderr:10:05:32 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/perf 2026-02-17T10:05:33.021 INFO:teuthology.orchestra.run.trial103.stderr:10:05:33 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-17T10:05:33.021 INFO:teuthology.orchestra.run.trial103.stderr:10:05:33 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/perf/perf_stat.${pid} & done 2026-02-17T10:05:33.023 INFO:teuthology.orchestra.run.trial103.stderr:10:05:33 - INFO - cbt - Running radosbench write test. 2026-02-17T10:05:33.023 INFO:teuthology.orchestra.run.trial103.stderr:10:05:33 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench -b 4096 60 write --concurrent-ios 4 --run-name `hostname -f`-0 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/objecter.0.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/output.0 2026-02-17T10:05:33.024 INFO:teuthology.orchestra.run.trial103.stderr:10:05:33 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench -b 4096 60 write --concurrent-ios 4 --run-name `hostname -f`-1 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/objecter.1.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/output.1 2026-02-17T10:06:34.249 INFO:teuthology.orchestra.run.trial103.stderr:10:06:34 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-17T10:06:34.427 INFO:teuthology.orchestra.run.trial103.stderr:10:06:34 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-17T10:06:34.612 INFO:teuthology.orchestra.run.trial103.stderr:10:06:34 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/perf/perf.data 2026-02-17T10:06:34.614 INFO:teuthology.orchestra.run.trial103.stderr:10:06:34 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/perf/perf_stat.* 2026-02-17T10:06:34.616 INFO:teuthology.orchestra.run.trial103.stderr:10:06:34 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com find /var/run/ceph/ceph-osd*.asok -maxdepth 1 -exec sudo /usr/bin/ceph --admin-daemon {} dump_historic_ops \; > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/historic_ops.out 2026-02-17T10:06:34.981 INFO:teuthology.orchestra.run.trial103.stderr:10:06:34 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/* 2026-02-17T10:06:35.160 INFO:teuthology.orchestra.run.trial103.stderr:10:06:35 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial103.front.sepia.ceph.com -r /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/write/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3/write 2026-02-17T10:06:35.324 INFO:teuthology.orchestra.run.trial103.stderr:10:06:35 - INFO - cbt - Convert results to json format. 2026-02-17T10:06:35.325 INFO:teuthology.orchestra.run.trial103.stderr:10:06:35 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sync 2026-02-17T10:06:35.612 INFO:teuthology.orchestra.run.trial103.stderr:10:06:35 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com echo 3 | sudo tee /proc/sys/vm/drop_caches 2026-02-17T10:06:36.120 INFO:teuthology.orchestra.run.trial103.stderr:10:06:36 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -v 2026-02-17T10:06:36.332 INFO:teuthology.orchestra.run.trial103.stderr:10:06:36 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/rand 2026-02-17T10:06:36.496 INFO:teuthology.orchestra.run.trial103.stderr:10:06:36 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo /usr/bin/ceph -c /etc/ceph/ceph.conf daemon osd.0 config show > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/rand/ceph_settings.out 2026-02-17T10:06:36.784 INFO:teuthology.orchestra.run.trial103.stderr:10:06:36 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/rand/collectl 2026-02-17T10:06:36.952 INFO:teuthology.orchestra.run.trial103.stderr:10:06:36 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com collectl -s+mYZ -i 1:10 -F0 -f /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/rand/collectl --rawdskfilt \"+cciss/c\d+d\d+ |hd[ab] | sd[a-z]+ |dm-\d+ |xvd[a-z] |fio[a-z]+ | vd[a-z]+ |emcpower[a-z]+ |psv\d+ |nvme[0-9]n[0-9]+p[0-9]+ \" 2026-02-17T10:06:36.954 INFO:teuthology.orchestra.run.trial103.stderr:10:06:36 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com mkdir -p -m0755 -- /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/rand/perf 2026-02-17T10:06:37.117 INFO:teuthology.orchestra.run.trial103.stderr:10:06:37 - DEBUG - cbt - PerfMonitoring: remote_node 2026-02-17T10:06:37.117 INFO:teuthology.orchestra.run.trial103.stderr:10:06:37 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com for pid in `cat /var/run/ceph/crimson-osd.*.pid`; do perf stat -p ${pid} -o /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/rand/perf/perf_stat.${pid} & done 2026-02-17T10:06:37.118 INFO:teuthology.orchestra.run.trial103.stderr:10:06:37 - INFO - cbt - Running radosbench rand test. 2026-02-17T10:06:37.118 INFO:teuthology.orchestra.run.trial103.stderr:10:06:37 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench 60 rand --concurrent-ios 4 --run-name `hostname -f`-0 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/rand/objecter.0.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/rand/output.0 2026-02-17T10:06:37.120 INFO:teuthology.orchestra.run.trial103.stderr:10:06:37 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com /usr/bin/rados -c /etc/ceph/ceph.conf -p rados-bench-cbt bench 60 rand --concurrent-ios 4 --run-name `hostname -f`-1 --no-cleanup 2> /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/rand/objecter.1.log > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/rand/output.1 2026-02-17T10:07:37.330 INFO:teuthology.orchestra.run.trial103.stderr:10:07:37 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com pkill -SIGINT -f collectl 2026-02-17T10:07:37.505 INFO:teuthology.orchestra.run.trial103.stderr:10:07:37 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo pkill -SIGINT -f perf\ 2026-02-17T10:07:37.688 INFO:teuthology.orchestra.run.trial103.stderr:10:07:37 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/rand/perf/perf.data 2026-02-17T10:07:37.689 INFO:teuthology.orchestra.run.trial103.stderr:10:07:37 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo chown ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/rand/perf/perf_stat.* 2026-02-17T10:07:37.691 INFO:teuthology.orchestra.run.trial103.stderr:10:07:37 - DEBUG - cbt - CheckedPopen continue_if_error=True, shell=False args=pdsh -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com find /var/run/ceph/ceph-osd*.asok -maxdepth 1 -exec sudo /usr/bin/ceph --admin-daemon {} dump_historic_ops \; > /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/rand/historic_ops.out 2026-02-17T10:07:38.052 INFO:teuthology.orchestra.run.trial103.stderr:10:07:38 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=pdsh -S -f 1 -R ssh -w ubuntu@trial103.front.sepia.ceph.com sudo chown -R ubuntu.ubuntu /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/rand/* 2026-02-17T10:07:38.227 INFO:teuthology.orchestra.run.trial103.stderr:10:07:38 - DEBUG - cbt - CheckedPopen continue_if_error=False, shell=False args=rpdcp -f 10 -R ssh -w ubuntu@trial103.front.sepia.ceph.com -r /tmp/cbt/00000000/Radosbench/osd_ra-00000000/op_size-00004096/concurrent_ops-00000004/rand/* /home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3/rand 2026-02-17T10:07:38.401 INFO:teuthology.orchestra.run.trial103.stderr:10:07:38 - INFO - cbt - Convert results to json format. 2026-02-17T10:07:38.412 DEBUG:teuthology.run_tasks:Unwinding manager cbt 2026-02-17T10:07:38.427 DEBUG:teuthology.orchestra.run.trial103:> rm --one-file-system -rf -- /home/ubuntu/cephtest/cbt 2026-02-17T10:07:38.456 INFO:tasks.cbt_performance:Collecting CBT performance data config 2026-02-17T10:07:38.456 INFO:tasks.cbt_performance:reading cbt preferences 2026-02-17T10:07:38.456 DEBUG:teuthology.orchestra.run.trial103:> find /home/ubuntu/cephtest -name 'json_output.*' 2026-02-17T10:07:38.515 INFO:teuthology.orchestra.run.trial103.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3/write/json_output.1.trial103.front.sepia.ceph.com 2026-02-17T10:07:38.515 INFO:teuthology.orchestra.run.trial103.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3/write/json_output.0.trial103.front.sepia.ceph.com 2026-02-17T10:07:38.515 INFO:teuthology.orchestra.run.trial103.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3/rand/json_output.1.trial103.front.sepia.ceph.com 2026-02-17T10:07:38.516 INFO:teuthology.orchestra.run.trial103.stdout:/home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3/rand/json_output.0.trial103.front.sepia.ceph.com 2026-02-17T10:07:38.516 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:07:38.517 DEBUG:teuthology.orchestra.run.trial103:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3/write/json_output.1.trial103.front.sepia.ceph.com of=/dev/stdout 2026-02-17T10:07:38.573 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-17T10:07:38.573 DEBUG:teuthology.orchestra.run.trial103:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3/write -name 'perf_stat.*' 2026-02-17T10:07:38.629 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-17T10:07:38.630 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-17T10:07:38.630 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:07:38.630 DEBUG:teuthology.orchestra.run.trial103:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3/write/json_output.0.trial103.front.sepia.ceph.com of=/dev/stdout 2026-02-17T10:07:38.686 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-17T10:07:38.687 DEBUG:teuthology.orchestra.run.trial103:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3/write -name 'perf_stat.*' 2026-02-17T10:07:38.743 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-17T10:07:38.743 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-17T10:07:38.744 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:07:38.744 DEBUG:teuthology.orchestra.run.trial103:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3/rand/json_output.1.trial103.front.sepia.ceph.com of=/dev/stdout 2026-02-17T10:07:38.800 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-17T10:07:38.801 DEBUG:teuthology.orchestra.run.trial103:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3/rand -name 'perf_stat.*' 2026-02-17T10:07:38.857 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-17T10:07:38.857 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-17T10:07:38.857 DEBUG:teuthology.orchestra.run.trial103:> set -ex 2026-02-17T10:07:38.857 DEBUG:teuthology.orchestra.run.trial103:> dd if=/home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3/rand/json_output.0.trial103.front.sepia.ceph.com of=/dev/stdout 2026-02-17T10:07:38.913 INFO:tasks.cbt_performance:reading total cpu cycles 2026-02-17T10:07:38.914 DEBUG:teuthology.orchestra.run.trial103:> find /home/ubuntu/cephtest/archive/cbt/results/00000000/id-a0b9bdc3/rand -name 'perf_stat.*' 2026-02-17T10:07:38.970 INFO:tasks.cbt_performance:cpu_cycles_paths: [''] 2026-02-17T10:07:38.970 INFO:tasks.cbt_performance:total cpu cycles: 0 2026-02-17T10:07:38.975 WARNING:tasks.cbt_performance:Could not send performance data to http://mira118.front.sepia.ceph.com:4000/cbt_performance: HTTPConnectionPool(host='mira118.front.sepia.ceph.com', port=4000): Max retries exceeded with url: /cbt_performance (Caused by NewConnectionError(': Failed to establish a new connection: [Errno -2] Name or service not known')) 2026-02-17T10:07:38.975 WARNING:tasks.cbt_performance:Continuing without performance data collection 2026-02-17T10:07:38.975 DEBUG:teuthology.run_tasks:Unwinding manager ssh_keys 2026-02-17T10:07:38.991 INFO:teuthology.task.ssh_keys:Cleaning up SSH keys 2026-02-17T10:07:38.991 INFO:teuthology.task.ssh_keys:cleaning up keys added for testing 2026-02-17T10:07:38.991 INFO:teuthology.task.ssh_keys: cleaning up keys for user ubuntu on trial103.front.sepia.ceph.com 2026-02-17T10:07:38.991 DEBUG:teuthology.orchestra.run.trial103:> rm -- /home/ubuntu/.ssh/id_rsa 2026-02-17T10:07:39.027 DEBUG:teuthology.orchestra.run.trial103:> rm -- /home/ubuntu/.ssh/id_rsa.pub 2026-02-17T10:07:39.083 DEBUG:teuthology.orchestra.run.trial103:> stat -c '"%a"' /home/ubuntu/.ssh/authorized_keys 2026-02-17T10:07:39.140 INFO:teuthology.orchestra.run.trial103.stdout:"600" 2026-02-17T10:07:39.140 DEBUG:teuthology.orchestra.run.trial103:> mv -- /home/ubuntu/.ssh/authorized_keys_2026-02-17_10:04:08:799201 /home/ubuntu/.ssh/authorized_keys 2026-02-17T10:07:39.197 DEBUG:teuthology.orchestra.run.trial103:> chmod 600 /home/ubuntu/.ssh/authorized_keys 2026-02-17T10:07:39.253 DEBUG:teuthology.orchestra.run.trial103:> rm /home/ubuntu/.ssh/config 2026-02-17T10:07:39.310 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-17T10:07:39.327 DEBUG:teuthology.orchestra.run.trial103:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-17T10:07:39.427 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:07:39.421+0000 7fe4b725a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:07:39.428 INFO:teuthology.orchestra.run.trial103.stderr:2026-02-17T10:07:39.426+0000 7fe4b725a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-17T10:07:39.554 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-17T10:07:39.554 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-17T10:07:39.555 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-17T10:07:39.555 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T10:07:39.634 INFO:tasks.ceph.osd.0:Stopped 2026-02-17T10:07:39.634 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-17T10:07:39.634 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T10:07:39.725 INFO:tasks.ceph.osd.1:Stopped 2026-02-17T10:07:39.725 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-17T10:07:39.725 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T10:07:39.817 INFO:tasks.ceph.osd.2:Stopped 2026-02-17T10:07:39.817 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-17T10:07:39.818 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-17T10:07:39.818 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T10:07:39.853 INFO:tasks.ceph.mgr.x:Stopped 2026-02-17T10:07:39.853 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-17T10:07:39.853 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-17T10:07:39.854 INFO:teuthology.orchestra.run:waiting for 300 2026-02-17T10:07:39.905 INFO:tasks.ceph.mon.a:Stopped 2026-02-17T10:07:39.905 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-17T10:07:39.905 DEBUG:teuthology.orchestra.run.trial103:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | grep -E -v '\(PG_' | grep -E -v '\(OSD_' | grep -E -v '\(OBJECT_' | grep -E -v 'overall HEALTH' | head -n 1 2026-02-17T10:07:39.971 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial103.front.sepia.ceph.com 2026-02-17T10:07:39.971 DEBUG:teuthology.orchestra.run.trial103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-17T10:07:40.041 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial103.front.sepia.ceph.com 2026-02-17T10:07:40.042 DEBUG:teuthology.orchestra.run.trial103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-17T10:07:40.111 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial103.front.sepia.ceph.com 2026-02-17T10:07:40.111 DEBUG:teuthology.orchestra.run.trial103:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-17T10:07:40.185 INFO:tasks.ceph:Archiving mon data... 2026-02-17T10:07:40.187 DEBUG:teuthology.misc:Transferring archived files from trial103:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/matan-2026-02-17_08:51:22-crimson-rados-wip-sjust-seastore-2026-02-16-1771285632-distro-debug-trial/52100/data/mon.a.tgz 2026-02-17T10:07:40.187 DEBUG:teuthology.orchestra.run.trial103:> mktemp 2026-02-17T10:07:40.242 INFO:teuthology.orchestra.run.trial103.stdout:/tmp/tmp.Vcx9rpPFWB 2026-02-17T10:07:40.243 DEBUG:teuthology.orchestra.run.trial103:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.Vcx9rpPFWB 2026-02-17T10:07:40.335 DEBUG:teuthology.orchestra.run.trial103:> sudo chmod 0666 /tmp/tmp.Vcx9rpPFWB 2026-02-17T10:07:40.416 DEBUG:teuthology.orchestra.remote:trial103:/tmp/tmp.Vcx9rpPFWB is 205KB 2026-02-17T10:07:40.479 DEBUG:teuthology.orchestra.run.trial103:> rm -fr /tmp/tmp.Vcx9rpPFWB 2026-02-17T10:07:40.496 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-17T10:07:40.496 DEBUG:teuthology.orchestra.run.trial103:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-17T10:07:40.617 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-17T10:07:40.618 INFO:tasks.ceph:Archiving crash dumps... 2026-02-17T10:07:40.620 DEBUG:teuthology.misc:Transferring archived files from trial103:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/matan-2026-02-17_08:51:22-crimson-rados-wip-sjust-seastore-2026-02-16-1771285632-distro-debug-trial/52100/remote/trial103/crash 2026-02-17T10:07:40.621 DEBUG:teuthology.orchestra.run.trial103:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-17T10:07:40.682 INFO:tasks.ceph:Compressing logs... 2026-02-17T10:07:40.683 DEBUG:teuthology.orchestra.run.trial103:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T10:07:40.749 INFO:teuthology.orchestra.run.trial103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40324.log 2026-02-17T10:07:40.761 INFO:teuthology.orchestra.run.trial103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41212.log 2026-02-17T10:07:40.761 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.40324.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-17T10:07:40.761 INFO:teuthology.orchestra.run.trial103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40324.log.gz 2026-02-17T10:07:40.761 INFO:teuthology.orchestra.run.trial103.stderr:gzip -5/var/log/ceph/ceph-client.admin.41212.log: --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-17T10:07:40.761 INFO:teuthology.orchestra.run.trial103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41212.log.gz 2026-02-17T10:07:40.761 INFO:teuthology.orchestra.run.trial103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40720.log 2026-02-17T10:07:40.762 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-osd.2.log: /var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45011.log 2026-02-17T10:07:40.762 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.40720.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-17T10:07:40.762 INFO:teuthology.orchestra.run.trial103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40720.log.gz 2026-02-17T10:07:40.762 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.45011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40277.log 2026-02-17T10:07:40.762 INFO:teuthology.orchestra.run.trial103.stderr: 66.4% -- replaced with /var/log/ceph/ceph-client.admin.45011.log.gz 2026-02-17T10:07:40.762 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40093.log 2026-02-17T10:07:40.762 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.40277.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44309.log 2026-02-17T10:07:40.762 INFO:teuthology.orchestra.run.trial103.stderr: 91.2% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-17T10:07:40.762 INFO:teuthology.orchestra.run.trial103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40277.log.gz 2026-02-17T10:07:40.763 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.40093.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45708.log 2026-02-17T10:07:40.763 INFO:teuthology.orchestra.run.trial103.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40093.log.gz 2026-02-17T10:07:40.763 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.44309.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41211.log 2026-02-17T10:07:40.763 INFO:teuthology.orchestra.run.trial103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44309.log.gz 2026-02-17T10:07:40.763 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.45708.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40230.log 2026-02-17T10:07:40.763 INFO:teuthology.orchestra.run.trial103.stderr: 66.7% -- replaced with /var/log/ceph/ceph-client.admin.45708.log.gz 2026-02-17T10:07:40.763 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.41211.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40573.log 2026-02-17T10:07:40.763 INFO:teuthology.orchestra.run.trial103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41211.log.gz 2026-02-17T10:07:40.763 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.40230.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44405.log 2026-02-17T10:07:40.763 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.40573.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41357.log 2026-02-17T10:07:40.763 INFO:teuthology.orchestra.run.trial103.stderr: 50.0% 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40573.log.gz -- replaced with /var/log/ceph/ceph-client.admin.40230.log.gz 2026-02-17T10:07:40.763 INFO:teuthology.orchestra.run.trial103.stderr: 2026-02-17T10:07:40.764 INFO:teuthology.orchestra.run.trial103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40802.log 2026-02-17T10:07:40.764 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.44405.log: /var/log/ceph/ceph-client.admin.41357.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44405.log.gz 2026-02-17T10:07:40.764 INFO:teuthology.orchestra.run.trial103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41210.log 2026-02-17T10:07:40.764 INFO:teuthology.orchestra.run.trial103.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41357.log.gz 2026-02-17T10:07:40.764 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.40802.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40872.log 2026-02-17T10:07:40.764 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.41210.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40802.log.gz 2026-02-17T10:07:40.764 INFO:teuthology.orchestra.run.trial103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39951.log 2026-02-17T10:07:40.764 INFO:teuthology.orchestra.run.trial103.stderr: 48.4% -- replaced with /var/log/ceph/ceph-client.admin.41210.log.gz 2026-02-17T10:07:40.764 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.40872.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44459.log 2026-02-17T10:07:40.764 INFO:teuthology.orchestra.run.trial103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40872.log.gz 2026-02-17T10:07:40.764 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.39951.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41501.log 2026-02-17T10:07:40.764 INFO:teuthology.orchestra.run.trial103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39951.log.gz 2026-02-17T10:07:40.765 INFO:teuthology.orchestra.run.trial103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41011.log 2026-02-17T10:07:40.765 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.44459.log: /var/log/ceph/ceph-client.admin.41501.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.38664.log 2026-02-17T10:07:40.765 INFO:teuthology.orchestra.run.trial103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44459.log.gz 2026-02-17T10:07:40.765 INFO:teuthology.orchestra.run.trial103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41501.log.gz 2026-02-17T10:07:40.765 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.41011.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.46387.log 2026-02-17T10:07:40.765 INFO:teuthology.orchestra.run.trial103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41011.log.gz 2026-02-17T10:07:40.765 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph.tmp-client.admin.38664.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-17T10:07:40.765 INFO:teuthology.orchestra.run.trial103.stderr: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.38664.log.gz 2026-02-17T10:07:40.765 INFO:teuthology.orchestra.run.trial103.stderr:gzip/var/log/ceph/ceph-client.admin.46387.log: -5 --verbose -- /var/log/ceph/ceph-client.admin.41060.log 2026-02-17T10:07:40.765 INFO:teuthology.orchestra.run.trial103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.46387.log.gz 2026-02-17T10:07:40.765 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-17T10:07:40.765 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.41060.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41550.log 2026-02-17T10:07:40.766 INFO:teuthology.orchestra.run.trial103.stderr: 87.6% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-17T10:07:40.766 INFO:teuthology.orchestra.run.trial103.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41060.log.gz 2026-02-17T10:07:40.766 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40962.log 2026-02-17T10:07:40.766 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.41550.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44367.log 2026-02-17T10:07:40.766 INFO:teuthology.orchestra.run.trial103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41550.log.gz 2026-02-17T10:07:40.766 INFO:teuthology.orchestra.run.trial103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40543.log 2026-02-17T10:07:40.766 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.40962.log: /var/log/ceph/ceph-client.admin.44367.log: 48.4%gzip -- replaced with /var/log/ceph/ceph-client.admin.40962.log.gz -5 2026-02-17T10:07:40.766 INFO:teuthology.orchestra.run.trial103.stderr: --verbose -- /var/log/ceph/ceph-client.admin.41359.log 2026-02-17T10:07:40.766 INFO:teuthology.orchestra.run.trial103.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.44367.log.gz 2026-02-17T10:07:40.766 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.40543.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40543.log.gz 2026-02-17T10:07:40.766 INFO:teuthology.orchestra.run.trial103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41452.log 2026-02-17T10:07:40.766 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.41359.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45013.log 2026-02-17T10:07:40.767 INFO:teuthology.orchestra.run.trial103.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41359.log.gz 2026-02-17T10:07:40.767 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.41452.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-17T10:07:40.767 INFO:teuthology.orchestra.run.trial103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41452.log.gz 2026-02-17T10:07:40.767 INFO:teuthology.orchestra.run.trial103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41109.log 2026-02-17T10:07:40.767 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.45013.log: /var/log/ceph/ceph-mgr.x.log: 66.4% -- replaced with /var/log/ceph/ceph-client.admin.45013.log.gz 2026-02-17T10:07:40.767 INFO:teuthology.orchestra.run.trial103.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40897.log 2026-02-17T10:07:40.767 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.41109.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-17T10:07:40.767 INFO:teuthology.orchestra.run.trial103.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41109.log.gz 2026-02-17T10:07:40.767 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.40897.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.44253.log 2026-02-17T10:07:40.767 INFO:teuthology.orchestra.run.trial103.stderr: 65.6% -- replaced with /var/log/ceph/ceph-client.admin.40897.log.gz 2026-02-17T10:07:40.767 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41358.log 2026-02-17T10:07:40.767 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.44253.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40046.log 2026-02-17T10:07:40.767 INFO:teuthology.orchestra.run.trial103.stderr: 86.2% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-17T10:07:40.768 INFO:teuthology.orchestra.run.trial103.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.44253.log.gz 2026-02-17T10:07:40.768 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.41358.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.45710.log 2026-02-17T10:07:40.768 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.40046.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40046.log.gz 2026-02-17T10:07:40.768 INFO:teuthology.orchestra.run.trial103.stderr:/var/log/ceph/ceph-client.admin.45710.log: 50.0% 66.7% -- replaced with /var/log/ceph/ceph-client.admin.41358.log.gz -- replaced with /var/log/ceph/ceph-client.admin.45710.log.gz 2026-02-17T10:07:40.768 INFO:teuthology.orchestra.run.trial103.stderr: 2026-02-17T10:07:40.769 INFO:teuthology.orchestra.run.trial103.stderr: 92.2% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-17T10:07:46.994 INFO:teuthology.orchestra.run.trial103.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-17T10:07:47.126 INFO:teuthology.orchestra.run.trial103.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-17T10:07:47.712 INFO:teuthology.orchestra.run.trial103.stderr: 92.7% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-17T10:07:47.713 INFO:teuthology.orchestra.run.trial103.stderr: 2026-02-17T10:07:47.713 INFO:teuthology.orchestra.run.trial103.stderr:real 0m6.972s 2026-02-17T10:07:47.713 INFO:teuthology.orchestra.run.trial103.stderr:user 0m18.275s 2026-02-17T10:07:47.713 INFO:teuthology.orchestra.run.trial103.stderr:sys 0m1.280s 2026-02-17T10:07:47.714 INFO:tasks.ceph:Archiving logs... 2026-02-17T10:07:47.715 DEBUG:teuthology.misc:Transferring archived files from trial103:/var/log/ceph to /home/teuthworker/mnt/teuthology/matan-2026-02-17_08:51:22-crimson-rados-wip-sjust-seastore-2026-02-16-1771285632-distro-debug-trial/52100/remote/trial103/log 2026-02-17T10:07:47.717 DEBUG:teuthology.orchestra.run.trial103:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-17T10:07:49.353 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-17T10:07:49.421 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-17T10:07:49.433 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-17T10:07:49.433 DEBUG:teuthology.orchestra.run.trial103:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-17T10:07:49.458 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-17T10:07:49.469 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-17T10:07:49.469 DEBUG:teuthology.orchestra.run.trial103:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-17T10:07:49.513 INFO:teuthology.orchestra.run.trial103.stderr:bash: line 1: ntpq: command not found 2026-02-17T10:07:49.536 INFO:teuthology.orchestra.run.trial103.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-17T10:07:49.537 INFO:teuthology.orchestra.run.trial103.stdout:=============================================================================== 2026-02-17T10:07:49.537 INFO:teuthology.orchestra.run.trial103.stdout:^- lax1.us.ntp.li 2 6 377 24 -952us[-1126us] +/- 34ms 2026-02-17T10:07:49.537 INFO:teuthology.orchestra.run.trial103.stdout:^? 198-12-95-197-host.coloc> 2 7 340 480 +337us[ -293us] +/- 70ms 2026-02-17T10:07:49.537 INFO:teuthology.orchestra.run.trial103.stdout:^- 23-93-251-54.dedicated.s> 1 6 377 25 -236us[ -409us] +/- 136ms 2026-02-17T10:07:49.537 INFO:teuthology.orchestra.run.trial103.stdout:^* time.cloudflare.com 3 6 377 23 -120us[ -293us] +/- 6708us 2026-02-17T10:07:49.539 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-17T10:07:49.551 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-17T10:07:49.552 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-17T10:07:49.567 DEBUG:teuthology.orchestra.run.trial103:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-17T10:07:49.640 DEBUG:teuthology.orchestra.run.trial103:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-17T10:07:49.666 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T10:07:49.666 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-17T10:07:49.682 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-17T10:07:49.698 INFO:teuthology.task.internal:Duration was 653.015996 seconds 2026-02-17T10:07:49.698 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-17T10:07:49.715 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-17T10:07:49.716 DEBUG:teuthology.orchestra.run.trial103:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-17T10:07:49.756 INFO:teuthology.orchestra.run.trial103.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-17T10:07:50.137 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-17T10:07:50.138 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial103.front.sepia.ceph.com 2026-02-17T10:07:50.138 DEBUG:teuthology.orchestra.run.trial103:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-17T10:07:50.197 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-17T10:07:50.197 DEBUG:teuthology.orchestra.run.trial103:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T10:07:50.301 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-17T10:07:50.301 DEBUG:teuthology.orchestra.run.trial103:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-17T10:07:50.324 INFO:teuthology.orchestra.run.trial103.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-17T10:07:50.325 INFO:teuthology.orchestra.run.trial103.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-17T10:07:50.326 INFO:teuthology.orchestra.run.trial103.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-17T10:07:50.326 INFO:teuthology.orchestra.run.trial103.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-17T10:07:50.326 INFO:teuthology.orchestra.run.trial103.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-17T10:07:50.335 INFO:teuthology.orchestra.run.trial103.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.5% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-17T10:07:50.337 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-17T10:07:50.355 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-17T10:07:50.355 DEBUG:teuthology.orchestra.run.trial103:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-17T10:07:50.402 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-17T10:07:50.420 DEBUG:teuthology.orchestra.run.trial103:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-17T10:07:50.469 INFO:teuthology.orchestra.run.trial103.stdout:kernel.core_pattern = core 2026-02-17T10:07:50.483 DEBUG:teuthology.orchestra.run.trial103:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-17T10:07:50.538 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-17T10:07:50.539 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-17T10:07:50.557 INFO:teuthology.task.internal:Transferring archived files... 2026-02-17T10:07:50.558 DEBUG:teuthology.misc:Transferring archived files from trial103:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/matan-2026-02-17_08:51:22-crimson-rados-wip-sjust-seastore-2026-02-16-1771285632-distro-debug-trial/52100/remote/trial103 2026-02-17T10:07:50.558 DEBUG:teuthology.orchestra.run.trial103:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-17T10:07:50.657 INFO:teuthology.task.internal:Removing archive directory... 2026-02-17T10:07:50.657 DEBUG:teuthology.orchestra.run.trial103:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-17T10:07:50.677 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-17T10:07:50.698 INFO:teuthology.task.internal:Not uploading archives. 2026-02-17T10:07:50.699 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-17T10:07:50.716 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-17T10:07:50.717 DEBUG:teuthology.orchestra.run.trial103:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-17T10:07:50.734 INFO:teuthology.orchestra.run.trial103.stdout: 791885 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 17 10:07 /home/ubuntu/cephtest 2026-02-17T10:07:50.735 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-17T10:07:50.753 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-17T10:07:50.792 INFO:teuthology.run:Summary data: description: crimson-rados/perf/{0-crimson_install clusters/{crimson-fixed-1} crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/rbm$/{crimson_seastore_rbm_2q} perf_ignorelist settings/optimized workloads/radosbench_4K_rand_read} duration: 653.0159962177277 flavor: debug owner: scheduled_matan@soko04.front.sepia.ceph.com success: true 2026-02-17T10:07:50.792 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-17T10:07:50.907 INFO:teuthology.run:pass