2026-02-25T09:06:41.119 INFO:root:teuthology version: 1.2.4.dev9+gfa17720d0 2026-02-25T09:06:41.133 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-25T09:06:41.210 INFO:teuthology.run:Config: archive_path: /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:58:25-crimson-rados-main-distro-debug-trial/69215 branch: main description: crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/cls} email: null first_in_suite: false flavor: debug job_id: '69215' kernel: branch: distro kdb: 1 sha1: distro last_in_suite: false machine_type: trial name: matan-2026-02-25_07:58:25-crimson-rados-main-distro-debug-trial no_nested_subset: false os_type: centos os_version: 9.stream overrides: admin_socket: branch: main ceph: conf: global: enable experimental unrecoverable data corrupting features: crimson ms client mode: crc ms cluster mode: crc ms mon client mode: crc ms mon cluster mode: crc ms mon service mode: crc ms service mode: crc mgr: debug mgr: 20 debug ms: 1 mon: debug mon: 20 debug ms: 1 debug paxos: 20 osd pool default crimson: true osd: crimson cpu num: 4 crimson osd obc lru size: 10 debug ms: 20 debug objclass: 20 debug osd: 20 debug seastore: 20 debug seastore backref: 20 debug seastore cache: 20 debug seastore cleaner: 20 debug seastore device: 20 debug seastore epm: 20 debug seastore fixedkv tree: 20 debug seastore journal: 20 debug seastore lba: 20 debug seastore odata: 20 debug seastore omap: 20 debug seastore onode: 20 debug seastore t: 20 debug seastore tm: 20 osd objectstore: seastore osd shutdown pgref assert: true seastore cachepin size pershard: 64M seastore cachepin type: 2Q seastore max concurrent transactions: 8 crimson_compat: true flavor: debug log-ignorelist: - \(MDS_ALL_DOWN\) - \(MDS_UP_LESS_THAN_MAX\) - \(POOL_APP_NOT_ENABLED\) sha1: e6eec7683022e6824acc16f4c26421a3804e27e6 ceph-deploy: conf: client: log file: /var/log/ceph/ceph-$name.$pid.log mon: {} install: ceph: flavor: debug sha1: e6eec7683022e6824acc16f4c26421a3804e27e6 workunit: branch: main env: CRIMSON_COMPAT: '1' sha1: c32469671c6b0222f786dd36c34eb702f780a11d owner: scheduled_matan@soko04.front.sepia.ceph.com priority: 50 repo: https://git.ceph.com/ceph-ci.git roles: - - mon.a - osd.0 - osd.1 - client.0 - node-exporter.a - - mgr.x - osd.2 - osd.3 - client.1 - prometheus.a - node-exporter.b seed: 6481 sha1: e6eec7683022e6824acc16f4c26421a3804e27e6 sleep_before_teardown: 0 suite: crimson-rados suite_branch: main suite_path: /home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa suite_relpath: qa suite_repo: https://github.com/ceph/ceph.git suite_sha1: c32469671c6b0222f786dd36c34eb702f780a11d targets: trial094.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNItKcM4F54n75w9bYtOy+mNooHfRZ0e9nBYaGEz+Fwne9dqiisMQDaPCAKfnA3EU8Wwe796Ug5dwYqxYwGlPOs= trial151.front.sepia.ceph.com: ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEWZ+lNTu2ibpKKgOhd/cEcvkOJ9586tIGMTc7D50i+WUZai6cS4c4HmCNcwWvmtIz8PpHRs1az9tjS00k1s+iw= tasks: - install: extra_packages: deb: - ceph-osd-crimson - ceph-osd-crimson-dbg rpm: - ceph-osd-crimson - ceph-osd-crimson-debuginfo - pexec: all: - sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson - install: null - ceph: conf: mon: mon min osdmap epochs: 50 mon osdmap full prune interval: 2 mon osdmap full prune min: 15 mon osdmap full prune txsize: 2 paxos service trim min: 10 osd: debug monc: 20 - workunit: clients: client.0: - cls/test_cls_cmpomap.sh teuthology: fragments_dropped: [] meta: {} postmerge: [] teuthology_branch: main teuthology_sha1: fa17720d0088c3ac28e473468bfc79eeaff5cd38 timestamp: 2026-02-25_07:58:25 tube: trial user: matan verbose: false worker_log: /home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699 2026-02-25T09:06:41.211 INFO:teuthology.run:suite_path is set to /home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa; will attempt to use it 2026-02-25T09:06:41.211 INFO:teuthology.run:Found tasks at /home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa/tasks 2026-02-25T09:06:41.211 INFO:teuthology.run_tasks:Running task internal.check_packages... 2026-02-25T09:06:41.213 INFO:teuthology.task.internal:Checking packages... 2026-02-25T09:06:41.213 INFO:teuthology.task.internal:Checking packages for os_type 'centos', flavor 'debug' and ceph hash 'e6eec7683022e6824acc16f4c26421a3804e27e6' 2026-02-25T09:06:41.213 WARNING:teuthology.packaging:More than one of ref, tag, branch, or sha1 supplied; using branch 2026-02-25T09:06:41.213 INFO:teuthology.packaging:ref: None 2026-02-25T09:06:41.213 INFO:teuthology.packaging:tag: None 2026-02-25T09:06:41.213 INFO:teuthology.packaging:branch: main 2026-02-25T09:06:41.213 INFO:teuthology.packaging:sha1: e6eec7683022e6824acc16f4c26421a3804e27e6 2026-02-25T09:06:41.213 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&ref=main 2026-02-25T09:06:41.338 INFO:teuthology.task.internal:Found packages for ceph version 20.3.0-5536.ge6eec768 2026-02-25T09:06:41.339 INFO:teuthology.run_tasks:Running task internal.buildpackages_prep... 2026-02-25T09:06:41.350 INFO:teuthology.task.internal:no buildpackages task found 2026-02-25T09:06:41.351 INFO:teuthology.run_tasks:Running task internal.save_config... 2026-02-25T09:06:41.364 INFO:teuthology.task.internal:Saving configuration 2026-02-25T09:06:41.376 INFO:teuthology.run_tasks:Running task internal.check_lock... 2026-02-25T09:06:41.390 INFO:teuthology.task.internal.check_lock:Checking locks... 2026-02-25T09:06:41.471 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial094.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/matan-2026-02-25_07:58:25-crimson-rados-main-distro-debug-trial/69215', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-25 09:03:12.953847', 'locked_by': 'scheduled_matan@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNItKcM4F54n75w9bYtOy+mNooHfRZ0e9nBYaGEz+Fwne9dqiisMQDaPCAKfnA3EU8Wwe796Ug5dwYqxYwGlPOs='} 2026-02-25T09:06:41.530 DEBUG:teuthology.task.internal.check_lock:machine status is {'name': 'trial151.front.sepia.ceph.com', 'description': '/home/teuthworker/mnt/teuthology/matan-2026-02-25_07:58:25-crimson-rados-main-distro-debug-trial/69215', 'up': True, 'machine_type': 'trial', 'is_vm': False, 'vm_host': None, 'os_type': 'centos', 'os_version': '9.stream', 'arch': 'x86_64', 'locked': True, 'locked_since': '2026-02-25 09:03:12.954646', 'locked_by': 'scheduled_matan@soko04.front.sepia.ceph.com', 'mac_address': None, 'ssh_pub_key': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEWZ+lNTu2ibpKKgOhd/cEcvkOJ9586tIGMTc7D50i+WUZai6cS4c4HmCNcwWvmtIz8PpHRs1az9tjS00k1s+iw='} 2026-02-25T09:06:41.530 INFO:teuthology.run_tasks:Running task internal.add_remotes... 2026-02-25T09:06:41.542 INFO:teuthology.task.internal:roles: ubuntu@trial094.front.sepia.ceph.com - ['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'] 2026-02-25T09:06:41.542 INFO:teuthology.task.internal:roles: ubuntu@trial151.front.sepia.ceph.com - ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b'] 2026-02-25T09:06:41.542 INFO:teuthology.run_tasks:Running task console_log... 2026-02-25T09:06:41.677 DEBUG:teuthology.exit:Installing handler: Handler(exiter=, func=.kill_console_loggers at 0x7f5cc2365620>, signals=[15]) 2026-02-25T09:06:41.677 INFO:teuthology.run_tasks:Running task internal.connect... 2026-02-25T09:06:41.691 INFO:teuthology.task.internal:Opening connections... 2026-02-25T09:06:41.691 DEBUG:teuthology.task.internal:connecting to ubuntu@trial094.front.sepia.ceph.com 2026-02-25T09:06:41.692 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T09:06:41.759 DEBUG:teuthology.task.internal:connecting to ubuntu@trial151.front.sepia.ceph.com 2026-02-25T09:06:41.760 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T09:06:41.822 INFO:teuthology.run_tasks:Running task internal.push_inventory... 2026-02-25T09:06:41.837 DEBUG:teuthology.orchestra.run.trial094:> uname -m 2026-02-25T09:06:41.853 INFO:teuthology.orchestra.run.trial094.stdout:x86_64 2026-02-25T09:06:41.853 DEBUG:teuthology.orchestra.run.trial094:> cat /etc/os-release 2026-02-25T09:06:41.908 INFO:teuthology.orchestra.run.trial094.stdout:NAME="CentOS Stream" 2026-02-25T09:06:41.908 INFO:teuthology.orchestra.run.trial094.stdout:VERSION="9" 2026-02-25T09:06:41.908 INFO:teuthology.orchestra.run.trial094.stdout:ID="centos" 2026-02-25T09:06:41.908 INFO:teuthology.orchestra.run.trial094.stdout:ID_LIKE="rhel fedora" 2026-02-25T09:06:41.908 INFO:teuthology.orchestra.run.trial094.stdout:VERSION_ID="9" 2026-02-25T09:06:41.908 INFO:teuthology.orchestra.run.trial094.stdout:PLATFORM_ID="platform:el9" 2026-02-25T09:06:41.908 INFO:teuthology.orchestra.run.trial094.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-25T09:06:41.908 INFO:teuthology.orchestra.run.trial094.stdout:ANSI_COLOR="0;31" 2026-02-25T09:06:41.908 INFO:teuthology.orchestra.run.trial094.stdout:LOGO="fedora-logo-icon" 2026-02-25T09:06:41.909 INFO:teuthology.orchestra.run.trial094.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-25T09:06:41.909 INFO:teuthology.orchestra.run.trial094.stdout:HOME_URL="https://centos.org/" 2026-02-25T09:06:41.909 INFO:teuthology.orchestra.run.trial094.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-25T09:06:41.909 INFO:teuthology.orchestra.run.trial094.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-25T09:06:41.909 INFO:teuthology.orchestra.run.trial094.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-25T09:06:41.909 INFO:teuthology.lock.ops:Updating trial094.front.sepia.ceph.com on lock server 2026-02-25T09:06:41.980 DEBUG:teuthology.orchestra.run.trial151:> uname -m 2026-02-25T09:06:41.996 INFO:teuthology.orchestra.run.trial151.stdout:x86_64 2026-02-25T09:06:41.996 DEBUG:teuthology.orchestra.run.trial151:> cat /etc/os-release 2026-02-25T09:06:42.052 INFO:teuthology.orchestra.run.trial151.stdout:NAME="CentOS Stream" 2026-02-25T09:06:42.052 INFO:teuthology.orchestra.run.trial151.stdout:VERSION="9" 2026-02-25T09:06:42.052 INFO:teuthology.orchestra.run.trial151.stdout:ID="centos" 2026-02-25T09:06:42.052 INFO:teuthology.orchestra.run.trial151.stdout:ID_LIKE="rhel fedora" 2026-02-25T09:06:42.052 INFO:teuthology.orchestra.run.trial151.stdout:VERSION_ID="9" 2026-02-25T09:06:42.052 INFO:teuthology.orchestra.run.trial151.stdout:PLATFORM_ID="platform:el9" 2026-02-25T09:06:42.052 INFO:teuthology.orchestra.run.trial151.stdout:PRETTY_NAME="CentOS Stream 9" 2026-02-25T09:06:42.052 INFO:teuthology.orchestra.run.trial151.stdout:ANSI_COLOR="0;31" 2026-02-25T09:06:42.052 INFO:teuthology.orchestra.run.trial151.stdout:LOGO="fedora-logo-icon" 2026-02-25T09:06:42.052 INFO:teuthology.orchestra.run.trial151.stdout:CPE_NAME="cpe:/o:centos:centos:9" 2026-02-25T09:06:42.052 INFO:teuthology.orchestra.run.trial151.stdout:HOME_URL="https://centos.org/" 2026-02-25T09:06:42.052 INFO:teuthology.orchestra.run.trial151.stdout:BUG_REPORT_URL="https://issues.redhat.com/" 2026-02-25T09:06:42.052 INFO:teuthology.orchestra.run.trial151.stdout:REDHAT_SUPPORT_PRODUCT="Red Hat Enterprise Linux 9" 2026-02-25T09:06:42.053 INFO:teuthology.orchestra.run.trial151.stdout:REDHAT_SUPPORT_PRODUCT_VERSION="CentOS Stream" 2026-02-25T09:06:42.053 INFO:teuthology.lock.ops:Updating trial151.front.sepia.ceph.com on lock server 2026-02-25T09:06:42.178 INFO:teuthology.run_tasks:Running task internal.serialize_remote_roles... 2026-02-25T09:06:42.191 INFO:teuthology.run_tasks:Running task internal.check_conflict... 2026-02-25T09:06:42.211 INFO:teuthology.task.internal:Checking for old test directory... 2026-02-25T09:06:42.211 DEBUG:teuthology.orchestra.run.trial094:> test '!' -e /home/ubuntu/cephtest 2026-02-25T09:06:42.217 DEBUG:teuthology.orchestra.run.trial151:> test '!' -e /home/ubuntu/cephtest 2026-02-25T09:06:42.238 INFO:teuthology.run_tasks:Running task internal.check_ceph_data... 2026-02-25T09:06:42.267 INFO:teuthology.task.internal:Checking for non-empty /var/lib/ceph... 2026-02-25T09:06:42.268 DEBUG:teuthology.orchestra.run.trial094:> test -z $(ls -A /var/lib/ceph) 2026-02-25T09:06:42.279 DEBUG:teuthology.orchestra.run.trial151:> test -z $(ls -A /var/lib/ceph) 2026-02-25T09:06:42.304 INFO:teuthology.run_tasks:Running task internal.vm_setup... 2026-02-25T09:06:42.451 INFO:teuthology.run_tasks:Running task kernel... 2026-02-25T09:06:42.470 INFO:teuthology.task.kernel:normalize config orig: {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'} 2026-02-25T09:06:42.470 INFO:teuthology.task.kernel:config {'mon.a': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'mgr.x': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, timeout 300 2026-02-25T09:06:42.470 DEBUG:teuthology.orchestra.run.trial094:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-25T09:06:42.471 DEBUG:teuthology.orchestra.run.trial151:> test -f /run/.containerenv -o -f /.dockerenv 2026-02-25T09:06:42.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T09:06:42.485 DEBUG:teuthology.orchestra.run.trial151:> uname -r 2026-02-25T09:06:42.485 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T09:06:42.485 DEBUG:teuthology.orchestra.run.trial094:> uname -r 2026-02-25T09:06:42.541 INFO:teuthology.orchestra.run.trial151.stdout:5.14.0-681.el9.x86_64 2026-02-25T09:06:42.541 INFO:teuthology.task.kernel:Running kernel on trial151: 5.14.0-681.el9.x86_64 2026-02-25T09:06:42.541 DEBUG:teuthology.orchestra.run.trial151:> sudo yum install -y kernel 2026-02-25T09:06:42.541 INFO:teuthology.orchestra.run.trial094.stdout:5.14.0-681.el9.x86_64 2026-02-25T09:06:42.541 INFO:teuthology.task.kernel:Running kernel on trial094: 5.14.0-681.el9.x86_64 2026-02-25T09:06:42.541 DEBUG:teuthology.orchestra.run.trial094:> sudo yum install -y kernel 2026-02-25T09:06:43.627 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - BaseOS 13 MB/s | 8.9 MB 00:00 2026-02-25T09:06:43.701 INFO:teuthology.orchestra.run.trial151.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-25T09:06:46.169 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - AppStream 13 MB/s | 27 MB 00:02 2026-02-25T09:06:46.976 INFO:teuthology.orchestra.run.trial151.stdout:CentOS Stream 9 - AppStream 9.8 MB/s | 27 MB 00:02 2026-02-25T09:06:49.571 INFO:teuthology.orchestra.run.trial151.stdout:CentOS Stream 9 - CRB 36 MB/s | 8.0 MB 00:00 2026-02-25T09:06:50.202 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - CRB 4.8 MB/s | 8.0 MB 00:01 2026-02-25T09:06:50.413 INFO:teuthology.orchestra.run.trial151.stdout:CentOS Stream 9 - Extras packages 192 kB/s | 20 kB 00:00 2026-02-25T09:06:51.023 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - Extras packages 255 kB/s | 20 kB 00:00 2026-02-25T09:06:51.789 INFO:teuthology.orchestra.run.trial094.stdout:Extra Packages for Enterprise Linux 29 MB/s | 20 MB 00:00 2026-02-25T09:06:52.189 INFO:teuthology.orchestra.run.trial151.stdout:Extra Packages for Enterprise Linux 12 MB/s | 20 MB 00:01 2026-02-25T09:06:54.989 INFO:teuthology.orchestra.run.trial094.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-25T09:06:55.356 INFO:teuthology.orchestra.run.trial151.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-25T09:06:55.995 INFO:teuthology.orchestra.run.trial094.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-25T09:06:55.995 INFO:teuthology.orchestra.run.trial094.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-25T09:06:56.015 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:06:56.016 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:06:56.016 INFO:teuthology.orchestra.run.trial094.stdout: Package Architecture Version Repository Size 2026-02-25T09:06:56.016 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:06:56.016 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:06:56.017 INFO:teuthology.orchestra.run.trial094.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-25T09:06:56.017 INFO:teuthology.orchestra.run.trial094.stdout:Installing dependencies: 2026-02-25T09:06:56.017 INFO:teuthology.orchestra.run.trial094.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-25T09:06:56.017 INFO:teuthology.orchestra.run.trial094.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-25T09:06:56.017 INFO:teuthology.orchestra.run.trial094.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-25T09:06:56.017 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:06:56.017 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:06:56.017 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:06:56.017 INFO:teuthology.orchestra.run.trial094.stdout:Install 4 Packages 2026-02-25T09:06:56.018 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:06:56.018 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 89 M 2026-02-25T09:06:56.018 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 135 M 2026-02-25T09:06:56.018 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:06:56.271 INFO:teuthology.orchestra.run.trial094.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 5.2 MB/s | 901 kB 00:00 2026-02-25T09:06:56.357 INFO:teuthology.orchestra.run.trial151.stdout:Package kernel-5.14.0-645.el9.x86_64 is already installed. 2026-02-25T09:06:56.358 INFO:teuthology.orchestra.run.trial151.stdout:Package kernel-5.14.0-681.el9.x86_64 is already installed. 2026-02-25T09:06:56.377 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:06:56.378 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:06:56.379 INFO:teuthology.orchestra.run.trial151.stdout: Package Architecture Version Repository Size 2026-02-25T09:06:56.379 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:06:56.379 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:06:56.379 INFO:teuthology.orchestra.run.trial151.stdout: kernel x86_64 5.14.0-686.el9 baseos 901 k 2026-02-25T09:06:56.379 INFO:teuthology.orchestra.run.trial151.stdout:Installing dependencies: 2026-02-25T09:06:56.379 INFO:teuthology.orchestra.run.trial151.stdout: kernel-core x86_64 5.14.0-686.el9 baseos 17 M 2026-02-25T09:06:56.379 INFO:teuthology.orchestra.run.trial151.stdout: kernel-modules x86_64 5.14.0-686.el9 baseos 40 M 2026-02-25T09:06:56.379 INFO:teuthology.orchestra.run.trial151.stdout: kernel-modules-core x86_64 5.14.0-686.el9 baseos 31 M 2026-02-25T09:06:56.379 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:06:56.379 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:06:56.380 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:06:56.380 INFO:teuthology.orchestra.run.trial151.stdout:Install 4 Packages 2026-02-25T09:06:56.380 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:06:56.380 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 89 M 2026-02-25T09:06:56.380 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 135 M 2026-02-25T09:06:56.380 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:06:56.414 INFO:teuthology.orchestra.run.trial094.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 55 MB/s | 17 MB 00:00 2026-02-25T09:06:56.559 INFO:teuthology.orchestra.run.trial094.stdout:(3/4): kernel-modules-core-5.14.0-686.el9.x86_6 108 MB/s | 31 MB 00:00 2026-02-25T09:06:56.612 INFO:teuthology.orchestra.run.trial151.stdout:(1/4): kernel-5.14.0-686.el9.x86_64.rpm 5.1 MB/s | 901 kB 00:00 2026-02-25T09:06:56.633 INFO:teuthology.orchestra.run.trial094.stdout:(4/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 75 MB/s | 40 MB 00:00 2026-02-25T09:06:56.633 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:06:56.633 INFO:teuthology.orchestra.run.trial094.stdout:Total 144 MB/s | 89 MB 00:00 2026-02-25T09:06:56.824 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:06:57.026 INFO:teuthology.orchestra.run.trial151.stdout:(2/4): kernel-core-5.14.0-686.el9.x86_64.rpm 29 MB/s | 17 MB 00:00 2026-02-25T09:06:57.029 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:06:57.029 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:06:57.414 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:06:57.415 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:06:58.552 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:06:58.660 INFO:teuthology.orchestra.run.trial151.stdout:(3/4): kernel-modules-core-5.14.0-686.el9.x86_6 15 MB/s | 31 MB 00:02 2026-02-25T09:06:58.712 INFO:teuthology.orchestra.run.trial151.stdout:(4/4): kernel-modules-5.14.0-686.el9.x86_64.rpm 18 MB/s | 40 MB 00:02 2026-02-25T09:06:58.712 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:06:58.713 INFO:teuthology.orchestra.run.trial151.stdout:Total 38 MB/s | 89 MB 00:02 2026-02-25T09:06:58.856 INFO:teuthology.orchestra.run.trial094.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-25T09:06:58.878 INFO:teuthology.orchestra.run.trial094.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T09:06:58.904 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:06:59.114 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:06:59.114 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:06:59.498 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:06:59.498 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:07:00.606 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:07:00.894 INFO:teuthology.orchestra.run.trial151.stdout: Installing : kernel-modules-core-5.14.0-686.el9.x86_64 1/4 2026-02-25T09:07:00.916 INFO:teuthology.orchestra.run.trial151.stdout: Installing : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T09:07:01.441 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T09:07:01.474 INFO:teuthology.orchestra.run.trial094.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T09:07:03.501 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T09:07:03.534 INFO:teuthology.orchestra.run.trial151.stdout: Installing : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T09:07:04.525 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T09:07:04.531 INFO:teuthology.orchestra.run.trial094.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T09:07:06.572 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T09:07:06.579 INFO:teuthology.orchestra.run.trial151.stdout: Installing : kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T09:07:07.585 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T09:07:09.834 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T09:07:24.965 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T09:07:24.985 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-25T09:07:25.171 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T09:07:25.171 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-25T09:07:25.171 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T09:07:25.171 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T09:07:25.228 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T09:07:25.228 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:07:25.229 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:07:25.229 INFO:teuthology.orchestra.run.trial094.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-25T09:07:25.229 INFO:teuthology.orchestra.run.trial094.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-25T09:07:25.229 INFO:teuthology.orchestra.run.trial094.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-25T09:07:25.229 INFO:teuthology.orchestra.run.trial094.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-25T09:07:25.229 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:07:25.229 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:07:25.292 DEBUG:teuthology.orchestra.run.trial094:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T09:07:25.320 INFO:teuthology.orchestra.run.trial094.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T09:07:25.322 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T09:07:25.322 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-25T09:07:25.322 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-25T09:07:25.322 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-25T09:07:25.322 INFO:teuthology.task.kernel:Installing distro kernel on mon.a... 2026-02-25T09:07:25.322 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial094.front.sepia.ceph.com, path=None, version=distro) 2026-02-25T09:07:25.322 DEBUG:teuthology.orchestra.run.trial094:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T09:07:25.389 INFO:teuthology.orchestra.run.trial094.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T09:07:25.390 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T09:07:25.391 DEBUG:teuthology.orchestra.run.trial094:> sudo rpm -qi grub2-tools 2026-02-25T09:07:25.460 INFO:teuthology.orchestra.run.trial094.stdout:Name : grub2-tools 2026-02-25T09:07:25.460 INFO:teuthology.orchestra.run.trial094.stdout:Epoch : 1 2026-02-25T09:07:25.460 INFO:teuthology.orchestra.run.trial094.stdout:Version : 2.06 2026-02-25T09:07:25.460 INFO:teuthology.orchestra.run.trial094.stdout:Release : 120.el9 2026-02-25T09:07:25.461 INFO:teuthology.orchestra.run.trial094.stdout:Architecture: x86_64 2026-02-25T09:07:25.461 INFO:teuthology.orchestra.run.trial094.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-25T09:07:25.461 INFO:teuthology.orchestra.run.trial094.stdout:Group : Unspecified 2026-02-25T09:07:25.461 INFO:teuthology.orchestra.run.trial094.stdout:Size : 9108611 2026-02-25T09:07:25.461 INFO:teuthology.orchestra.run.trial094.stdout:License : GPLv3+ 2026-02-25T09:07:25.461 INFO:teuthology.orchestra.run.trial094.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-25T09:07:25.461 INFO:teuthology.orchestra.run.trial094.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-25T09:07:25.461 INFO:teuthology.orchestra.run.trial094.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-25T09:07:25.461 INFO:teuthology.orchestra.run.trial094.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-25T09:07:25.461 INFO:teuthology.orchestra.run.trial094.stdout:Packager : builder@centos.org 2026-02-25T09:07:25.462 INFO:teuthology.orchestra.run.trial094.stdout:Vendor : CentOS 2026-02-25T09:07:25.462 INFO:teuthology.orchestra.run.trial094.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-25T09:07:25.462 INFO:teuthology.orchestra.run.trial094.stdout:Summary : Support tools for GRUB. 2026-02-25T09:07:25.462 INFO:teuthology.orchestra.run.trial094.stdout:Description : 2026-02-25T09:07:25.462 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:07:25.462 INFO:teuthology.orchestra.run.trial094.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-25T09:07:25.462 INFO:teuthology.orchestra.run.trial094.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-25T09:07:25.462 INFO:teuthology.orchestra.run.trial094.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-25T09:07:25.462 INFO:teuthology.orchestra.run.trial094.stdout:hardware devices. 2026-02-25T09:07:25.462 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:07:25.462 INFO:teuthology.orchestra.run.trial094.stdout:This subpackage provides tools for support of all platforms. 2026-02-25T09:07:25.463 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-25T09:07:25.464 INFO:teuthology.task.kernel:Updating grub on trial094 to boot 5.14.0-686.el9.x86_64 2026-02-25T09:07:25.464 DEBUG:teuthology.orchestra.run.trial094:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-25T09:07:25.527 INFO:teuthology.orchestra.run.trial094.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-25T09:07:25.529 DEBUG:teuthology.orchestra.run.trial094:> sudo command -v grubby && echo yes || echo no 2026-02-25T09:07:25.592 INFO:teuthology.orchestra.run.trial094.stdout:/sbin/grubby 2026-02-25T09:07:25.594 INFO:teuthology.orchestra.run.trial094.stdout:yes 2026-02-25T09:07:25.594 DEBUG:teuthology.orchestra.run.trial094:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-25T09:07:25.658 INFO:teuthology.orchestra.run.trial094.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-25T09:07:25.659 DEBUG:teuthology.orchestra.run.trial094:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-25T09:07:25.731 DEBUG:teuthology.orchestra.run.trial094:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-25T09:07:25.936 INFO:teuthology.orchestra.run.trial094.stderr:Generating grub configuration file ... 2026-02-25T09:07:26.003 INFO:teuthology.orchestra.run.trial094.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-25T09:07:26.010 INFO:teuthology.orchestra.run.trial094.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-25T09:07:26.259 INFO:teuthology.orchestra.run.trial094.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-25T09:07:26.266 INFO:teuthology.orchestra.run.trial094.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-25T09:07:26.300 INFO:teuthology.orchestra.run.trial094.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-25T09:07:26.306 INFO:teuthology.orchestra.run.trial094.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-25T09:07:26.337 INFO:teuthology.orchestra.run.trial094.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-25T09:07:26.344 INFO:teuthology.orchestra.run.trial094.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-25T09:07:26.367 INFO:teuthology.orchestra.run.trial094.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-25T09:07:26.373 INFO:teuthology.orchestra.run.trial094.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-25T09:07:26.458 INFO:teuthology.orchestra.run.trial094.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-25T09:07:26.467 INFO:teuthology.orchestra.run.trial094.stderr:done 2026-02-25T09:07:26.469 DEBUG:teuthology.orchestra.run.trial094:> test -d /sys/firmware/efi 2026-02-25T09:07:26.485 DEBUG:teuthology.orchestra.run.trial094:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-25T09:07:26.550 INFO:teuthology.orchestra.run.trial094.stdout:/boot/efi/EFI/BOOT 2026-02-25T09:07:26.550 INFO:teuthology.orchestra.run.trial094.stdout:/boot/efi/EFI/centos 2026-02-25T09:07:26.551 DEBUG:teuthology.orchestra.run.trial094:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-25T09:07:26.618 INFO:teuthology.orchestra.run.trial094.stdout:no 2026-02-25T09:07:26.619 DEBUG:teuthology.orchestra.run.trial094:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-25T09:07:26.682 INFO:teuthology.orchestra.run.trial094.stdout:yes 2026-02-25T09:07:26.683 DEBUG:teuthology.orchestra.run.trial094:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-25T09:07:26.745 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial094 2026-02-25T09:07:26.746 DEBUG:teuthology.orchestra.run.trial094:> sudo bash -c ' 2026-02-25T09:07:26.746 DEBUG:teuthology.orchestra.run.trial094:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-25T09:07:26.746 DEBUG:teuthology.orchestra.run.trial094:> ' 2026-02-25T09:07:27.251 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: kernel-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T09:07:27.273 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: kernel-modules-5.14.0-686.el9.x86_64 4/4 2026-02-25T09:07:27.460 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: kernel-5.14.0-686.el9.x86_64 4/4 2026-02-25T09:07:27.460 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : kernel-5.14.0-686.el9.x86_64 1/4 2026-02-25T09:07:27.460 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : kernel-core-5.14.0-686.el9.x86_64 2/4 2026-02-25T09:07:27.460 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : kernel-modules-5.14.0-686.el9.x86_64 3/4 2026-02-25T09:07:27.517 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : kernel-modules-core-5.14.0-686.el9.x86_64 4/4 2026-02-25T09:07:27.518 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:07:27.518 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:07:27.518 INFO:teuthology.orchestra.run.trial151.stdout: kernel-5.14.0-686.el9.x86_64 2026-02-25T09:07:27.518 INFO:teuthology.orchestra.run.trial151.stdout: kernel-core-5.14.0-686.el9.x86_64 2026-02-25T09:07:27.518 INFO:teuthology.orchestra.run.trial151.stdout: kernel-modules-5.14.0-686.el9.x86_64 2026-02-25T09:07:27.518 INFO:teuthology.orchestra.run.trial151.stdout: kernel-modules-core-5.14.0-686.el9.x86_64 2026-02-25T09:07:27.518 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:07:27.518 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:07:27.579 DEBUG:teuthology.orchestra.run.trial151:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T09:07:27.609 INFO:teuthology.orchestra.run.trial151.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T09:07:27.610 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T09:07:27.610 INFO:teuthology.task.kernel:Not newest distro kernel. Current: 5.14.0-681.el9.x86_64 Expected: 5.14.0-686.el9.x86_64 2026-02-25T09:07:27.610 INFO:teuthology.task.kernel:Skipping firmware on distro kernel 2026-02-25T09:07:27.610 DEBUG:teuthology.task.kernel:src is distro, skipping download 2026-02-25T09:07:27.610 INFO:teuthology.task.kernel:Installing distro kernel on mgr.x... 2026-02-25T09:07:27.610 DEBUG:teuthology.task.kernel:install_kernel(remote=ubuntu@trial151.front.sepia.ceph.com, path=None, version=distro) 2026-02-25T09:07:27.611 DEBUG:teuthology.orchestra.run.trial151:> rpm -q kernel | sort -rV | head -n 1 2026-02-25T09:07:27.674 INFO:teuthology.orchestra.run.trial151.stdout:kernel-5.14.0-686.el9.x86_64 2026-02-25T09:07:27.675 DEBUG:teuthology.task.kernel:get_latest_image_version_rpm: 5.14.0-686.el9.x86_64 2026-02-25T09:07:27.675 DEBUG:teuthology.orchestra.run.trial151:> sudo rpm -qi grub2-tools 2026-02-25T09:07:27.744 INFO:teuthology.orchestra.run.trial151.stdout:Name : grub2-tools 2026-02-25T09:07:27.744 INFO:teuthology.orchestra.run.trial151.stdout:Epoch : 1 2026-02-25T09:07:27.744 INFO:teuthology.orchestra.run.trial151.stdout:Version : 2.06 2026-02-25T09:07:27.745 INFO:teuthology.orchestra.run.trial151.stdout:Release : 120.el9 2026-02-25T09:07:27.745 INFO:teuthology.orchestra.run.trial151.stdout:Architecture: x86_64 2026-02-25T09:07:27.745 INFO:teuthology.orchestra.run.trial151.stdout:Install Date: Wed 18 Feb 2026 05:18:42 PM UTC 2026-02-25T09:07:27.745 INFO:teuthology.orchestra.run.trial151.stdout:Group : Unspecified 2026-02-25T09:07:27.745 INFO:teuthology.orchestra.run.trial151.stdout:Size : 9108611 2026-02-25T09:07:27.745 INFO:teuthology.orchestra.run.trial151.stdout:License : GPLv3+ 2026-02-25T09:07:27.745 INFO:teuthology.orchestra.run.trial151.stdout:Signature : RSA/SHA256, Thu 04 Dec 2025 09:42:59 AM UTC, Key ID 05b555b38483c65d 2026-02-25T09:07:27.745 INFO:teuthology.orchestra.run.trial151.stdout:Source RPM : grub2-2.06-120.el9.src.rpm 2026-02-25T09:07:27.745 INFO:teuthology.orchestra.run.trial151.stdout:Build Date : Wed 03 Dec 2025 12:10:38 PM UTC 2026-02-25T09:07:27.745 INFO:teuthology.orchestra.run.trial151.stdout:Build Host : x86-05.stream.rdu2.redhat.com 2026-02-25T09:07:27.745 INFO:teuthology.orchestra.run.trial151.stdout:Packager : builder@centos.org 2026-02-25T09:07:27.746 INFO:teuthology.orchestra.run.trial151.stdout:Vendor : CentOS 2026-02-25T09:07:27.746 INFO:teuthology.orchestra.run.trial151.stdout:URL : http://www.gnu.org/software/grub/ 2026-02-25T09:07:27.746 INFO:teuthology.orchestra.run.trial151.stdout:Summary : Support tools for GRUB. 2026-02-25T09:07:27.746 INFO:teuthology.orchestra.run.trial151.stdout:Description : 2026-02-25T09:07:27.746 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:07:27.746 INFO:teuthology.orchestra.run.trial151.stdout:The GRand Unified Bootloader (GRUB) is a highly configurable and 2026-02-25T09:07:27.746 INFO:teuthology.orchestra.run.trial151.stdout:customizable bootloader with modular architecture. It supports a rich 2026-02-25T09:07:27.746 INFO:teuthology.orchestra.run.trial151.stdout:variety of kernel formats, file systems, computer architectures and 2026-02-25T09:07:27.746 INFO:teuthology.orchestra.run.trial151.stdout:hardware devices. 2026-02-25T09:07:27.746 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:07:27.746 INFO:teuthology.orchestra.run.trial151.stdout:This subpackage provides tools for support of all platforms. 2026-02-25T09:07:27.748 INFO:teuthology.task.kernel:Updating Grub Version: grub2 2026-02-25T09:07:27.748 INFO:teuthology.task.kernel:Updating grub on trial151 to boot 5.14.0-686.el9.x86_64 2026-02-25T09:07:27.748 DEBUG:teuthology.orchestra.run.trial151:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f -print -quit 2026-02-25T09:07:27.810 INFO:teuthology.orchestra.run.trial151.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-681.el9.x86_64.conf 2026-02-25T09:07:27.811 DEBUG:teuthology.orchestra.run.trial151:> sudo command -v grubby && echo yes || echo no 2026-02-25T09:07:27.874 INFO:teuthology.orchestra.run.trial151.stdout:/sbin/grubby 2026-02-25T09:07:27.875 INFO:teuthology.orchestra.run.trial151.stdout:yes 2026-02-25T09:07:27.876 DEBUG:teuthology.orchestra.run.trial151:> sudo find /boot/loader/entries -maxdepth 1 -name "*.conf" -type f | grep -F "5.14.0-686.el9.x86_64" | head -n 1 2026-02-25T09:07:27.939 INFO:teuthology.orchestra.run.trial151.stdout:/boot/loader/entries/b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64.conf 2026-02-25T09:07:27.939 DEBUG:teuthology.orchestra.run.trial151:> sudo grub2-set-default b0373be789df49cdafedf423703d2566-5.14.0-686.el9.x86_64 2026-02-25T09:07:28.013 DEBUG:teuthology.orchestra.run.trial151:> sudo grub2-mkconfig -o /boot/grub2/grub.cfg 2026-02-25T09:07:28.221 INFO:teuthology.orchestra.run.trial151.stderr:Generating grub configuration file ... 2026-02-25T09:07:28.290 INFO:teuthology.orchestra.run.trial151.stderr:Found linux image: /boot/vmlinuz-5.14.0-686.el9.x86_64 2026-02-25T09:07:28.296 INFO:teuthology.orchestra.run.trial151.stderr:Found initrd image: /boot/initramfs-5.14.0-686.el9.x86_64.img 2026-02-25T09:07:28.552 INFO:teuthology.orchestra.run.trial151.stderr:Found linux image: /boot/vmlinuz-5.14.0-681.el9.x86_64 2026-02-25T09:07:28.558 INFO:teuthology.orchestra.run.trial151.stderr:Found initrd image: /boot/initramfs-5.14.0-681.el9.x86_64.img 2026-02-25T09:07:28.593 INFO:teuthology.orchestra.run.trial151.stderr:Found linux image: /boot/vmlinuz-5.14.0-645.el9.x86_64 2026-02-25T09:07:28.600 INFO:teuthology.orchestra.run.trial151.stderr:Found initrd image: /boot/initramfs-5.14.0-645.el9.x86_64.img 2026-02-25T09:07:28.631 INFO:teuthology.orchestra.run.trial151.stderr:Found linux image: /boot/vmlinuz-0-rescue-b0373be789df49cdafedf423703d2566 2026-02-25T09:07:28.638 INFO:teuthology.orchestra.run.trial151.stderr:Found initrd image: /boot/initramfs-0-rescue-b0373be789df49cdafedf423703d2566.img 2026-02-25T09:07:28.661 INFO:teuthology.orchestra.run.trial151.stderr:Found linux image: /boot/vmlinuz-0-rescue-2734456005d743dca7f3dc77c35a2d0d 2026-02-25T09:07:28.668 INFO:teuthology.orchestra.run.trial151.stderr:Found initrd image: /boot/initramfs-0-rescue-2734456005d743dca7f3dc77c35a2d0d.img 2026-02-25T09:07:28.754 INFO:teuthology.orchestra.run.trial151.stderr:Adding boot menu entry for UEFI Firmware Settings ... 2026-02-25T09:07:28.762 INFO:teuthology.orchestra.run.trial151.stderr:done 2026-02-25T09:07:28.764 DEBUG:teuthology.orchestra.run.trial151:> test -d /sys/firmware/efi 2026-02-25T09:07:28.781 DEBUG:teuthology.orchestra.run.trial151:> sudo find /boot/efi/EFI -mindepth 1 -maxdepth 1 -type d || true 2026-02-25T09:07:28.847 INFO:teuthology.orchestra.run.trial151.stdout:/boot/efi/EFI/BOOT 2026-02-25T09:07:28.847 INFO:teuthology.orchestra.run.trial151.stdout:/boot/efi/EFI/centos 2026-02-25T09:07:28.848 DEBUG:teuthology.orchestra.run.trial151:> sudo test -f /boot/efi/EFI/BOOT/grub.cfg && echo yes || echo no 2026-02-25T09:07:28.914 INFO:teuthology.orchestra.run.trial151.stdout:no 2026-02-25T09:07:28.915 DEBUG:teuthology.orchestra.run.trial151:> sudo test -f /boot/efi/EFI/centos/grub.cfg && echo yes || echo no 2026-02-25T09:07:28.978 INFO:teuthology.orchestra.run.trial151.stdout:yes 2026-02-25T09:07:28.978 DEBUG:teuthology.orchestra.run.trial151:> sudo cp -f /boot/grub2/grub.cfg /boot/efi/EFI/centos/grub.cfg 2026-02-25T09:07:29.042 INFO:teuthology.orchestra.remote:Triggering safe hard reboot via SysRq on trial151 2026-02-25T09:07:29.043 DEBUG:teuthology.orchestra.run.trial151:> sudo bash -c ' 2026-02-25T09:07:29.043 DEBUG:teuthology.orchestra.run.trial151:> echo pci > /sys/kernel/reboot/type && echo 1 > /sys/kernel/reboot/force && sync && echo s > /proc/sysrq-trigger && echo u > /proc/sysrq-trigger && sleep 2 && echo b > /proc/sysrq-trigger 2026-02-25T09:07:29.043 DEBUG:teuthology.orchestra.run.trial151:> ' 2026-02-25T09:07:56.790 INFO:teuthology.task.kernel:Checking client mon.a for new kernel version... 2026-02-25T09:07:56.790 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial094.front.sepia.ceph.com' 2026-02-25T09:07:56.791 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T09:07:59.085 INFO:teuthology.task.kernel:Checking client mgr.x for new kernel version... 2026-02-25T09:07:59.086 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial151.front.sepia.ceph.com' 2026-02-25T09:07:59.087 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T09:08:11.073 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.94 2026-02-25T09:08:13.375 DEBUG:teuthology.orchestra.remote:[Errno None] Unable to connect to port 22 on 10.20.193.151 2026-02-25T09:08:20.080 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial094.front.sepia.ceph.com' 2026-02-25T09:08:20.081 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T09:08:22.378 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial151.front.sepia.ceph.com' 2026-02-25T09:08:22.380 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T09:08:55.946 DEBUG:teuthology.orchestra.run.trial094:> true 2026-02-25T09:08:56.100 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial094.front.sepia.ceph.com' 2026-02-25T09:08:56.100 INFO:teuthology.task.kernel:Checking kernel version of mon.a, want "5.14.0-686.el9.x86_64"... 2026-02-25T09:08:56.100 DEBUG:teuthology.orchestra.run.trial094:> uname -r 2026-02-25T09:08:56.156 INFO:teuthology.orchestra.run.trial094.stdout:5.14.0-686.el9.x86_64 2026-02-25T09:08:56.156 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-25T09:08:56.156 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-25T09:08:56.156 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-25T09:08:57.157 INFO:teuthology.task.kernel:Enabling kdb on mon.a... 2026-02-25T09:08:57.158 DEBUG:teuthology.orchestra.run.trial094:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-25T09:08:57.186 INFO:teuthology.orchestra.run.trial094.stdout:ttyS1 2026-02-25T09:08:57.220 DEBUG:teuthology.parallel:result is None 2026-02-25T09:08:58.502 DEBUG:teuthology.orchestra.run.trial151:> true 2026-02-25T09:08:58.651 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial151.front.sepia.ceph.com' 2026-02-25T09:08:58.651 INFO:teuthology.task.kernel:Checking kernel version of mgr.x, want "5.14.0-686.el9.x86_64"... 2026-02-25T09:08:58.651 DEBUG:teuthology.orchestra.run.trial151:> uname -r 2026-02-25T09:08:58.707 INFO:teuthology.orchestra.run.trial151.stdout:5.14.0-686.el9.x86_64 2026-02-25T09:08:58.708 DEBUG:teuthology.task.kernel:current kernel version is 5.14.0-686.el9.x86_64 vs 5.14.0-686.el9.x86_64 2026-02-25T09:08:58.708 DEBUG:teuthology.task.kernel:utsrelease strings match, do not need to install 2026-02-25T09:08:58.708 DEBUG:teuthology.task.kernel:Distro of this test job: centos 2026-02-25T09:08:59.708 INFO:teuthology.task.kernel:Enabling kdb on mgr.x... 2026-02-25T09:08:59.709 DEBUG:teuthology.orchestra.run.trial151:> echo ttyS1 | sudo tee /sys/module/kgdboc/parameters/kgdboc 2026-02-25T09:08:59.740 INFO:teuthology.orchestra.run.trial151.stdout:ttyS1 2026-02-25T09:08:59.770 DEBUG:teuthology.parallel:result is None 2026-02-25T09:08:59.770 INFO:teuthology.run_tasks:Running task internal.base... 2026-02-25T09:08:59.780 INFO:teuthology.task.internal:Creating test directory... 2026-02-25T09:08:59.781 DEBUG:teuthology.orchestra.run.trial094:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-25T09:08:59.784 DEBUG:teuthology.orchestra.run.trial151:> mkdir -p -m0755 -- /home/ubuntu/cephtest 2026-02-25T09:08:59.825 INFO:teuthology.run_tasks:Running task internal.archive_upload... 2026-02-25T09:08:59.836 INFO:teuthology.run_tasks:Running task internal.archive... 2026-02-25T09:08:59.850 INFO:teuthology.task.internal:Creating archive directory... 2026-02-25T09:08:59.850 DEBUG:teuthology.orchestra.run.trial094:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-25T09:08:59.852 DEBUG:teuthology.orchestra.run.trial151:> install -d -m0755 -- /home/ubuntu/cephtest/archive 2026-02-25T09:08:59.889 INFO:teuthology.run_tasks:Running task internal.coredump... 2026-02-25T09:08:59.900 INFO:teuthology.task.internal:Enabling coredump saving... 2026-02-25T09:08:59.900 DEBUG:teuthology.orchestra.run.trial094:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-25T09:08:59.908 DEBUG:teuthology.orchestra.run.trial151:> install -d -m0755 -- /home/ubuntu/cephtest/archive/coredump && sudo sysctl -w kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core && echo kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core | sudo tee -a /etc/sysctl.conf 2026-02-25T09:08:59.930 INFO:teuthology.orchestra.run.trial094.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T09:08:59.939 INFO:teuthology.orchestra.run.trial094.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T09:08:59.946 INFO:teuthology.orchestra.run.trial151.stdout:kernel.core_pattern = /home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T09:08:59.955 INFO:teuthology.orchestra.run.trial151.stdout:kernel.core_pattern=/home/ubuntu/cephtest/archive/coredump/%t.%p.core 2026-02-25T09:08:59.957 INFO:teuthology.run_tasks:Running task internal.sudo... 2026-02-25T09:08:59.968 INFO:teuthology.task.internal:Configuring sudo... 2026-02-25T09:08:59.969 DEBUG:teuthology.orchestra.run.trial094:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-25T09:08:59.983 DEBUG:teuthology.orchestra.run.trial151:> sudo sed -i.orig.teuthology -e 's/^\([^#]*\) \(requiretty\)/\1 !\2/g' -e 's/^\([^#]*\) !\(visiblepw\)/\1 \2/g' /etc/sudoers 2026-02-25T09:09:00.021 INFO:teuthology.run_tasks:Running task internal.syslog... 2026-02-25T09:09:00.037 INFO:teuthology.task.internal.syslog:Starting syslog monitoring... 2026-02-25T09:09:00.037 DEBUG:teuthology.orchestra.run.trial094:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-25T09:09:00.047 DEBUG:teuthology.orchestra.run.trial151:> mkdir -p -m0755 -- /home/ubuntu/cephtest/archive/syslog 2026-02-25T09:09:00.078 DEBUG:teuthology.orchestra.run.trial094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T09:09:00.223 DEBUG:teuthology.orchestra.run.trial094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T09:09:00.248 DEBUG:teuthology.orchestra.run.trial094:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T09:09:00.401 DEBUG:teuthology.orchestra.run.trial094:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T09:09:00.425 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:09:00.425 DEBUG:teuthology.orchestra.run.trial094:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-25T09:09:00.488 DEBUG:teuthology.orchestra.run.trial151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T09:09:00.601 DEBUG:teuthology.orchestra.run.trial151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T09:09:00.626 DEBUG:teuthology.orchestra.run.trial151:> install -m 666 /dev/null /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T09:09:00.765 DEBUG:teuthology.orchestra.run.trial151:> sudo chcon system_u:object_r:var_log_t:s0 /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T09:09:00.791 DEBUG:teuthology.orchestra.run.trial151:> set -ex 2026-02-25T09:09:00.791 DEBUG:teuthology.orchestra.run.trial151:> sudo dd of=/etc/rsyslog.d/80-cephtest.conf 2026-02-25T09:09:00.856 DEBUG:teuthology.orchestra.run.trial094:> sudo service rsyslog restart 2026-02-25T09:09:00.859 DEBUG:teuthology.orchestra.run.trial151:> sudo service rsyslog restart 2026-02-25T09:09:00.885 INFO:teuthology.orchestra.run.trial094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T09:09:00.924 INFO:teuthology.orchestra.run.trial151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T09:09:01.454 INFO:teuthology.run_tasks:Running task internal.timer... 2026-02-25T09:09:01.470 INFO:teuthology.task.internal:Starting timer... 2026-02-25T09:09:01.470 INFO:teuthology.run_tasks:Running task pcp... 2026-02-25T09:09:01.490 INFO:teuthology.run_tasks:Running task selinux... 2026-02-25T09:09:01.509 DEBUG:teuthology.orchestra.run.trial094:> sudo service auditd rotate 2026-02-25T09:09:01.540 INFO:teuthology.orchestra.run.trial094.stdout:Rotating logs: 2026-02-25T09:09:01.542 DEBUG:teuthology.orchestra.run.trial151:> sudo service auditd rotate 2026-02-25T09:09:01.571 INFO:teuthology.orchestra.run.trial151.stdout:Rotating logs: 2026-02-25T09:09:01.573 DEBUG:teuthology.task.selinux:Getting current SELinux state 2026-02-25T09:09:01.573 DEBUG:teuthology.orchestra.run.trial094:> /usr/sbin/getenforce 2026-02-25T09:09:01.598 INFO:teuthology.orchestra.run.trial094.stdout:Permissive 2026-02-25T09:09:01.598 DEBUG:teuthology.orchestra.run.trial151:> /usr/sbin/getenforce 2026-02-25T09:09:01.629 INFO:teuthology.orchestra.run.trial151.stdout:Permissive 2026-02-25T09:09:01.630 DEBUG:teuthology.task.selinux:Existing SELinux modes: {'ubuntu@trial094.front.sepia.ceph.com': 'permissive', 'ubuntu@trial151.front.sepia.ceph.com': 'permissive'} 2026-02-25T09:09:01.630 DEBUG:teuthology.orchestra.run.trial094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T09:09:01.662 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T09:09:01.663 DEBUG:teuthology.orchestra.run.trial151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T09:09:01.693 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T09:09:01.694 INFO:teuthology.task.selinux:Putting SELinux into permissive mode 2026-02-25T09:09:01.694 DEBUG:teuthology.orchestra.run.trial094:> sudo /usr/sbin/setenforce permissive 2026-02-25T09:09:01.727 DEBUG:teuthology.orchestra.run.trial151:> sudo /usr/sbin/setenforce permissive 2026-02-25T09:09:01.758 INFO:teuthology.run_tasks:Running task ansible.cephlab... 2026-02-25T09:09:01.774 DEBUG:teuthology.repo_utils:Setting repo remote to https://git.ceph.com/ceph-cm-ansible.git 2026-02-25T09:09:01.776 INFO:teuthology.repo_utils:Fetching git.ceph.com_ceph-cm-ansible_main from origin 2026-02-25T09:09:01.886 DEBUG:teuthology.repo_utils:Resetting repo at /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main to origin/main 2026-02-25T09:09:01.897 INFO:teuthology.task.ansible:Playbook: [{'import_playbook': 'ansible_managed.yml'}, {'import_playbook': 'teuthology.yml'}, {'hosts': 'testnodes', 'tasks': [{'set_fact': {'ran_from_cephlab_playbook': True}}]}, {'import_playbook': 'testnodes.yml'}, {'import_playbook': 'container-host.yml'}, {'import_playbook': 'cobbler.yml'}, {'import_playbook': 'paddles.yml'}, {'import_playbook': 'pulpito.yml'}, {'hosts': 'testnodes', 'become': True, 'tasks': [{'name': 'Touch /ceph-qa-ready', 'file': {'path': '/ceph-qa-ready', 'state': 'touch'}, 'when': 'ran_from_cephlab_playbook|bool'}]}] 2026-02-25T09:09:01.898 DEBUG:teuthology.task.ansible:Running ansible-playbook -v --extra-vars '{"ansible_ssh_user": "ubuntu"}' -i /etc/ansible/hosts --limit trial094.front.sepia.ceph.com,trial151.front.sepia.ceph.com /home/teuthworker/src/git.ceph.com_ceph-cm-ansible_main/cephlab.yml 2026-02-25T09:11:56.998 DEBUG:teuthology.task.ansible:Reconnecting to [Remote(name='ubuntu@trial094.front.sepia.ceph.com'), Remote(name='ubuntu@trial151.front.sepia.ceph.com')] 2026-02-25T09:11:57.000 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial094.front.sepia.ceph.com' 2026-02-25T09:11:57.002 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial094.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T09:11:57.113 DEBUG:teuthology.orchestra.run.trial094:> true 2026-02-25T09:11:57.196 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial094.front.sepia.ceph.com' 2026-02-25T09:11:57.196 INFO:teuthology.orchestra.remote:Trying to reconnect to host 'ubuntu@trial151.front.sepia.ceph.com' 2026-02-25T09:11:57.197 DEBUG:teuthology.orchestra.connection:{'hostname': 'trial151.front.sepia.ceph.com', 'username': 'ubuntu', 'timeout': 60} 2026-02-25T09:11:57.277 DEBUG:teuthology.orchestra.run.trial151:> true 2026-02-25T09:11:57.352 INFO:teuthology.orchestra.remote:Successfully reconnected to host 'ubuntu@trial151.front.sepia.ceph.com' 2026-02-25T09:11:57.352 INFO:teuthology.run_tasks:Running task clock... 2026-02-25T09:11:57.361 INFO:teuthology.task.clock:Syncing clocks and checking initial clock skew... 2026-02-25T09:11:57.361 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-25T09:11:57.361 DEBUG:teuthology.orchestra.run.trial094:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T09:11:57.363 INFO:teuthology.orchestra.run:Running command with timeout 360 2026-02-25T09:11:57.363 DEBUG:teuthology.orchestra.run.trial151:> sudo systemctl stop ntp.service || sudo systemctl stop ntpd.service || sudo systemctl stop chronyd.service ; sudo ntpd -gq || sudo chronyc makestep ; sudo systemctl start ntp.service || sudo systemctl start ntpd.service || sudo systemctl start chronyd.service ; PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T09:11:57.388 INFO:teuthology.orchestra.run.trial094.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-25T09:11:57.400 INFO:teuthology.orchestra.run.trial094.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-25T09:11:57.419 INFO:teuthology.orchestra.run.trial151.stderr:Failed to stop ntp.service: Unit ntp.service not loaded. 2026-02-25T09:11:57.420 INFO:teuthology.orchestra.run.trial094.stderr:sudo: ntpd: command not found 2026-02-25T09:11:57.429 INFO:teuthology.orchestra.run.trial094.stdout:506 Cannot talk to daemon 2026-02-25T09:11:57.431 INFO:teuthology.orchestra.run.trial151.stderr:Failed to stop ntpd.service: Unit ntpd.service not loaded. 2026-02-25T09:11:57.440 INFO:teuthology.orchestra.run.trial094.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-25T09:11:57.452 INFO:teuthology.orchestra.run.trial151.stderr:sudo: ntpd: command not found 2026-02-25T09:11:57.452 INFO:teuthology.orchestra.run.trial094.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-25T09:11:57.461 INFO:teuthology.orchestra.run.trial151.stdout:506 Cannot talk to daemon 2026-02-25T09:11:57.472 INFO:teuthology.orchestra.run.trial151.stderr:Failed to start ntp.service: Unit ntp.service not found. 2026-02-25T09:11:57.484 INFO:teuthology.orchestra.run.trial151.stderr:Failed to start ntpd.service: Unit ntpd.service not found. 2026-02-25T09:11:57.495 INFO:teuthology.orchestra.run.trial094.stderr:bash: line 1: ntpq: command not found 2026-02-25T09:11:57.497 INFO:teuthology.orchestra.run.trial094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T09:11:57.497 INFO:teuthology.orchestra.run.trial094.stdout:=============================================================================== 2026-02-25T09:11:57.543 INFO:teuthology.orchestra.run.trial151.stderr:bash: line 1: ntpq: command not found 2026-02-25T09:11:57.561 INFO:teuthology.orchestra.run.trial151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T09:11:57.561 INFO:teuthology.orchestra.run.trial151.stdout:=============================================================================== 2026-02-25T09:11:57.561 INFO:teuthology.orchestra.run.trial151.stdout:^? 198.71.50.75 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T09:11:57.561 INFO:teuthology.orchestra.run.trial151.stdout:^? ns2.fortrockdc.com 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T09:11:57.562 INFO:teuthology.orchestra.run.trial151.stdout:^? time.trtnw.net 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T09:11:57.562 INFO:teuthology.orchestra.run.trial151.stdout:^? nyc2.us.ntp.li 0 6 0 - +0ns[ +0ns] +/- 0ns 2026-02-25T09:11:57.564 INFO:teuthology.run_tasks:Running task install... 2026-02-25T09:11:57.577 DEBUG:teuthology.task.install:project ceph 2026-02-25T09:11:57.578 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': 'e6eec7683022e6824acc16f4c26421a3804e27e6'}} 2026-02-25T09:11:57.578 DEBUG:teuthology.task.install:config {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': 'e6eec7683022e6824acc16f4c26421a3804e27e6'} 2026-02-25T09:11:57.578 INFO:teuthology.task.install:Using flavor: debug 2026-02-25T09:11:57.588 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-25T09:11:57.588 INFO:teuthology.task.install:extra packages: {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']} 2026-02-25T09:11:57.589 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e6eec7683022e6824acc16f4c26421a3804e27e6', 'tag': None, 'wait_for_package': False} 2026-02-25T09:11:57.590 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=e6eec7683022e6824acc16f4c26421a3804e27e6 2026-02-25T09:11:57.593 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e6eec7683022e6824acc16f4c26421a3804e27e6', 'tag': None, 'wait_for_package': False} 2026-02-25T09:11:57.593 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=e6eec7683022e6824acc16f4c26421a3804e27e6 2026-02-25T09:11:57.784 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/e6eec7683022e6824acc16f4c26421a3804e27e6/centos/9/flavors/debug/ 2026-02-25T09:11:57.784 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5536.ge6eec768 2026-02-25T09:11:57.793 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/e6eec7683022e6824acc16f4c26421a3804e27e6/centos/9/flavors/debug/ 2026-02-25T09:11:57.794 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5536.ge6eec768 2026-02-25T09:11:58.129 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/e6eec7683022e6824acc16f4c26421a3804e27e6/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/e6eec7683022e6824acc16f4c26421a3804e27e6/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/e6eec7683022e6824acc16f4c26421a3804e27e6/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T09:11:58.129 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:11:58.129 DEBUG:teuthology.orchestra.run.trial094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T09:11:58.155 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-25T09:11:58.156 DEBUG:teuthology.orchestra.run.trial094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e6eec7683022e6824acc16f4c26421a3804e27e6/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T09:11:58.171 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/e6eec7683022e6824acc16f4c26421a3804e27e6/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/e6eec7683022e6824acc16f4c26421a3804e27e6/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/e6eec7683022e6824acc16f4c26421a3804e27e6/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T09:11:58.172 DEBUG:teuthology.orchestra.run.trial151:> set -ex 2026-02-25T09:11:58.172 DEBUG:teuthology.orchestra.run.trial151:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T09:11:58.200 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd, ceph-osd-crimson, ceph-osd-crimson-debuginfo on remote rpm x86_64 2026-02-25T09:11:58.200 DEBUG:teuthology.orchestra.run.trial151:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e6eec7683022e6824acc16f4c26421a3804e27e6/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T09:11:58.221 DEBUG:teuthology.orchestra.run.trial094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T09:11:58.264 DEBUG:teuthology.orchestra.run.trial151:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T09:11:58.294 DEBUG:teuthology.orchestra.run.trial094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T09:11:58.338 DEBUG:teuthology.orchestra.run.trial151:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T09:11:58.357 INFO:teuthology.orchestra.run.trial094.stdout:check_obsoletes = 1 2026-02-25T09:11:58.359 DEBUG:teuthology.orchestra.run.trial094:> sudo yum clean all 2026-02-25T09:11:58.402 INFO:teuthology.orchestra.run.trial151.stdout:check_obsoletes = 1 2026-02-25T09:11:58.403 DEBUG:teuthology.orchestra.run.trial151:> sudo yum clean all 2026-02-25T09:11:58.507 INFO:teuthology.orchestra.run.trial094.stdout:41 files removed 2026-02-25T09:11:58.520 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-radosgw 2026-02-25T09:11:58.549 INFO:teuthology.orchestra.run.trial151.stdout:41 files removed 2026-02-25T09:11:58.563 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-radosgw 2026-02-25T09:11:59.373 INFO:teuthology.orchestra.run.trial094.stdout:ceph packages for x86_64 130 kB/s | 92 kB 00:00 2026-02-25T09:11:59.416 INFO:teuthology.orchestra.run.trial151.stdout:ceph packages for x86_64 130 kB/s | 92 kB 00:00 2026-02-25T09:12:00.040 INFO:teuthology.orchestra.run.trial094.stdout:ceph noarch packages 37 kB/s | 24 kB 00:00 2026-02-25T09:12:00.078 INFO:teuthology.orchestra.run.trial151.stdout:ceph noarch packages 37 kB/s | 24 kB 00:00 2026-02-25T09:12:00.633 INFO:teuthology.orchestra.run.trial094.stdout:ceph source packages 3.6 kB/s | 2.1 kB 00:00 2026-02-25T09:12:00.674 INFO:teuthology.orchestra.run.trial151.stdout:ceph source packages 3.6 kB/s | 2.1 kB 00:00 2026-02-25T09:12:01.503 INFO:teuthology.orchestra.run.trial151.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-25T09:12:03.965 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - BaseOS 2.7 MB/s | 8.9 MB 00:03 2026-02-25T09:12:04.571 INFO:teuthology.orchestra.run.trial151.stdout:CentOS Stream 9 - AppStream 11 MB/s | 27 MB 00:02 2026-02-25T09:12:06.649 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - AppStream 13 MB/s | 27 MB 00:02 2026-02-25T09:12:07.843 INFO:teuthology.orchestra.run.trial151.stdout:CentOS Stream 9 - CRB 8.9 MB/s | 8.0 MB 00:00 2026-02-25T09:12:08.695 INFO:teuthology.orchestra.run.trial151.stdout:CentOS Stream 9 - Extras packages 174 kB/s | 20 kB 00:00 2026-02-25T09:12:09.310 INFO:teuthology.orchestra.run.trial151.stdout:Extra Packages for Enterprise Linux 37 MB/s | 20 MB 00:00 2026-02-25T09:12:12.322 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - CRB 2.5 MB/s | 8.0 MB 00:03 2026-02-25T09:12:12.559 INFO:teuthology.orchestra.run.trial151.stdout:lab-extras 2.6 MB/s | 50 kB 00:00 2026-02-25T09:12:13.269 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - Extras packages 104 kB/s | 20 kB 00:00 2026-02-25T09:12:13.775 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:12:13.777 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:12:13.777 INFO:teuthology.orchestra.run.trial151.stdout: Package Arch Version Repository Size 2026-02-25T09:12:13.777 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:12:13.777 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:12:13.778 INFO:teuthology.orchestra.run.trial151.stdout: ceph-radosgw x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 11 M 2026-02-25T09:12:13.778 INFO:teuthology.orchestra.run.trial151.stdout:Upgrading: 2026-02-25T09:12:13.778 INFO:teuthology.orchestra.run.trial151.stdout: librados2 x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 3.3 M 2026-02-25T09:12:13.778 INFO:teuthology.orchestra.run.trial151.stdout: librbd1 x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 2.7 M 2026-02-25T09:12:13.778 INFO:teuthology.orchestra.run.trial151.stdout:Installing dependencies: 2026-02-25T09:12:13.778 INFO:teuthology.orchestra.run.trial151.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-25T09:12:13.778 INFO:teuthology.orchestra.run.trial151.stdout: ceph-base x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 5.1 M 2026-02-25T09:12:13.778 INFO:teuthology.orchestra.run.trial151.stdout: ceph-common x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 21 M 2026-02-25T09:12:13.778 INFO:teuthology.orchestra.run.trial151.stdout: ceph-selinux x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 25 k 2026-02-25T09:12:13.778 INFO:teuthology.orchestra.run.trial151.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-25T09:12:13.779 INFO:teuthology.orchestra.run.trial151.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-25T09:12:13.779 INFO:teuthology.orchestra.run.trial151.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-25T09:12:13.779 INFO:teuthology.orchestra.run.trial151.stdout: libcephfs2 x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 860 k 2026-02-25T09:12:13.779 INFO:teuthology.orchestra.run.trial151.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-25T09:12:13.779 INFO:teuthology.orchestra.run.trial151.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-25T09:12:13.779 INFO:teuthology.orchestra.run.trial151.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-25T09:12:13.779 INFO:teuthology.orchestra.run.trial151.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-25T09:12:13.779 INFO:teuthology.orchestra.run.trial151.stdout: libradosstriper1 x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 263 k 2026-02-25T09:12:13.779 INFO:teuthology.orchestra.run.trial151.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-25T09:12:13.779 INFO:teuthology.orchestra.run.trial151.stdout: librgw2 x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 5.7 M 2026-02-25T09:12:13.780 INFO:teuthology.orchestra.run.trial151.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-25T09:12:13.780 INFO:teuthology.orchestra.run.trial151.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-25T09:12:13.780 INFO:teuthology.orchestra.run.trial151.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-25T09:12:13.780 INFO:teuthology.orchestra.run.trial151.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-25T09:12:13.780 INFO:teuthology.orchestra.run.trial151.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-25T09:12:13.780 INFO:teuthology.orchestra.run.trial151.stdout: python3-ceph-argparse x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 45 k 2026-02-25T09:12:13.780 INFO:teuthology.orchestra.run.trial151.stdout: python3-ceph-common x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 186 k 2026-02-25T09:12:13.780 INFO:teuthology.orchestra.run.trial151.stdout: python3-cephfs x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 203 k 2026-02-25T09:12:13.780 INFO:teuthology.orchestra.run.trial151.stdout: python3-rados x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 324 k 2026-02-25T09:12:13.780 INFO:teuthology.orchestra.run.trial151.stdout: python3-rbd x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 305 k 2026-02-25T09:12:13.780 INFO:teuthology.orchestra.run.trial151.stdout: python3-rgw x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 100 k 2026-02-25T09:12:13.780 INFO:teuthology.orchestra.run.trial151.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-25T09:12:13.781 INFO:teuthology.orchestra.run.trial151.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-25T09:12:13.781 INFO:teuthology.orchestra.run.trial151.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-25T09:12:13.781 INFO:teuthology.orchestra.run.trial151.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-25T09:12:13.781 INFO:teuthology.orchestra.run.trial151.stdout:Installing weak dependencies: 2026-02-25T09:12:13.781 INFO:teuthology.orchestra.run.trial151.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-25T09:12:13.781 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:12:13.781 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:12:13.781 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:12:13.781 INFO:teuthology.orchestra.run.trial151.stdout:Install 32 Packages 2026-02-25T09:12:13.781 INFO:teuthology.orchestra.run.trial151.stdout:Upgrade 2 Packages 2026-02-25T09:12:13.781 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:12:13.782 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 60 M 2026-02-25T09:12:13.782 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:12:14.513 INFO:teuthology.orchestra.run.trial094.stdout:Extra Packages for Enterprise Linux 17 MB/s | 20 MB 00:01 2026-02-25T09:12:17.756 INFO:teuthology.orchestra.run.trial094.stdout:lab-extras 1.7 MB/s | 50 kB 00:00 2026-02-25T09:12:18.967 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:12:18.968 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:12:18.968 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repository Size 2026-02-25T09:12:18.968 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:12:18.968 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:12:18.969 INFO:teuthology.orchestra.run.trial094.stdout: ceph-radosgw x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 11 M 2026-02-25T09:12:18.969 INFO:teuthology.orchestra.run.trial094.stdout:Upgrading: 2026-02-25T09:12:18.969 INFO:teuthology.orchestra.run.trial094.stdout: librados2 x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 3.3 M 2026-02-25T09:12:18.969 INFO:teuthology.orchestra.run.trial094.stdout: librbd1 x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 2.7 M 2026-02-25T09:12:18.969 INFO:teuthology.orchestra.run.trial094.stdout:Installing dependencies: 2026-02-25T09:12:18.969 INFO:teuthology.orchestra.run.trial094.stdout: boost-program-options x86_64 1.75.0-13.el9 appstream 104 k 2026-02-25T09:12:18.969 INFO:teuthology.orchestra.run.trial094.stdout: ceph-base x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 5.1 M 2026-02-25T09:12:18.969 INFO:teuthology.orchestra.run.trial094.stdout: ceph-common x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 21 M 2026-02-25T09:12:18.969 INFO:teuthology.orchestra.run.trial094.stdout: ceph-selinux x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 25 k 2026-02-25T09:12:18.969 INFO:teuthology.orchestra.run.trial094.stdout: gperftools-libs x86_64 2.9.1-3.el9 epel 308 k 2026-02-25T09:12:18.969 INFO:teuthology.orchestra.run.trial094.stdout: libarrow x86_64 9.0.0-15.el9 epel 4.4 M 2026-02-25T09:12:18.969 INFO:teuthology.orchestra.run.trial094.stdout: libarrow-doc noarch 9.0.0-15.el9 epel 25 k 2026-02-25T09:12:18.969 INFO:teuthology.orchestra.run.trial094.stdout: libcephfs2 x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 860 k 2026-02-25T09:12:18.970 INFO:teuthology.orchestra.run.trial094.stdout: libnbd x86_64 1.20.3-4.el9 appstream 164 k 2026-02-25T09:12:18.970 INFO:teuthology.orchestra.run.trial094.stdout: liboath x86_64 2.6.12-1.el9 epel 49 k 2026-02-25T09:12:18.970 INFO:teuthology.orchestra.run.trial094.stdout: libpmemobj x86_64 1.12.1-1.el9 appstream 160 k 2026-02-25T09:12:18.970 INFO:teuthology.orchestra.run.trial094.stdout: librabbitmq x86_64 0.11.0-7.el9 appstream 45 k 2026-02-25T09:12:18.970 INFO:teuthology.orchestra.run.trial094.stdout: libradosstriper1 x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 263 k 2026-02-25T09:12:18.970 INFO:teuthology.orchestra.run.trial094.stdout: librdkafka x86_64 1.6.1-102.el9 appstream 662 k 2026-02-25T09:12:18.970 INFO:teuthology.orchestra.run.trial094.stdout: librgw2 x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 5.7 M 2026-02-25T09:12:18.970 INFO:teuthology.orchestra.run.trial094.stdout: libunwind x86_64 1.6.2-1.el9 epel 67 k 2026-02-25T09:12:18.970 INFO:teuthology.orchestra.run.trial094.stdout: lttng-ust x86_64 2.12.0-6.el9 appstream 292 k 2026-02-25T09:12:18.970 INFO:teuthology.orchestra.run.trial094.stdout: mailcap noarch 2.1.49-5.el9 baseos 33 k 2026-02-25T09:12:18.970 INFO:teuthology.orchestra.run.trial094.stdout: parquet-libs x86_64 9.0.0-15.el9 epel 838 k 2026-02-25T09:12:18.970 INFO:teuthology.orchestra.run.trial094.stdout: pciutils x86_64 3.7.0-7.el9 baseos 93 k 2026-02-25T09:12:18.971 INFO:teuthology.orchestra.run.trial094.stdout: python3-ceph-argparse x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 45 k 2026-02-25T09:12:18.971 INFO:teuthology.orchestra.run.trial094.stdout: python3-ceph-common x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 186 k 2026-02-25T09:12:18.971 INFO:teuthology.orchestra.run.trial094.stdout: python3-cephfs x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 203 k 2026-02-25T09:12:18.971 INFO:teuthology.orchestra.run.trial094.stdout: python3-rados x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 324 k 2026-02-25T09:12:18.971 INFO:teuthology.orchestra.run.trial094.stdout: python3-rbd x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 305 k 2026-02-25T09:12:18.971 INFO:teuthology.orchestra.run.trial094.stdout: python3-rgw x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 100 k 2026-02-25T09:12:18.971 INFO:teuthology.orchestra.run.trial094.stdout: qatlib x86_64 25.08.0-2.el9 appstream 240 k 2026-02-25T09:12:18.971 INFO:teuthology.orchestra.run.trial094.stdout: qatzip-libs x86_64 1.3.1-1.el9 appstream 66 k 2026-02-25T09:12:18.971 INFO:teuthology.orchestra.run.trial094.stdout: re2 x86_64 1:20211101-20.el9 epel 191 k 2026-02-25T09:12:18.971 INFO:teuthology.orchestra.run.trial094.stdout: thrift x86_64 0.15.0-4.el9 epel 1.6 M 2026-02-25T09:12:18.971 INFO:teuthology.orchestra.run.trial094.stdout:Installing weak dependencies: 2026-02-25T09:12:18.971 INFO:teuthology.orchestra.run.trial094.stdout: qatlib-service x86_64 25.08.0-2.el9 appstream 37 k 2026-02-25T09:12:18.971 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:12:18.972 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:12:18.972 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:12:18.972 INFO:teuthology.orchestra.run.trial094.stdout:Install 32 Packages 2026-02-25T09:12:18.972 INFO:teuthology.orchestra.run.trial094.stdout:Upgrade 2 Packages 2026-02-25T09:12:18.972 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:12:18.973 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 60 M 2026-02-25T09:12:18.973 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:12:22.064 INFO:teuthology.orchestra.run.trial151.stdout:(1/34): ceph-base-20.3.0-5536.ge6eec768.el9.x86 662 kB/s | 5.1 MB 00:07 2026-02-25T09:12:22.133 INFO:teuthology.orchestra.run.trial151.stdout:(2/34): ceph-selinux-20.3.0-5536.ge6eec768.el9. 366 kB/s | 25 kB 00:00 2026-02-25T09:12:23.570 INFO:teuthology.orchestra.run.trial151.stdout:(3/34): libcephfs2-20.3.0-5536.ge6eec768.el9.x8 598 kB/s | 860 kB 00:01 2026-02-25T09:12:24.056 INFO:teuthology.orchestra.run.trial151.stdout:(4/34): libradosstriper1-20.3.0-5536.ge6eec768. 542 kB/s | 263 kB 00:00 2026-02-25T09:12:28.276 INFO:teuthology.orchestra.run.trial094.stdout:(1/34): ceph-radosgw-20.3.0-5536.ge6eec768.el9. 1.3 MB/s | 11 MB 00:08 2026-02-25T09:12:28.345 INFO:teuthology.orchestra.run.trial094.stdout:(2/34): ceph-selinux-20.3.0-5536.ge6eec768.el9. 367 kB/s | 25 kB 00:00 2026-02-25T09:12:29.178 INFO:teuthology.orchestra.run.trial094.stdout:(3/34): libcephfs2-20.3.0-5536.ge6eec768.el9.x8 1.0 MB/s | 860 kB 00:00 2026-02-25T09:12:29.458 INFO:teuthology.orchestra.run.trial094.stdout:(4/34): libradosstriper1-20.3.0-5536.ge6eec768. 939 kB/s | 263 kB 00:00 2026-02-25T09:12:31.404 INFO:teuthology.orchestra.run.trial094.stdout:(5/34): ceph-base-20.3.0-5536.ge6eec768.el9.x86 436 kB/s | 5.1 MB 00:11 2026-02-25T09:12:31.546 INFO:teuthology.orchestra.run.trial094.stdout:(6/34): python3-ceph-argparse-20.3.0-5536.ge6ee 319 kB/s | 45 kB 00:00 2026-02-25T09:12:31.897 INFO:teuthology.orchestra.run.trial094.stdout:(7/34): python3-ceph-common-20.3.0-5536.ge6eec7 531 kB/s | 186 kB 00:00 2026-02-25T09:12:32.247 INFO:teuthology.orchestra.run.trial094.stdout:(8/34): python3-cephfs-20.3.0-5536.ge6eec768.el 580 kB/s | 203 kB 00:00 2026-02-25T09:12:32.741 INFO:teuthology.orchestra.run.trial094.stdout:(9/34): python3-rados-20.3.0-5536.ge6eec768.el9 656 kB/s | 324 kB 00:00 2026-02-25T09:12:33.229 INFO:teuthology.orchestra.run.trial094.stdout:(10/34): python3-rbd-20.3.0-5536.ge6eec768.el9. 626 kB/s | 305 kB 00:00 2026-02-25T09:12:33.368 INFO:teuthology.orchestra.run.trial094.stdout:(11/34): python3-rgw-20.3.0-5536.ge6eec768.el9. 720 kB/s | 100 kB 00:00 2026-02-25T09:12:33.468 INFO:teuthology.orchestra.run.trial094.stdout:(12/34): mailcap-2.1.49-5.el9.noarch.rpm 334 kB/s | 33 kB 00:00 2026-02-25T09:12:33.522 INFO:teuthology.orchestra.run.trial094.stdout:(13/34): librgw2-20.3.0-5536.ge6eec768.el9.x86_ 1.4 MB/s | 5.7 MB 00:04 2026-02-25T09:12:33.529 INFO:teuthology.orchestra.run.trial094.stdout:(14/34): pciutils-3.7.0-7.el9.x86_64.rpm 1.5 MB/s | 93 kB 00:00 2026-02-25T09:12:33.620 INFO:teuthology.orchestra.run.trial094.stdout:(15/34): boost-program-options-1.75.0-13.el9.x8 1.0 MB/s | 104 kB 00:00 2026-02-25T09:12:33.680 INFO:teuthology.orchestra.run.trial094.stdout:(16/34): libpmemobj-1.12.1-1.el9.x86_64.rpm 2.6 MB/s | 160 kB 00:00 2026-02-25T09:12:33.735 INFO:teuthology.orchestra.run.trial094.stdout:(17/34): librabbitmq-0.11.0-7.el9.x86_64.rpm 832 kB/s | 45 kB 00:00 2026-02-25T09:12:33.794 INFO:teuthology.orchestra.run.trial094.stdout:(18/34): librdkafka-1.6.1-102.el9.x86_64.rpm 11 MB/s | 662 kB 00:00 2026-02-25T09:12:33.810 INFO:teuthology.orchestra.run.trial094.stdout:(19/34): libnbd-1.20.3-4.el9.x86_64.rpm 581 kB/s | 164 kB 00:00 2026-02-25T09:12:33.840 INFO:teuthology.orchestra.run.trial094.stdout:(20/34): qatlib-25.08.0-2.el9.x86_64.rpm 8.1 MB/s | 240 kB 00:00 2026-02-25T09:12:33.864 INFO:teuthology.orchestra.run.trial094.stdout:(21/34): qatlib-service-25.08.0-2.el9.x86_64.rp 1.5 MB/s | 37 kB 00:00 2026-02-25T09:12:33.897 INFO:teuthology.orchestra.run.trial094.stdout:(22/34): qatzip-libs-1.3.1-1.el9.x86_64.rpm 2.0 MB/s | 66 kB 00:00 2026-02-25T09:12:33.901 INFO:teuthology.orchestra.run.trial094.stdout:(23/34): lttng-ust-2.12.0-6.el9.x86_64.rpm 2.7 MB/s | 292 kB 00:00 2026-02-25T09:12:34.142 INFO:teuthology.orchestra.run.trial094.stdout:(24/34): gperftools-libs-2.9.1-3.el9.x86_64.rpm 1.2 MB/s | 308 kB 00:00 2026-02-25T09:12:34.172 INFO:teuthology.orchestra.run.trial094.stdout:(25/34): libarrow-doc-9.0.0-15.el9.noarch.rpm 815 kB/s | 25 kB 00:00 2026-02-25T09:12:34.205 INFO:teuthology.orchestra.run.trial094.stdout:(26/34): liboath-2.6.12-1.el9.x86_64.rpm 1.5 MB/s | 49 kB 00:00 2026-02-25T09:12:34.238 INFO:teuthology.orchestra.run.trial094.stdout:(27/34): libunwind-1.6.2-1.el9.x86_64.rpm 2.0 MB/s | 67 kB 00:00 2026-02-25T09:12:34.259 INFO:teuthology.orchestra.run.trial094.stdout:(28/34): libarrow-9.0.0-15.el9.x86_64.rpm 12 MB/s | 4.4 MB 00:00 2026-02-25T09:12:34.290 INFO:teuthology.orchestra.run.trial094.stdout:(29/34): re2-20211101-20.el9.x86_64.rpm 6.0 MB/s | 191 kB 00:00 2026-02-25T09:12:34.324 INFO:teuthology.orchestra.run.trial094.stdout:(30/34): parquet-libs-9.0.0-15.el9.x86_64.rpm 9.6 MB/s | 838 kB 00:00 2026-02-25T09:12:34.329 INFO:teuthology.orchestra.run.trial094.stdout:(31/34): thrift-0.15.0-4.el9.x86_64.rpm 41 MB/s | 1.6 MB 00:00 2026-02-25T09:12:34.396 INFO:teuthology.orchestra.run.trial151.stdout:(5/34): librgw2-20.3.0-5536.ge6eec768.el9.x86_6 562 kB/s | 5.7 MB 00:10 2026-02-25T09:12:34.674 INFO:teuthology.orchestra.run.trial151.stdout:(6/34): python3-ceph-argparse-20.3.0-5536.ge6ee 162 kB/s | 45 kB 00:00 2026-02-25T09:12:34.839 INFO:teuthology.orchestra.run.trial151.stdout:(7/34): ceph-radosgw-20.3.0-5536.ge6eec768.el9. 558 kB/s | 11 MB 00:20 2026-02-25T09:12:35.118 INFO:teuthology.orchestra.run.trial151.stdout:(8/34): python3-cephfs-20.3.0-5536.ge6eec768.el 732 kB/s | 203 kB 00:00 2026-02-25T09:12:35.226 INFO:teuthology.orchestra.run.trial151.stdout:(9/34): python3-ceph-common-20.3.0-5536.ge6eec7 338 kB/s | 186 kB 00:00 2026-02-25T09:12:35.465 INFO:teuthology.orchestra.run.trial151.stdout:(10/34): python3-rados-20.3.0-5536.ge6eec768.el 931 kB/s | 324 kB 00:00 2026-02-25T09:12:35.604 INFO:teuthology.orchestra.run.trial151.stdout:(11/34): python3-rgw-20.3.0-5536.ge6eec768.el9. 727 kB/s | 100 kB 00:00 2026-02-25T09:12:35.874 INFO:teuthology.orchestra.run.trial151.stdout:(12/34): ceph-common-20.3.0-5536.ge6eec768.el9. 986 kB/s | 21 MB 00:21 2026-02-25T09:12:35.983 INFO:teuthology.orchestra.run.trial151.stdout:(13/34): python3-rbd-20.3.0-5536.ge6eec768.el9. 402 kB/s | 305 kB 00:00 2026-02-25T09:12:36.024 INFO:teuthology.orchestra.run.trial151.stdout:(14/34): boost-program-options-1.75.0-13.el9.x8 2.5 MB/s | 104 kB 00:00 2026-02-25T09:12:36.041 INFO:teuthology.orchestra.run.trial151.stdout:(15/34): libnbd-1.20.3-4.el9.x86_64.rpm 9.2 MB/s | 164 kB 00:00 2026-02-25T09:12:36.053 INFO:teuthology.orchestra.run.trial151.stdout:(16/34): libpmemobj-1.12.1-1.el9.x86_64.rpm 14 MB/s | 160 kB 00:00 2026-02-25T09:12:36.062 INFO:teuthology.orchestra.run.trial151.stdout:(17/34): librabbitmq-0.11.0-7.el9.x86_64.rpm 4.8 MB/s | 45 kB 00:00 2026-02-25T09:12:36.083 INFO:teuthology.orchestra.run.trial151.stdout:(18/34): librdkafka-1.6.1-102.el9.x86_64.rpm 31 MB/s | 662 kB 00:00 2026-02-25T09:12:36.094 INFO:teuthology.orchestra.run.trial151.stdout:(19/34): lttng-ust-2.12.0-6.el9.x86_64.rpm 26 MB/s | 292 kB 00:00 2026-02-25T09:12:36.105 INFO:teuthology.orchestra.run.trial151.stdout:(20/34): qatlib-25.08.0-2.el9.x86_64.rpm 22 MB/s | 240 kB 00:00 2026-02-25T09:12:36.114 INFO:teuthology.orchestra.run.trial151.stdout:(21/34): qatlib-service-25.08.0-2.el9.x86_64.rp 4.0 MB/s | 37 kB 00:00 2026-02-25T09:12:36.124 INFO:teuthology.orchestra.run.trial151.stdout:(22/34): qatzip-libs-1.3.1-1.el9.x86_64.rpm 7.1 MB/s | 66 kB 00:00 2026-02-25T09:12:36.171 INFO:teuthology.orchestra.run.trial151.stdout:(23/34): mailcap-2.1.49-5.el9.noarch.rpm 58 kB/s | 33 kB 00:00 2026-02-25T09:12:36.522 INFO:teuthology.orchestra.run.trial151.stdout:(24/34): pciutils-3.7.0-7.el9.x86_64.rpm 144 kB/s | 93 kB 00:00 2026-02-25T09:12:36.630 INFO:teuthology.orchestra.run.trial151.stdout:(25/34): gperftools-libs-2.9.1-3.el9.x86_64.rpm 608 kB/s | 308 kB 00:00 2026-02-25T09:12:36.630 INFO:teuthology.orchestra.run.trial151.stdout:(26/34): libarrow-doc-9.0.0-15.el9.noarch.rpm 229 kB/s | 25 kB 00:00 2026-02-25T09:12:36.702 INFO:teuthology.orchestra.run.trial151.stdout:(27/34): liboath-2.6.12-1.el9.x86_64.rpm 677 kB/s | 49 kB 00:00 2026-02-25T09:12:36.703 INFO:teuthology.orchestra.run.trial151.stdout:(28/34): libunwind-1.6.2-1.el9.x86_64.rpm 917 kB/s | 67 kB 00:00 2026-02-25T09:12:36.815 INFO:teuthology.orchestra.run.trial094.stdout:(32/34): librbd1-20.3.0-5536.ge6eec768.el9.x86_ 1.1 MB/s | 2.7 MB 00:02 2026-02-25T09:12:36.893 INFO:teuthology.orchestra.run.trial151.stdout:(29/34): parquet-libs-9.0.0-15.el9.x86_64.rpm 4.3 MB/s | 838 kB 00:00 2026-02-25T09:12:36.911 INFO:teuthology.orchestra.run.trial151.stdout:(30/34): re2-20211101-20.el9.x86_64.rpm 921 kB/s | 191 kB 00:00 2026-02-25T09:12:36.970 INFO:teuthology.orchestra.run.trial151.stdout:(31/34): libarrow-9.0.0-15.el9.x86_64.rpm 5.5 MB/s | 4.4 MB 00:00 2026-02-25T09:12:37.042 INFO:teuthology.orchestra.run.trial151.stdout:(32/34): thrift-0.15.0-4.el9.x86_64.rpm 11 MB/s | 1.6 MB 00:00 2026-02-25T09:12:39.705 INFO:teuthology.orchestra.run.trial094.stdout:(33/34): librados2-20.3.0-5536.ge6eec768.el9.x8 625 kB/s | 3.3 MB 00:05 2026-02-25T09:12:39.929 INFO:teuthology.orchestra.run.trial151.stdout:(33/34): librbd1-20.3.0-5536.ge6eec768.el9.x86_ 930 kB/s | 2.7 MB 00:02 2026-02-25T09:12:41.127 INFO:teuthology.orchestra.run.trial151.stdout:(34/34): librados2-20.3.0-5536.ge6eec768.el9.x8 798 kB/s | 3.3 MB 00:04 2026-02-25T09:12:41.131 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:12:41.131 INFO:teuthology.orchestra.run.trial151.stdout:Total 2.2 MB/s | 60 MB 00:27 2026-02-25T09:12:41.217 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:12:41.234 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:12:41.234 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:12:41.475 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:12:41.475 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:12:41.758 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:12:41.896 INFO:teuthology.orchestra.run.trial151.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/36 2026-02-25T09:12:41.901 INFO:teuthology.orchestra.run.trial151.stdout: Upgrading : librados2-2:20.3.0-5536.ge6eec768.el9.x86_64 2/36 2026-02-25T09:12:41.930 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: librados2-2:20.3.0-5536.ge6eec768.el9.x86_64 2/36 2026-02-25T09:12:41.966 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-rados-2:20.3.0-5536.ge6eec768.el9.x86_64 3/36 2026-02-25T09:12:41.967 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libcephfs2-2:20.3.0-5536.ge6eec768.el9.x86_64 4/36 2026-02-25T09:12:41.978 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: libcephfs2-2:20.3.0-5536.ge6eec768.el9.x86_64 4/36 2026-02-25T09:12:41.985 INFO:teuthology.orchestra.run.trial151.stdout: Installing : liboath-2.6.12-1.el9.x86_64 5/36 2026-02-25T09:12:41.988 INFO:teuthology.orchestra.run.trial151.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/36 2026-02-25T09:12:41.991 INFO:teuthology.orchestra.run.trial151.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/36 2026-02-25T09:12:41.995 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 8/36 2026-02-25T09:12:42.006 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-ceph-argparse-2:20.3.0-5536.ge6eec768.el9. 9/36 2026-02-25T09:12:42.019 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-cephfs-2:20.3.0-5536.ge6eec768.el9.x86_64 10/36 2026-02-25T09:12:42.020 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libradosstriper1-2:20.3.0-5536.ge6eec768.el9.x86_6 11/36 2026-02-25T09:12:42.042 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5536.ge6eec768.el9.x86_6 11/36 2026-02-25T09:12:42.046 INFO:teuthology.orchestra.run.trial151.stdout: Installing : thrift-0.15.0-4.el9.x86_64 12/36 2026-02-25T09:12:42.078 INFO:teuthology.orchestra.run.trial151.stdout: Installing : re2-1:20211101-20.el9.x86_64 13/36 2026-02-25T09:12:42.083 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 14/36 2026-02-25T09:12:42.089 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 15/36 2026-02-25T09:12:42.091 INFO:teuthology.orchestra.run.trial151.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 16/36 2026-02-25T09:12:42.096 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 17/36 2026-02-25T09:12:42.211 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/36 2026-02-25T09:12:42.212 INFO:teuthology.orchestra.run.trial151.stdout: Upgrading : librbd1-2:20.3.0-5536.ge6eec768.el9.x86_64 19/36 2026-02-25T09:12:42.239 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: librbd1-2:20.3.0-5536.ge6eec768.el9.x86_64 19/36 2026-02-25T09:12:42.241 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-rbd-2:20.3.0-5536.ge6eec768.el9.x86_64 20/36 2026-02-25T09:12:42.247 INFO:teuthology.orchestra.run.trial151.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/36 2026-02-25T09:12:42.496 INFO:teuthology.orchestra.run.trial151.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/36 2026-02-25T09:12:42.497 INFO:teuthology.orchestra.run.trial151.stdout: Installing : librgw2-2:20.3.0-5536.ge6eec768.el9.x86_64 23/36 2026-02-25T09:12:42.514 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: librgw2-2:20.3.0-5536.ge6eec768.el9.x86_64 23/36 2026-02-25T09:12:42.519 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-rgw-2:20.3.0-5536.ge6eec768.el9.x86_64 24/36 2026-02-25T09:12:42.523 INFO:teuthology.orchestra.run.trial151.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/36 2026-02-25T09:12:42.525 INFO:teuthology.orchestra.run.trial151.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/36 2026-02-25T09:12:42.526 INFO:teuthology.orchestra.run.trial151.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-25T09:12:42.542 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-25T09:12:42.545 INFO:teuthology.orchestra.run.trial151.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/36 2026-02-25T09:12:42.579 INFO:teuthology.orchestra.run.trial151.stdout: Installing : mailcap-2.1.49-5.el9.noarch 29/36 2026-02-25T09:12:42.581 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-ceph-common-2:20.3.0-5536.ge6eec768.el9.x8 30/36 2026-02-25T09:12:43.523 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-common-2:20.3.0-5536.ge6eec768.el9.x86_64 31/36 2026-02-25T09:12:43.525 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-common-2:20.3.0-5536.ge6eec768.el9.x86_64 31/36 2026-02-25T09:12:43.784 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-common-2:20.3.0-5536.ge6eec768.el9.x86_64 31/36 2026-02-25T09:12:43.788 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-base-2:20.3.0-5536.ge6eec768.el9.x86_64 32/36 2026-02-25T09:12:43.828 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-base-2:20.3.0-5536.ge6eec768.el9.x86_64 32/36 2026-02-25T09:12:43.828 INFO:teuthology.orchestra.run.trial151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-25T09:12:43.828 INFO:teuthology.orchestra.run.trial151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-25T09:12:43.828 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:12:43.831 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-selinux-2:20.3.0-5536.ge6eec768.el9.x86_64 33/36 2026-02-25T09:12:45.225 INFO:teuthology.orchestra.run.trial094.stdout:(34/34): ceph-common-20.3.0-5536.ge6eec768.el9. 829 kB/s | 21 MB 00:25 2026-02-25T09:12:45.228 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:12:45.228 INFO:teuthology.orchestra.run.trial094.stdout:Total 2.3 MB/s | 60 MB 00:26 2026-02-25T09:12:45.304 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:12:45.320 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:12:45.320 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:12:45.559 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:12:45.559 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:12:45.840 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:12:45.979 INFO:teuthology.orchestra.run.trial094.stdout: Installing : lttng-ust-2.12.0-6.el9.x86_64 1/36 2026-02-25T09:12:45.983 INFO:teuthology.orchestra.run.trial094.stdout: Upgrading : librados2-2:20.3.0-5536.ge6eec768.el9.x86_64 2/36 2026-02-25T09:12:46.011 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: librados2-2:20.3.0-5536.ge6eec768.el9.x86_64 2/36 2026-02-25T09:12:46.047 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-rados-2:20.3.0-5536.ge6eec768.el9.x86_64 3/36 2026-02-25T09:12:46.048 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libcephfs2-2:20.3.0-5536.ge6eec768.el9.x86_64 4/36 2026-02-25T09:12:46.059 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: libcephfs2-2:20.3.0-5536.ge6eec768.el9.x86_64 4/36 2026-02-25T09:12:46.067 INFO:teuthology.orchestra.run.trial094.stdout: Installing : liboath-2.6.12-1.el9.x86_64 5/36 2026-02-25T09:12:46.070 INFO:teuthology.orchestra.run.trial094.stdout: Installing : librdkafka-1.6.1-102.el9.x86_64 6/36 2026-02-25T09:12:46.073 INFO:teuthology.orchestra.run.trial094.stdout: Installing : librabbitmq-0.11.0-7.el9.x86_64 7/36 2026-02-25T09:12:46.077 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libpmemobj-1.12.1-1.el9.x86_64 8/36 2026-02-25T09:12:46.088 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-ceph-argparse-2:20.3.0-5536.ge6eec768.el9. 9/36 2026-02-25T09:12:46.101 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-cephfs-2:20.3.0-5536.ge6eec768.el9.x86_64 10/36 2026-02-25T09:12:46.102 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libradosstriper1-2:20.3.0-5536.ge6eec768.el9.x86_6 11/36 2026-02-25T09:12:46.122 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: libradosstriper1-2:20.3.0-5536.ge6eec768.el9.x86_6 11/36 2026-02-25T09:12:46.125 INFO:teuthology.orchestra.run.trial094.stdout: Installing : thrift-0.15.0-4.el9.x86_64 12/36 2026-02-25T09:12:46.156 INFO:teuthology.orchestra.run.trial094.stdout: Installing : re2-1:20211101-20.el9.x86_64 13/36 2026-02-25T09:12:46.160 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libarrow-9.0.0-15.el9.x86_64 14/36 2026-02-25T09:12:46.166 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libunwind-1.6.2-1.el9.x86_64 15/36 2026-02-25T09:12:46.168 INFO:teuthology.orchestra.run.trial094.stdout: Installing : gperftools-libs-2.9.1-3.el9.x86_64 16/36 2026-02-25T09:12:46.172 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libarrow-doc-9.0.0-15.el9.noarch 17/36 2026-02-25T09:12:46.286 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libnbd-1.20.3-4.el9.x86_64 18/36 2026-02-25T09:12:46.287 INFO:teuthology.orchestra.run.trial094.stdout: Upgrading : librbd1-2:20.3.0-5536.ge6eec768.el9.x86_64 19/36 2026-02-25T09:12:46.312 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: librbd1-2:20.3.0-5536.ge6eec768.el9.x86_64 19/36 2026-02-25T09:12:46.314 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-rbd-2:20.3.0-5536.ge6eec768.el9.x86_64 20/36 2026-02-25T09:12:46.321 INFO:teuthology.orchestra.run.trial094.stdout: Installing : boost-program-options-1.75.0-13.el9.x86_64 21/36 2026-02-25T09:12:46.571 INFO:teuthology.orchestra.run.trial094.stdout: Installing : parquet-libs-9.0.0-15.el9.x86_64 22/36 2026-02-25T09:12:46.572 INFO:teuthology.orchestra.run.trial094.stdout: Installing : librgw2-2:20.3.0-5536.ge6eec768.el9.x86_64 23/36 2026-02-25T09:12:46.588 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: librgw2-2:20.3.0-5536.ge6eec768.el9.x86_64 23/36 2026-02-25T09:12:46.592 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-rgw-2:20.3.0-5536.ge6eec768.el9.x86_64 24/36 2026-02-25T09:12:46.598 INFO:teuthology.orchestra.run.trial094.stdout: Installing : pciutils-3.7.0-7.el9.x86_64 25/36 2026-02-25T09:12:46.600 INFO:teuthology.orchestra.run.trial094.stdout: Installing : qatlib-25.08.0-2.el9.x86_64 26/36 2026-02-25T09:12:46.601 INFO:teuthology.orchestra.run.trial094.stdout: Installing : qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-25T09:12:46.618 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: qatlib-service-25.08.0-2.el9.x86_64 27/36 2026-02-25T09:12:46.622 INFO:teuthology.orchestra.run.trial094.stdout: Installing : qatzip-libs-1.3.1-1.el9.x86_64 28/36 2026-02-25T09:12:46.655 INFO:teuthology.orchestra.run.trial094.stdout: Installing : mailcap-2.1.49-5.el9.noarch 29/36 2026-02-25T09:12:46.657 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-ceph-common-2:20.3.0-5536.ge6eec768.el9.x8 30/36 2026-02-25T09:12:47.600 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-common-2:20.3.0-5536.ge6eec768.el9.x86_64 31/36 2026-02-25T09:12:47.602 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-common-2:20.3.0-5536.ge6eec768.el9.x86_64 31/36 2026-02-25T09:12:47.860 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-common-2:20.3.0-5536.ge6eec768.el9.x86_64 31/36 2026-02-25T09:12:47.864 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-base-2:20.3.0-5536.ge6eec768.el9.x86_64 32/36 2026-02-25T09:12:47.905 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-base-2:20.3.0-5536.ge6eec768.el9.x86_64 32/36 2026-02-25T09:12:47.906 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph.target → /usr/lib/systemd/system/ceph.target. 2026-02-25T09:12:47.906 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-crash.service → /usr/lib/systemd/system/ceph-crash.service. 2026-02-25T09:12:47.906 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:12:47.908 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-selinux-2:20.3.0-5536.ge6eec768.el9.x86_64 33/36 2026-02-25T09:12:48.287 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5536.ge6eec768.el9.x86_64 33/36 2026-02-25T09:12:48.287 INFO:teuthology.orchestra.run.trial151.stdout:skipping the directory /sys 2026-02-25T09:12:48.287 INFO:teuthology.orchestra.run.trial151.stdout:skipping the directory /proc 2026-02-25T09:12:48.287 INFO:teuthology.orchestra.run.trial151.stdout:skipping the directory /mnt 2026-02-25T09:12:48.287 INFO:teuthology.orchestra.run.trial151.stdout:skipping the directory /var/tmp 2026-02-25T09:12:48.288 INFO:teuthology.orchestra.run.trial151.stdout:skipping the directory /home 2026-02-25T09:12:48.288 INFO:teuthology.orchestra.run.trial151.stdout:skipping the directory /root 2026-02-25T09:12:48.288 INFO:teuthology.orchestra.run.trial151.stdout:skipping the directory /tmp 2026-02-25T09:12:48.288 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:12:48.781 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-radosgw-2:20.3.0-5536.ge6eec768.el9.x86_64 34/36 2026-02-25T09:12:48.804 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5536.ge6eec768.el9.x86_64 34/36 2026-02-25T09:12:48.804 INFO:teuthology.orchestra.run.trial151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T09:12:48.804 INFO:teuthology.orchestra.run.trial151.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-25T09:12:48.804 INFO:teuthology.orchestra.run.trial151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T09:12:48.804 INFO:teuthology.orchestra.run.trial151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T09:12:48.804 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:12:48.806 INFO:teuthology.orchestra.run.trial151.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-25T09:12:48.818 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-25T09:12:48.818 INFO:teuthology.orchestra.run.trial151.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-25T09:12:48.873 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-25T09:12:48.873 INFO:teuthology.orchestra.run.trial151.stdout:Creating group 'qat' with GID 992. 2026-02-25T09:12:48.873 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:12:49.160 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-base-2:20.3.0-5536.ge6eec768.el9.x86_64 1/36 2026-02-25T09:12:49.160 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-common-2:20.3.0-5536.ge6eec768.el9.x86_64 2/36 2026-02-25T09:12:49.160 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-radosgw-2:20.3.0-5536.ge6eec768.el9.x86_64 3/36 2026-02-25T09:12:49.160 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-selinux-2:20.3.0-5536.ge6eec768.el9.x86_64 4/36 2026-02-25T09:12:49.161 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libcephfs2-2:20.3.0-5536.ge6eec768.el9.x86_64 5/36 2026-02-25T09:12:49.161 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libradosstriper1-2:20.3.0-5536.ge6eec768.el9.x86_6 6/36 2026-02-25T09:12:49.161 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : librgw2-2:20.3.0-5536.ge6eec768.el9.x86_64 7/36 2026-02-25T09:12:49.161 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5536.ge6eec768.el9. 8/36 2026-02-25T09:12:49.161 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-ceph-common-2:20.3.0-5536.ge6eec768.el9.x8 9/36 2026-02-25T09:12:49.161 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-cephfs-2:20.3.0-5536.ge6eec768.el9.x86_64 10/36 2026-02-25T09:12:49.161 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-rados-2:20.3.0-5536.ge6eec768.el9.x86_64 11/36 2026-02-25T09:12:49.161 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-rbd-2:20.3.0-5536.ge6eec768.el9.x86_64 12/36 2026-02-25T09:12:49.161 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-rgw-2:20.3.0-5536.ge6eec768.el9.x86_64 13/36 2026-02-25T09:12:49.161 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/36 2026-02-25T09:12:49.161 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 15/36 2026-02-25T09:12:49.162 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 16/36 2026-02-25T09:12:49.162 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 17/36 2026-02-25T09:12:49.162 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 18/36 2026-02-25T09:12:49.162 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/36 2026-02-25T09:12:49.162 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 20/36 2026-02-25T09:12:49.162 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 21/36 2026-02-25T09:12:49.162 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 22/36 2026-02-25T09:12:49.162 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 23/36 2026-02-25T09:12:49.162 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 24/36 2026-02-25T09:12:49.162 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 25/36 2026-02-25T09:12:49.162 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 26/36 2026-02-25T09:12:49.162 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 27/36 2026-02-25T09:12:49.162 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 28/36 2026-02-25T09:12:49.163 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 29/36 2026-02-25T09:12:49.163 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 30/36 2026-02-25T09:12:49.163 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : re2-1:20211101-20.el9.x86_64 31/36 2026-02-25T09:12:49.163 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 32/36 2026-02-25T09:12:49.163 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : librados2-2:20.3.0-5536.ge6eec768.el9.x86_64 33/36 2026-02-25T09:12:49.163 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 34/36 2026-02-25T09:12:49.163 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : librbd1-2:20.3.0-5536.ge6eec768.el9.x86_64 35/36 2026-02-25T09:12:49.226 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 36/36 2026-02-25T09:12:49.226 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:12:49.226 INFO:teuthology.orchestra.run.trial151.stdout:Upgraded: 2026-02-25T09:12:49.226 INFO:teuthology.orchestra.run.trial151.stdout: librados2-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:49.226 INFO:teuthology.orchestra.run.trial151.stdout: librbd1-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:49.226 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:12:49.226 INFO:teuthology.orchestra.run.trial151.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-25T09:12:49.227 INFO:teuthology.orchestra.run.trial151.stdout: ceph-base-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:49.227 INFO:teuthology.orchestra.run.trial151.stdout: ceph-common-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:49.227 INFO:teuthology.orchestra.run.trial151.stdout: ceph-radosgw-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:49.227 INFO:teuthology.orchestra.run.trial151.stdout: ceph-selinux-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:49.227 INFO:teuthology.orchestra.run.trial151.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-25T09:12:49.227 INFO:teuthology.orchestra.run.trial151.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-25T09:12:49.227 INFO:teuthology.orchestra.run.trial151.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-25T09:12:49.227 INFO:teuthology.orchestra.run.trial151.stdout: libcephfs2-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:49.227 INFO:teuthology.orchestra.run.trial151.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-25T09:12:49.227 INFO:teuthology.orchestra.run.trial151.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-25T09:12:49.227 INFO:teuthology.orchestra.run.trial151.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-25T09:12:49.227 INFO:teuthology.orchestra.run.trial151.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-25T09:12:49.228 INFO:teuthology.orchestra.run.trial151.stdout: libradosstriper1-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:49.228 INFO:teuthology.orchestra.run.trial151.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-25T09:12:49.228 INFO:teuthology.orchestra.run.trial151.stdout: librgw2-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:49.228 INFO:teuthology.orchestra.run.trial151.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-25T09:12:49.228 INFO:teuthology.orchestra.run.trial151.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-25T09:12:49.228 INFO:teuthology.orchestra.run.trial151.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-25T09:12:49.228 INFO:teuthology.orchestra.run.trial151.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-25T09:12:49.228 INFO:teuthology.orchestra.run.trial151.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-25T09:12:49.228 INFO:teuthology.orchestra.run.trial151.stdout: python3-ceph-argparse-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:49.228 INFO:teuthology.orchestra.run.trial151.stdout: python3-ceph-common-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:49.228 INFO:teuthology.orchestra.run.trial151.stdout: python3-cephfs-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:49.228 INFO:teuthology.orchestra.run.trial151.stdout: python3-rados-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:49.228 INFO:teuthology.orchestra.run.trial151.stdout: python3-rbd-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:49.229 INFO:teuthology.orchestra.run.trial151.stdout: python3-rgw-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:49.229 INFO:teuthology.orchestra.run.trial151.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-25T09:12:49.229 INFO:teuthology.orchestra.run.trial151.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-25T09:12:49.229 INFO:teuthology.orchestra.run.trial151.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-25T09:12:49.229 INFO:teuthology.orchestra.run.trial151.stdout: re2-1:20211101-20.el9.x86_64 2026-02-25T09:12:49.229 INFO:teuthology.orchestra.run.trial151.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-25T09:12:49.229 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:12:49.229 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:12:49.265 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-test 2026-02-25T09:12:49.585 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:37 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:12:49.668 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:12:49.670 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:12:49.670 INFO:teuthology.orchestra.run.trial151.stdout: Package Arch Version Repository Size 2026-02-25T09:12:49.671 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:12:49.671 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:12:49.671 INFO:teuthology.orchestra.run.trial151.stdout: ceph-test x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 124 M 2026-02-25T09:12:49.671 INFO:teuthology.orchestra.run.trial151.stdout:Installing dependencies: 2026-02-25T09:12:49.671 INFO:teuthology.orchestra.run.trial151.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-25T09:12:49.671 INFO:teuthology.orchestra.run.trial151.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-25T09:12:49.672 INFO:teuthology.orchestra.run.trial151.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-25T09:12:49.672 INFO:teuthology.orchestra.run.trial151.stdout: libcephsqlite x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 153 k 2026-02-25T09:12:49.672 INFO:teuthology.orchestra.run.trial151.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-25T09:12:49.672 INFO:teuthology.orchestra.run.trial151.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-25T09:12:49.672 INFO:teuthology.orchestra.run.trial151.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-25T09:12:49.673 INFO:teuthology.orchestra.run.trial151.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-25T09:12:49.673 INFO:teuthology.orchestra.run.trial151.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-25T09:12:49.673 INFO:teuthology.orchestra.run.trial151.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-25T09:12:49.673 INFO:teuthology.orchestra.run.trial151.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-25T09:12:49.673 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:12:49.673 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:12:49.674 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:12:49.674 INFO:teuthology.orchestra.run.trial151.stdout:Install 12 Packages 2026-02-25T09:12:49.674 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:12:49.674 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 128 M 2026-02-25T09:12:49.674 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 969 M 2026-02-25T09:12:49.674 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:12:51.285 INFO:teuthology.orchestra.run.trial151.stdout:(1/12): libcephsqlite-20.3.0-5536.ge6eec768.el9 172 kB/s | 153 kB 00:00 2026-02-25T09:12:51.687 INFO:teuthology.orchestra.run.trial151.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 57 kB/s | 23 kB 00:00 2026-02-25T09:12:51.849 INFO:teuthology.orchestra.run.trial151.stdout:(3/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 582 kB/s | 94 kB 00:00 2026-02-25T09:12:52.357 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-selinux-2:20.3.0-5536.ge6eec768.el9.x86_64 33/36 2026-02-25T09:12:52.357 INFO:teuthology.orchestra.run.trial094.stdout:skipping the directory /sys 2026-02-25T09:12:52.357 INFO:teuthology.orchestra.run.trial094.stdout:skipping the directory /proc 2026-02-25T09:12:52.358 INFO:teuthology.orchestra.run.trial094.stdout:skipping the directory /mnt 2026-02-25T09:12:52.358 INFO:teuthology.orchestra.run.trial094.stdout:skipping the directory /var/tmp 2026-02-25T09:12:52.358 INFO:teuthology.orchestra.run.trial094.stdout:skipping the directory /home 2026-02-25T09:12:52.358 INFO:teuthology.orchestra.run.trial094.stdout:skipping the directory /root 2026-02-25T09:12:52.358 INFO:teuthology.orchestra.run.trial094.stdout:skipping the directory /tmp 2026-02-25T09:12:52.358 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:12:52.428 INFO:teuthology.orchestra.run.trial151.stdout:(4/12): libasan8-15.2.1-7.el9.x86_64.rpm 906 kB/s | 525 kB 00:00 2026-02-25T09:12:52.523 INFO:teuthology.orchestra.run.trial151.stdout:(5/12): libubsan-11.5.0-14.el9.x86_64.rpm 1.8 MB/s | 176 kB 00:00 2026-02-25T09:12:52.623 INFO:teuthology.orchestra.run.trial151.stdout:(6/12): libxslt-1.1.34-12.el9.x86_64.rpm 2.3 MB/s | 233 kB 00:00 2026-02-25T09:12:52.805 INFO:teuthology.orchestra.run.trial151.stdout:(7/12): protobuf-3.14.0-17.el9.x86_64.rpm 5.5 MB/s | 1.0 MB 00:00 2026-02-25T09:12:52.851 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-radosgw-2:20.3.0-5536.ge6eec768.el9.x86_64 34/36 2026-02-25T09:12:52.874 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-radosgw-2:20.3.0-5536.ge6eec768.el9.x86_64 34/36 2026-02-25T09:12:52.874 INFO:teuthology.orchestra.run.trial094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T09:12:52.874 INFO:teuthology.orchestra.run.trial094.stdout:Invalid unit name "ceph-radosgw@*.service" escaped as "ceph-radosgw@\x2a.service". 2026-02-25T09:12:52.875 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T09:12:52.875 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-radosgw.target → /usr/lib/systemd/system/ceph-radosgw.target. 2026-02-25T09:12:52.875 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:12:52.876 INFO:teuthology.orchestra.run.trial094.stdout: Cleanup : librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-25T09:12:52.889 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: librbd1-2:16.2.4-5.el9.x86_64 35/36 2026-02-25T09:12:52.889 INFO:teuthology.orchestra.run.trial094.stdout: Cleanup : librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-25T09:12:52.892 INFO:teuthology.orchestra.run.trial151.stdout:(8/12): socat-1.7.4.1-8.el9.x86_64.rpm 3.4 MB/s | 303 kB 00:00 2026-02-25T09:12:52.941 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: librados2-2:16.2.4-5.el9.x86_64 36/36 2026-02-25T09:12:52.941 INFO:teuthology.orchestra.run.trial094.stdout:Creating group 'qat' with GID 992. 2026-02-25T09:12:52.941 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:12:52.972 INFO:teuthology.orchestra.run.trial151.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 805 kB/s | 64 kB 00:00 2026-02-25T09:12:53.216 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-base-2:20.3.0-5536.ge6eec768.el9.x86_64 1/36 2026-02-25T09:12:53.217 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-common-2:20.3.0-5536.ge6eec768.el9.x86_64 2/36 2026-02-25T09:12:53.217 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-radosgw-2:20.3.0-5536.ge6eec768.el9.x86_64 3/36 2026-02-25T09:12:53.217 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-selinux-2:20.3.0-5536.ge6eec768.el9.x86_64 4/36 2026-02-25T09:12:53.217 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libcephfs2-2:20.3.0-5536.ge6eec768.el9.x86_64 5/36 2026-02-25T09:12:53.217 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libradosstriper1-2:20.3.0-5536.ge6eec768.el9.x86_6 6/36 2026-02-25T09:12:53.217 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librgw2-2:20.3.0-5536.ge6eec768.el9.x86_64 7/36 2026-02-25T09:12:53.217 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-ceph-argparse-2:20.3.0-5536.ge6eec768.el9. 8/36 2026-02-25T09:12:53.217 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-ceph-common-2:20.3.0-5536.ge6eec768.el9.x8 9/36 2026-02-25T09:12:53.217 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-cephfs-2:20.3.0-5536.ge6eec768.el9.x86_64 10/36 2026-02-25T09:12:53.217 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-rados-2:20.3.0-5536.ge6eec768.el9.x86_64 11/36 2026-02-25T09:12:53.218 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-rbd-2:20.3.0-5536.ge6eec768.el9.x86_64 12/36 2026-02-25T09:12:53.218 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-rgw-2:20.3.0-5536.ge6eec768.el9.x86_64 13/36 2026-02-25T09:12:53.218 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : mailcap-2.1.49-5.el9.noarch 14/36 2026-02-25T09:12:53.218 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : pciutils-3.7.0-7.el9.x86_64 15/36 2026-02-25T09:12:53.218 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : boost-program-options-1.75.0-13.el9.x86_64 16/36 2026-02-25T09:12:53.218 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libnbd-1.20.3-4.el9.x86_64 17/36 2026-02-25T09:12:53.218 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libpmemobj-1.12.1-1.el9.x86_64 18/36 2026-02-25T09:12:53.218 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librabbitmq-0.11.0-7.el9.x86_64 19/36 2026-02-25T09:12:53.218 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librdkafka-1.6.1-102.el9.x86_64 20/36 2026-02-25T09:12:53.218 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : lttng-ust-2.12.0-6.el9.x86_64 21/36 2026-02-25T09:12:53.218 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : qatlib-25.08.0-2.el9.x86_64 22/36 2026-02-25T09:12:53.218 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : qatlib-service-25.08.0-2.el9.x86_64 23/36 2026-02-25T09:12:53.218 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : qatzip-libs-1.3.1-1.el9.x86_64 24/36 2026-02-25T09:12:53.219 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : gperftools-libs-2.9.1-3.el9.x86_64 25/36 2026-02-25T09:12:53.219 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libarrow-9.0.0-15.el9.x86_64 26/36 2026-02-25T09:12:53.219 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libarrow-doc-9.0.0-15.el9.noarch 27/36 2026-02-25T09:12:53.219 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : liboath-2.6.12-1.el9.x86_64 28/36 2026-02-25T09:12:53.219 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libunwind-1.6.2-1.el9.x86_64 29/36 2026-02-25T09:12:53.219 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : parquet-libs-9.0.0-15.el9.x86_64 30/36 2026-02-25T09:12:53.219 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : re2-1:20211101-20.el9.x86_64 31/36 2026-02-25T09:12:53.219 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : thrift-0.15.0-4.el9.x86_64 32/36 2026-02-25T09:12:53.219 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librados2-2:20.3.0-5536.ge6eec768.el9.x86_64 33/36 2026-02-25T09:12:53.219 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librados2-2:16.2.4-5.el9.x86_64 34/36 2026-02-25T09:12:53.219 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librbd1-2:20.3.0-5536.ge6eec768.el9.x86_64 35/36 2026-02-25T09:12:53.252 INFO:teuthology.orchestra.run.trial151.stdout:(10/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 435 kB/s | 122 kB 00:00 2026-02-25T09:12:53.281 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librbd1-2:16.2.4-5.el9.x86_64 36/36 2026-02-25T09:12:53.281 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:12:53.281 INFO:teuthology.orchestra.run.trial094.stdout:Upgraded: 2026-02-25T09:12:53.281 INFO:teuthology.orchestra.run.trial094.stdout: librados2-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:53.281 INFO:teuthology.orchestra.run.trial094.stdout: librbd1-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:53.282 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:12:53.282 INFO:teuthology.orchestra.run.trial094.stdout: boost-program-options-1.75.0-13.el9.x86_64 2026-02-25T09:12:53.282 INFO:teuthology.orchestra.run.trial094.stdout: ceph-base-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:53.282 INFO:teuthology.orchestra.run.trial094.stdout: ceph-common-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:53.282 INFO:teuthology.orchestra.run.trial094.stdout: ceph-radosgw-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:53.282 INFO:teuthology.orchestra.run.trial094.stdout: ceph-selinux-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:53.282 INFO:teuthology.orchestra.run.trial094.stdout: gperftools-libs-2.9.1-3.el9.x86_64 2026-02-25T09:12:53.282 INFO:teuthology.orchestra.run.trial094.stdout: libarrow-9.0.0-15.el9.x86_64 2026-02-25T09:12:53.282 INFO:teuthology.orchestra.run.trial094.stdout: libarrow-doc-9.0.0-15.el9.noarch 2026-02-25T09:12:53.282 INFO:teuthology.orchestra.run.trial094.stdout: libcephfs2-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:53.282 INFO:teuthology.orchestra.run.trial094.stdout: libnbd-1.20.3-4.el9.x86_64 2026-02-25T09:12:53.283 INFO:teuthology.orchestra.run.trial094.stdout: liboath-2.6.12-1.el9.x86_64 2026-02-25T09:12:53.283 INFO:teuthology.orchestra.run.trial094.stdout: libpmemobj-1.12.1-1.el9.x86_64 2026-02-25T09:12:53.283 INFO:teuthology.orchestra.run.trial094.stdout: librabbitmq-0.11.0-7.el9.x86_64 2026-02-25T09:12:53.283 INFO:teuthology.orchestra.run.trial094.stdout: libradosstriper1-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:53.283 INFO:teuthology.orchestra.run.trial094.stdout: librdkafka-1.6.1-102.el9.x86_64 2026-02-25T09:12:53.283 INFO:teuthology.orchestra.run.trial094.stdout: librgw2-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:53.283 INFO:teuthology.orchestra.run.trial094.stdout: libunwind-1.6.2-1.el9.x86_64 2026-02-25T09:12:53.283 INFO:teuthology.orchestra.run.trial094.stdout: lttng-ust-2.12.0-6.el9.x86_64 2026-02-25T09:12:53.283 INFO:teuthology.orchestra.run.trial094.stdout: mailcap-2.1.49-5.el9.noarch 2026-02-25T09:12:53.283 INFO:teuthology.orchestra.run.trial094.stdout: parquet-libs-9.0.0-15.el9.x86_64 2026-02-25T09:12:53.283 INFO:teuthology.orchestra.run.trial094.stdout: pciutils-3.7.0-7.el9.x86_64 2026-02-25T09:12:53.283 INFO:teuthology.orchestra.run.trial094.stdout: python3-ceph-argparse-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:53.284 INFO:teuthology.orchestra.run.trial094.stdout: python3-ceph-common-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:53.284 INFO:teuthology.orchestra.run.trial094.stdout: python3-cephfs-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:53.284 INFO:teuthology.orchestra.run.trial094.stdout: python3-rados-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:53.284 INFO:teuthology.orchestra.run.trial094.stdout: python3-rbd-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:53.284 INFO:teuthology.orchestra.run.trial094.stdout: python3-rgw-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:12:53.284 INFO:teuthology.orchestra.run.trial094.stdout: qatlib-25.08.0-2.el9.x86_64 2026-02-25T09:12:53.284 INFO:teuthology.orchestra.run.trial094.stdout: qatlib-service-25.08.0-2.el9.x86_64 2026-02-25T09:12:53.284 INFO:teuthology.orchestra.run.trial094.stdout: qatzip-libs-1.3.1-1.el9.x86_64 2026-02-25T09:12:53.284 INFO:teuthology.orchestra.run.trial094.stdout: re2-1:20211101-20.el9.x86_64 2026-02-25T09:12:53.284 INFO:teuthology.orchestra.run.trial094.stdout: thrift-0.15.0-4.el9.x86_64 2026-02-25T09:12:53.284 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:12:53.284 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:12:53.320 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-test 2026-02-25T09:12:53.634 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:36 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:12:53.717 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:12:53.718 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:12:53.718 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repository Size 2026-02-25T09:12:53.718 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:12:53.718 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:12:53.718 INFO:teuthology.orchestra.run.trial094.stdout: ceph-test x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 124 M 2026-02-25T09:12:53.718 INFO:teuthology.orchestra.run.trial094.stdout:Installing dependencies: 2026-02-25T09:12:53.719 INFO:teuthology.orchestra.run.trial094.stdout: hwloc-libs x86_64 2.4.1-6.el9 baseos 2.1 M 2026-02-25T09:12:53.719 INFO:teuthology.orchestra.run.trial094.stdout: libasan8 x86_64 15.2.1-7.el9 appstream 525 k 2026-02-25T09:12:53.719 INFO:teuthology.orchestra.run.trial094.stdout: libatomic x86_64 11.5.0-14.el9 baseos 23 k 2026-02-25T09:12:53.719 INFO:teuthology.orchestra.run.trial094.stdout: libcephsqlite x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 153 k 2026-02-25T09:12:53.719 INFO:teuthology.orchestra.run.trial094.stdout: libubsan x86_64 11.5.0-14.el9 appstream 176 k 2026-02-25T09:12:53.719 INFO:teuthology.orchestra.run.trial094.stdout: libxslt x86_64 1.1.34-12.el9 appstream 233 k 2026-02-25T09:12:53.719 INFO:teuthology.orchestra.run.trial094.stdout: lksctp-tools x86_64 1.0.19-2.el9 baseos 94 k 2026-02-25T09:12:53.719 INFO:teuthology.orchestra.run.trial094.stdout: protobuf x86_64 3.14.0-17.el9 appstream 1.0 M 2026-02-25T09:12:53.719 INFO:teuthology.orchestra.run.trial094.stdout: socat x86_64 1.7.4.1-8.el9 appstream 303 k 2026-02-25T09:12:53.719 INFO:teuthology.orchestra.run.trial094.stdout: xmlstarlet x86_64 1.6.1-20.el9 appstream 64 k 2026-02-25T09:12:53.719 INFO:teuthology.orchestra.run.trial094.stdout: yaml-cpp x86_64 0.6.3-5.el9 epel 122 k 2026-02-25T09:12:53.719 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:12:53.720 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:12:53.720 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:12:53.720 INFO:teuthology.orchestra.run.trial094.stdout:Install 12 Packages 2026-02-25T09:12:53.720 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:12:53.720 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 128 M 2026-02-25T09:12:53.720 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 969 M 2026-02-25T09:12:53.720 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:12:54.247 INFO:teuthology.orchestra.run.trial151.stdout:(11/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 557 kB/s | 2.1 MB 00:03 2026-02-25T09:12:55.309 INFO:teuthology.orchestra.run.trial094.stdout:(1/12): libcephsqlite-20.3.0-5536.ge6eec768.el9 171 kB/s | 153 kB 00:00 2026-02-25T09:12:55.713 INFO:teuthology.orchestra.run.trial094.stdout:(2/12): libatomic-11.5.0-14.el9.x86_64.rpm 56 kB/s | 23 kB 00:00 2026-02-25T09:12:55.917 INFO:teuthology.orchestra.run.trial094.stdout:(3/12): lksctp-tools-1.0.19-2.el9.x86_64.rpm 463 kB/s | 94 kB 00:00 2026-02-25T09:12:56.027 INFO:teuthology.orchestra.run.trial094.stdout:(4/12): libasan8-15.2.1-7.el9.x86_64.rpm 4.7 MB/s | 525 kB 00:00 2026-02-25T09:12:56.050 INFO:teuthology.orchestra.run.trial094.stdout:(5/12): libubsan-11.5.0-14.el9.x86_64.rpm 7.4 MB/s | 176 kB 00:00 2026-02-25T09:12:56.074 INFO:teuthology.orchestra.run.trial094.stdout:(6/12): libxslt-1.1.34-12.el9.x86_64.rpm 9.7 MB/s | 233 kB 00:00 2026-02-25T09:12:56.126 INFO:teuthology.orchestra.run.trial094.stdout:(7/12): protobuf-3.14.0-17.el9.x86_64.rpm 20 MB/s | 1.0 MB 00:00 2026-02-25T09:12:56.149 INFO:teuthology.orchestra.run.trial094.stdout:(8/12): socat-1.7.4.1-8.el9.x86_64.rpm 13 MB/s | 303 kB 00:00 2026-02-25T09:12:56.171 INFO:teuthology.orchestra.run.trial094.stdout:(9/12): xmlstarlet-1.6.1-20.el9.x86_64.rpm 2.8 MB/s | 64 kB 00:00 2026-02-25T09:12:56.202 INFO:teuthology.orchestra.run.trial094.stdout:(10/12): yaml-cpp-0.6.3-5.el9.x86_64.rpm 3.9 MB/s | 122 kB 00:00 2026-02-25T09:12:57.279 INFO:teuthology.orchestra.run.trial094.stdout:(11/12): hwloc-libs-2.4.1-6.el9.x86_64.rpm 750 kB/s | 2.1 MB 00:02 2026-02-25T09:14:27.478 INFO:teuthology.orchestra.run.trial151.stdout:(12/12): ceph-test-20.3.0-5536.ge6eec768.el9.x8 1.3 MB/s | 124 MB 01:37 2026-02-25T09:14:27.480 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:14:27.480 INFO:teuthology.orchestra.run.trial151.stdout:Total 1.3 MB/s | 128 MB 01:37 2026-02-25T09:14:27.550 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:14:27.563 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:14:27.563 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:14:27.982 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:14:27.982 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:14:28.430 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:14:28.438 INFO:teuthology.orchestra.run.trial151.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-25T09:14:28.448 INFO:teuthology.orchestra.run.trial151.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-25T09:14:28.453 INFO:teuthology.orchestra.run.trial151.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-25T09:14:28.457 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-25T09:14:28.460 INFO:teuthology.orchestra.run.trial151.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-25T09:14:28.464 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-25T09:14:28.465 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-25T09:14:28.482 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-25T09:14:28.485 INFO:teuthology.orchestra.run.trial151.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-25T09:14:28.494 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-25T09:14:28.503 INFO:teuthology.orchestra.run.trial151.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-25T09:14:28.503 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libcephsqlite-2:20.3.0-5536.ge6eec768.el9.x86_64 11/12 2026-02-25T09:14:35.278 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5536.ge6eec768.el9.x86_64 11/12 2026-02-25T09:14:35.297 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-test-2:20.3.0-5536.ge6eec768.el9.x86_64 12/12 2026-02-25T09:14:36.046 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-test-2:20.3.0-5536.ge6eec768.el9.x86_64 12/12 2026-02-25T09:14:36.046 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-test-2:20.3.0-5536.ge6eec768.el9.x86_64 1/12 2026-02-25T09:14:36.046 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libcephsqlite-2:20.3.0-5536.ge6eec768.el9.x86_64 2/12 2026-02-25T09:14:36.046 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-25T09:14:36.046 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-25T09:14:36.046 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-25T09:14:36.046 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-25T09:14:36.047 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-25T09:14:36.047 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-25T09:14:36.047 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-25T09:14:36.047 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-25T09:14:36.047 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-25T09:14:36.104 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-25T09:14:36.104 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:14:36.104 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:14:36.104 INFO:teuthology.orchestra.run.trial151.stdout: ceph-test-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:14:36.104 INFO:teuthology.orchestra.run.trial151.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-25T09:14:36.104 INFO:teuthology.orchestra.run.trial151.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-25T09:14:36.104 INFO:teuthology.orchestra.run.trial151.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-25T09:14:36.105 INFO:teuthology.orchestra.run.trial151.stdout: libcephsqlite-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:14:36.105 INFO:teuthology.orchestra.run.trial151.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-25T09:14:36.105 INFO:teuthology.orchestra.run.trial151.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-25T09:14:36.105 INFO:teuthology.orchestra.run.trial151.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-25T09:14:36.105 INFO:teuthology.orchestra.run.trial151.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-25T09:14:36.105 INFO:teuthology.orchestra.run.trial151.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-25T09:14:36.105 INFO:teuthology.orchestra.run.trial151.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-25T09:14:36.105 INFO:teuthology.orchestra.run.trial151.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-25T09:14:36.105 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:14:36.105 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:14:36.145 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph 2026-02-25T09:14:36.458 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:02:24 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:14:36.542 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:14:36.543 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:14:36.543 INFO:teuthology.orchestra.run.trial151.stdout: Package Arch Version Repository Size 2026-02-25T09:14:36.543 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:14:36.544 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:14:36.544 INFO:teuthology.orchestra.run.trial151.stdout: ceph x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 6.5 k 2026-02-25T09:14:36.544 INFO:teuthology.orchestra.run.trial151.stdout:Installing dependencies: 2026-02-25T09:14:36.544 INFO:teuthology.orchestra.run.trial151.stdout: ceph-mds x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 2.0 M 2026-02-25T09:14:36.544 INFO:teuthology.orchestra.run.trial151.stdout: ceph-mgr x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 947 k 2026-02-25T09:14:36.544 INFO:teuthology.orchestra.run.trial151.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 299 k 2026-02-25T09:14:36.544 INFO:teuthology.orchestra.run.trial151.stdout: ceph-mon x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 4.9 M 2026-02-25T09:14:36.544 INFO:teuthology.orchestra.run.trial151.stdout: ceph-osd x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 163 k 2026-02-25T09:14:36.544 INFO:teuthology.orchestra.run.trial151.stdout: ceph-osd-classic x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 15 M 2026-02-25T09:14:36.544 INFO:teuthology.orchestra.run.trial151.stdout: ceph-osd-crimson x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 87 M 2026-02-25T09:14:36.544 INFO:teuthology.orchestra.run.trial151.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-25T09:14:36.545 INFO:teuthology.orchestra.run.trial151.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-25T09:14:36.545 INFO:teuthology.orchestra.run.trial151.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-25T09:14:36.545 INFO:teuthology.orchestra.run.trial151.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-25T09:14:36.545 INFO:teuthology.orchestra.run.trial151.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-25T09:14:36.545 INFO:teuthology.orchestra.run.trial151.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-25T09:14:36.545 INFO:teuthology.orchestra.run.trial151.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-25T09:14:36.545 INFO:teuthology.orchestra.run.trial151.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-25T09:14:36.545 INFO:teuthology.orchestra.run.trial151.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-25T09:14:36.545 INFO:teuthology.orchestra.run.trial151.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-25T09:14:36.545 INFO:teuthology.orchestra.run.trial151.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-25T09:14:36.545 INFO:teuthology.orchestra.run.trial151.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-25T09:14:36.545 INFO:teuthology.orchestra.run.trial151.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-25T09:14:36.545 INFO:teuthology.orchestra.run.trial151.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-25T09:14:36.546 INFO:teuthology.orchestra.run.trial151.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-25T09:14:36.546 INFO:teuthology.orchestra.run.trial151.stdout: python3-jaraco-collections 2026-02-25T09:14:36.546 INFO:teuthology.orchestra.run.trial151.stdout: noarch 3.0.0-8.el9 epel 23 k 2026-02-25T09:14:36.546 INFO:teuthology.orchestra.run.trial151.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-25T09:14:36.546 INFO:teuthology.orchestra.run.trial151.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-25T09:14:36.546 INFO:teuthology.orchestra.run.trial151.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-25T09:14:36.546 INFO:teuthology.orchestra.run.trial151.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-25T09:14:36.546 INFO:teuthology.orchestra.run.trial151.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-25T09:14:36.546 INFO:teuthology.orchestra.run.trial151.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-25T09:14:36.546 INFO:teuthology.orchestra.run.trial151.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-25T09:14:36.546 INFO:teuthology.orchestra.run.trial151.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-25T09:14:36.546 INFO:teuthology.orchestra.run.trial151.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-25T09:14:36.546 INFO:teuthology.orchestra.run.trial151.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-25T09:14:36.547 INFO:teuthology.orchestra.run.trial151.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-25T09:14:36.547 INFO:teuthology.orchestra.run.trial151.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-25T09:14:36.547 INFO:teuthology.orchestra.run.trial151.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-25T09:14:36.547 INFO:teuthology.orchestra.run.trial151.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-25T09:14:36.547 INFO:teuthology.orchestra.run.trial151.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-25T09:14:36.547 INFO:teuthology.orchestra.run.trial151.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-25T09:14:36.547 INFO:teuthology.orchestra.run.trial151.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-25T09:14:36.547 INFO:teuthology.orchestra.run.trial151.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-25T09:14:36.547 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:14:36.547 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:14:36.547 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:14:36.547 INFO:teuthology.orchestra.run.trial151.stdout:Install 42 Packages 2026-02-25T09:14:36.548 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:14:36.548 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 114 M 2026-02-25T09:14:36.549 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 961 M 2026-02-25T09:14:36.549 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:14:37.554 INFO:teuthology.orchestra.run.trial151.stdout:(1/42): ceph-20.3.0-5536.ge6eec768.el9.x86_64.r 22 kB/s | 6.5 kB 00:00 2026-02-25T09:14:38.446 INFO:teuthology.orchestra.run.trial151.stdout:(2/42): ceph-mgr-20.3.0-5536.ge6eec768.el9.x86_ 801 kB/s | 947 kB 00:01 2026-02-25T09:14:38.655 INFO:teuthology.orchestra.run.trial151.stdout:(3/42): ceph-osd-20.3.0-5536.ge6eec768.el9.x86_ 778 kB/s | 163 kB 00:00 2026-02-25T09:14:44.044 INFO:teuthology.orchestra.run.trial151.stdout:(4/42): ceph-mds-20.3.0-5536.ge6eec768.el9.x86_ 305 kB/s | 2.0 MB 00:06 2026-02-25T09:14:44.843 INFO:teuthology.orchestra.run.trial151.stdout:(5/42): ceph-mon-20.3.0-5536.ge6eec768.el9.x86_ 684 kB/s | 4.9 MB 00:07 2026-02-25T09:14:45.537 INFO:teuthology.orchestra.run.trial151.stdout:(6/42): ceph-mgr-modules-core-20.3.0-5536.ge6ee 430 kB/s | 299 kB 00:00 2026-02-25T09:14:45.612 INFO:teuthology.orchestra.run.trial151.stdout:(7/42): ledmon-libs-1.1.0-3.el9.x86_64.rpm 539 kB/s | 40 kB 00:00 2026-02-25T09:14:45.656 INFO:teuthology.orchestra.run.trial151.stdout:(8/42): libconfig-1.7.2-9.el9.x86_64.rpm 1.6 MB/s | 72 kB 00:00 2026-02-25T09:14:45.700 INFO:teuthology.orchestra.run.trial151.stdout:(9/42): python3-cffi-1.14.5-5.el9.x86_64.rpm 5.6 MB/s | 253 kB 00:00 2026-02-25T09:14:45.773 INFO:teuthology.orchestra.run.trial151.stdout:(10/42): python3-cryptography-36.0.1-5.el9.x86_ 17 MB/s | 1.2 MB 00:00 2026-02-25T09:14:45.797 INFO:teuthology.orchestra.run.trial151.stdout:(11/42): python3-ply-3.11-14.el9.noarch.rpm 4.4 MB/s | 106 kB 00:00 2026-02-25T09:14:45.821 INFO:teuthology.orchestra.run.trial151.stdout:(12/42): python3-pycparser-2.20-6.el9.noarch.rp 5.5 MB/s | 135 kB 00:00 2026-02-25T09:14:45.845 INFO:teuthology.orchestra.run.trial151.stdout:(13/42): python3-pyparsing-2.4.7-9.el9.noarch.r 6.1 MB/s | 150 kB 00:00 2026-02-25T09:14:45.912 INFO:teuthology.orchestra.run.trial151.stdout:(14/42): python3-requests-2.25.1-10.el9.noarch. 1.9 MB/s | 126 kB 00:00 2026-02-25T09:14:45.939 INFO:teuthology.orchestra.run.trial151.stdout:(15/42): python3-urllib3-1.26.5-7.el9.noarch.rp 7.9 MB/s | 218 kB 00:00 2026-02-25T09:14:45.963 INFO:teuthology.orchestra.run.trial151.stdout:(16/42): unzip-6.0-59.el9.x86_64.rpm 7.2 MB/s | 182 kB 00:00 2026-02-25T09:14:45.987 INFO:teuthology.orchestra.run.trial151.stdout:(17/42): zip-3.0-35.el9.x86_64.rpm 11 MB/s | 266 kB 00:00 2026-02-25T09:14:46.780 INFO:teuthology.orchestra.run.trial151.stdout:(18/42): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 311 kB/s | 246 kB 00:00 2026-02-25T09:14:46.938 INFO:teuthology.orchestra.run.trial151.stdout:(19/42): lua-5.4.4-4.el9.x86_64.rpm 1.2 MB/s | 188 kB 00:00 2026-02-25T09:14:47.097 INFO:teuthology.orchestra.run.trial151.stdout:(20/42): python3-libstoragemgmt-1.10.1-1.el9.x8 1.1 MB/s | 177 kB 00:00 2026-02-25T09:14:47.255 INFO:teuthology.orchestra.run.trial151.stdout:(21/42): python3-packaging-20.9-5.el9.noarch.rp 487 kB/s | 77 kB 00:00 2026-02-25T09:14:47.413 INFO:teuthology.orchestra.run.trial151.stdout:(22/42): python3-toml-0.10.2-6.el9.noarch.rpm 263 kB/s | 42 kB 00:00 2026-02-25T09:14:47.572 INFO:teuthology.orchestra.run.trial151.stdout:(23/42): lua-devel-5.4.4-4.el9.x86_64.rpm 141 kB/s | 22 kB 00:00 2026-02-25T09:14:48.006 INFO:teuthology.orchestra.run.trial151.stdout:(24/42): luarocks-3.9.2-5.el9.noarch.rpm 349 kB/s | 151 kB 00:00 2026-02-25T09:14:48.079 INFO:teuthology.orchestra.run.trial151.stdout:(25/42): python3-autocommand-2.2.2-8.el9.noarch 402 kB/s | 29 kB 00:00 2026-02-25T09:14:48.162 INFO:teuthology.orchestra.run.trial151.stdout:(26/42): python3-backports-tarfile-1.2.0-1.el9. 726 kB/s | 60 kB 00:00 2026-02-25T09:14:48.240 INFO:teuthology.orchestra.run.trial151.stdout:(27/42): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 559 kB/s | 43 kB 00:00 2026-02-25T09:14:48.358 INFO:teuthology.orchestra.run.trial151.stdout:(28/42): python3-cheroot-10.0.1-4.el9.noarch.rp 1.4 MB/s | 173 kB 00:00 2026-02-25T09:14:48.507 INFO:teuthology.orchestra.run.trial151.stdout:(29/42): python3-cherrypy-18.6.1-2.el9.noarch.r 2.3 MB/s | 358 kB 00:00 2026-02-25T09:14:48.574 INFO:teuthology.orchestra.run.trial151.stdout:(30/42): python3-jaraco-8.2.1-3.el9.noarch.rpm 160 kB/s | 11 kB 00:00 2026-02-25T09:14:48.641 INFO:teuthology.orchestra.run.trial151.stdout:(31/42): python3-jaraco-classes-3.2.1-5.el9.noa 262 kB/s | 18 kB 00:00 2026-02-25T09:14:48.707 INFO:teuthology.orchestra.run.trial151.stdout:(32/42): python3-jaraco-collections-3.0.0-8.el9 350 kB/s | 23 kB 00:00 2026-02-25T09:14:48.774 INFO:teuthology.orchestra.run.trial151.stdout:(33/42): python3-jaraco-context-6.0.1-3.el9.noa 297 kB/s | 20 kB 00:00 2026-02-25T09:14:48.840 INFO:teuthology.orchestra.run.trial151.stdout:(34/42): python3-jaraco-functools-3.5.0-2.el9.n 294 kB/s | 19 kB 00:00 2026-02-25T09:14:48.906 INFO:teuthology.orchestra.run.trial151.stdout:(35/42): python3-jaraco-text-4.0.0-2.el9.noarch 400 kB/s | 26 kB 00:00 2026-02-25T09:14:48.977 INFO:teuthology.orchestra.run.trial151.stdout:(36/42): python3-more-itertools-8.12.0-2.el9.no 1.1 MB/s | 79 kB 00:00 2026-02-25T09:14:49.044 INFO:teuthology.orchestra.run.trial151.stdout:(37/42): python3-portend-3.1.0-2.el9.noarch.rpm 247 kB/s | 16 kB 00:00 2026-02-25T09:14:49.116 INFO:teuthology.orchestra.run.trial151.stdout:(38/42): python3-pyOpenSSL-21.0.0-1.el9.noarch. 1.2 MB/s | 90 kB 00:00 2026-02-25T09:14:49.183 INFO:teuthology.orchestra.run.trial151.stdout:(39/42): python3-tempora-5.0.0-2.el9.noarch.rpm 534 kB/s | 36 kB 00:00 2026-02-25T09:14:49.249 INFO:teuthology.orchestra.run.trial151.stdout:(40/42): python3-zc-lockfile-2.0-10.el9.noarch. 302 kB/s | 20 kB 00:00 2026-02-25T09:14:52.224 INFO:teuthology.orchestra.run.trial151.stdout:(41/42): ceph-osd-classic-20.3.0-5536.ge6eec768 1.1 MB/s | 15 MB 00:13 2026-02-25T09:14:52.987 INFO:teuthology.orchestra.run.trial094.stdout:(12/12): ceph-test-20.3.0-5536.ge6eec768.el9.x8 1.0 MB/s | 124 MB 01:58 2026-02-25T09:14:52.990 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:14:52.990 INFO:teuthology.orchestra.run.trial094.stdout:Total 1.1 MB/s | 128 MB 01:59 2026-02-25T09:14:53.060 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:14:53.072 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:14:53.072 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:14:53.493 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:14:53.494 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:14:53.940 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:14:53.948 INFO:teuthology.orchestra.run.trial094.stdout: Installing : yaml-cpp-0.6.3-5.el9.x86_64 1/12 2026-02-25T09:14:53.957 INFO:teuthology.orchestra.run.trial094.stdout: Installing : socat-1.7.4.1-8.el9.x86_64 2/12 2026-02-25T09:14:53.962 INFO:teuthology.orchestra.run.trial094.stdout: Installing : protobuf-3.14.0-17.el9.x86_64 3/12 2026-02-25T09:14:53.966 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libxslt-1.1.34-12.el9.x86_64 4/12 2026-02-25T09:14:53.968 INFO:teuthology.orchestra.run.trial094.stdout: Installing : xmlstarlet-1.6.1-20.el9.x86_64 5/12 2026-02-25T09:14:53.973 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libubsan-11.5.0-14.el9.x86_64 6/12 2026-02-25T09:14:53.974 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-25T09:14:53.991 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: libasan8-15.2.1-7.el9.x86_64 7/12 2026-02-25T09:14:53.993 INFO:teuthology.orchestra.run.trial094.stdout: Installing : lksctp-tools-1.0.19-2.el9.x86_64 8/12 2026-02-25T09:14:54.003 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libatomic-11.5.0-14.el9.x86_64 9/12 2026-02-25T09:14:54.011 INFO:teuthology.orchestra.run.trial094.stdout: Installing : hwloc-libs-2.4.1-6.el9.x86_64 10/12 2026-02-25T09:14:54.012 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libcephsqlite-2:20.3.0-5536.ge6eec768.el9.x86_64 11/12 2026-02-25T09:15:00.851 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: libcephsqlite-2:20.3.0-5536.ge6eec768.el9.x86_64 11/12 2026-02-25T09:15:00.871 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-test-2:20.3.0-5536.ge6eec768.el9.x86_64 12/12 2026-02-25T09:15:01.688 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-test-2:20.3.0-5536.ge6eec768.el9.x86_64 12/12 2026-02-25T09:15:01.688 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-test-2:20.3.0-5536.ge6eec768.el9.x86_64 1/12 2026-02-25T09:15:01.688 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libcephsqlite-2:20.3.0-5536.ge6eec768.el9.x86_64 2/12 2026-02-25T09:15:01.688 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : hwloc-libs-2.4.1-6.el9.x86_64 3/12 2026-02-25T09:15:01.688 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libatomic-11.5.0-14.el9.x86_64 4/12 2026-02-25T09:15:01.688 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : lksctp-tools-1.0.19-2.el9.x86_64 5/12 2026-02-25T09:15:01.688 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libasan8-15.2.1-7.el9.x86_64 6/12 2026-02-25T09:15:01.688 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libubsan-11.5.0-14.el9.x86_64 7/12 2026-02-25T09:15:01.688 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libxslt-1.1.34-12.el9.x86_64 8/12 2026-02-25T09:15:01.688 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : protobuf-3.14.0-17.el9.x86_64 9/12 2026-02-25T09:15:01.688 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : socat-1.7.4.1-8.el9.x86_64 10/12 2026-02-25T09:15:01.688 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : xmlstarlet-1.6.1-20.el9.x86_64 11/12 2026-02-25T09:15:01.748 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : yaml-cpp-0.6.3-5.el9.x86_64 12/12 2026-02-25T09:15:01.748 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:15:01.748 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:15:01.748 INFO:teuthology.orchestra.run.trial094.stdout: ceph-test-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:15:01.748 INFO:teuthology.orchestra.run.trial094.stdout: hwloc-libs-2.4.1-6.el9.x86_64 2026-02-25T09:15:01.748 INFO:teuthology.orchestra.run.trial094.stdout: libasan8-15.2.1-7.el9.x86_64 2026-02-25T09:15:01.749 INFO:teuthology.orchestra.run.trial094.stdout: libatomic-11.5.0-14.el9.x86_64 2026-02-25T09:15:01.749 INFO:teuthology.orchestra.run.trial094.stdout: libcephsqlite-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:15:01.749 INFO:teuthology.orchestra.run.trial094.stdout: libubsan-11.5.0-14.el9.x86_64 2026-02-25T09:15:01.749 INFO:teuthology.orchestra.run.trial094.stdout: libxslt-1.1.34-12.el9.x86_64 2026-02-25T09:15:01.749 INFO:teuthology.orchestra.run.trial094.stdout: lksctp-tools-1.0.19-2.el9.x86_64 2026-02-25T09:15:01.749 INFO:teuthology.orchestra.run.trial094.stdout: protobuf-3.14.0-17.el9.x86_64 2026-02-25T09:15:01.749 INFO:teuthology.orchestra.run.trial094.stdout: socat-1.7.4.1-8.el9.x86_64 2026-02-25T09:15:01.749 INFO:teuthology.orchestra.run.trial094.stdout: xmlstarlet-1.6.1-20.el9.x86_64 2026-02-25T09:15:01.749 INFO:teuthology.orchestra.run.trial094.stdout: yaml-cpp-0.6.3-5.el9.x86_64 2026-02-25T09:15:01.749 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:15:01.749 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:15:01.790 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph 2026-02-25T09:15:02.106 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:02:45 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:15:02.191 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:15:02.192 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:15:02.192 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repository Size 2026-02-25T09:15:02.193 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:15:02.193 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:15:02.193 INFO:teuthology.orchestra.run.trial094.stdout: ceph x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 6.5 k 2026-02-25T09:15:02.193 INFO:teuthology.orchestra.run.trial094.stdout:Installing dependencies: 2026-02-25T09:15:02.193 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mds x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 2.0 M 2026-02-25T09:15:02.193 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 947 k 2026-02-25T09:15:02.193 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-modules-core noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 299 k 2026-02-25T09:15:02.193 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mon x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 4.9 M 2026-02-25T09:15:02.193 INFO:teuthology.orchestra.run.trial094.stdout: ceph-osd x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 163 k 2026-02-25T09:15:02.193 INFO:teuthology.orchestra.run.trial094.stdout: ceph-osd-classic x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 15 M 2026-02-25T09:15:02.194 INFO:teuthology.orchestra.run.trial094.stdout: ceph-osd-crimson x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 87 M 2026-02-25T09:15:02.194 INFO:teuthology.orchestra.run.trial094.stdout: ledmon-libs x86_64 1.1.0-3.el9 baseos 40 k 2026-02-25T09:15:02.194 INFO:teuthology.orchestra.run.trial094.stdout: libconfig x86_64 1.7.2-9.el9 baseos 72 k 2026-02-25T09:15:02.194 INFO:teuthology.orchestra.run.trial094.stdout: libstoragemgmt x86_64 1.10.1-1.el9 appstream 246 k 2026-02-25T09:15:02.194 INFO:teuthology.orchestra.run.trial094.stdout: lua x86_64 5.4.4-4.el9 appstream 188 k 2026-02-25T09:15:02.194 INFO:teuthology.orchestra.run.trial094.stdout: lua-devel x86_64 5.4.4-4.el9 crb 22 k 2026-02-25T09:15:02.194 INFO:teuthology.orchestra.run.trial094.stdout: luarocks noarch 3.9.2-5.el9 epel 151 k 2026-02-25T09:15:02.194 INFO:teuthology.orchestra.run.trial094.stdout: python3-autocommand noarch 2.2.2-8.el9 epel 29 k 2026-02-25T09:15:02.194 INFO:teuthology.orchestra.run.trial094.stdout: python3-backports-tarfile noarch 1.2.0-1.el9 epel 60 k 2026-02-25T09:15:02.194 INFO:teuthology.orchestra.run.trial094.stdout: python3-bcrypt x86_64 3.2.2-1.el9 epel 43 k 2026-02-25T09:15:02.194 INFO:teuthology.orchestra.run.trial094.stdout: python3-cffi x86_64 1.14.5-5.el9 baseos 253 k 2026-02-25T09:15:02.194 INFO:teuthology.orchestra.run.trial094.stdout: python3-cheroot noarch 10.0.1-4.el9 epel 173 k 2026-02-25T09:15:02.194 INFO:teuthology.orchestra.run.trial094.stdout: python3-cherrypy noarch 18.6.1-2.el9 epel 358 k 2026-02-25T09:15:02.195 INFO:teuthology.orchestra.run.trial094.stdout: python3-cryptography x86_64 36.0.1-5.el9 baseos 1.2 M 2026-02-25T09:15:02.195 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco noarch 8.2.1-3.el9 epel 11 k 2026-02-25T09:15:02.195 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-classes noarch 3.2.1-5.el9 epel 18 k 2026-02-25T09:15:02.195 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-collections 2026-02-25T09:15:02.195 INFO:teuthology.orchestra.run.trial094.stdout: noarch 3.0.0-8.el9 epel 23 k 2026-02-25T09:15:02.195 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-context noarch 6.0.1-3.el9 epel 20 k 2026-02-25T09:15:02.195 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-functools noarch 3.5.0-2.el9 epel 19 k 2026-02-25T09:15:02.195 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-text noarch 4.0.0-2.el9 epel 26 k 2026-02-25T09:15:02.195 INFO:teuthology.orchestra.run.trial094.stdout: python3-libstoragemgmt x86_64 1.10.1-1.el9 appstream 177 k 2026-02-25T09:15:02.195 INFO:teuthology.orchestra.run.trial094.stdout: python3-more-itertools noarch 8.12.0-2.el9 epel 79 k 2026-02-25T09:15:02.195 INFO:teuthology.orchestra.run.trial094.stdout: python3-packaging noarch 20.9-5.el9 appstream 77 k 2026-02-25T09:15:02.195 INFO:teuthology.orchestra.run.trial094.stdout: python3-ply noarch 3.11-14.el9 baseos 106 k 2026-02-25T09:15:02.195 INFO:teuthology.orchestra.run.trial094.stdout: python3-portend noarch 3.1.0-2.el9 epel 16 k 2026-02-25T09:15:02.196 INFO:teuthology.orchestra.run.trial094.stdout: python3-pyOpenSSL noarch 21.0.0-1.el9 epel 90 k 2026-02-25T09:15:02.196 INFO:teuthology.orchestra.run.trial094.stdout: python3-pycparser noarch 2.20-6.el9 baseos 135 k 2026-02-25T09:15:02.196 INFO:teuthology.orchestra.run.trial094.stdout: python3-pyparsing noarch 2.4.7-9.el9 baseos 150 k 2026-02-25T09:15:02.196 INFO:teuthology.orchestra.run.trial094.stdout: python3-requests noarch 2.25.1-10.el9 baseos 126 k 2026-02-25T09:15:02.196 INFO:teuthology.orchestra.run.trial094.stdout: python3-tempora noarch 5.0.0-2.el9 epel 36 k 2026-02-25T09:15:02.196 INFO:teuthology.orchestra.run.trial094.stdout: python3-toml noarch 0.10.2-6.el9 appstream 42 k 2026-02-25T09:15:02.196 INFO:teuthology.orchestra.run.trial094.stdout: python3-urllib3 noarch 1.26.5-7.el9 baseos 218 k 2026-02-25T09:15:02.196 INFO:teuthology.orchestra.run.trial094.stdout: python3-zc-lockfile noarch 2.0-10.el9 epel 20 k 2026-02-25T09:15:02.196 INFO:teuthology.orchestra.run.trial094.stdout: unzip x86_64 6.0-59.el9 baseos 182 k 2026-02-25T09:15:02.196 INFO:teuthology.orchestra.run.trial094.stdout: zip x86_64 3.0-35.el9 baseos 266 k 2026-02-25T09:15:02.196 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:15:02.196 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:15:02.196 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:15:02.197 INFO:teuthology.orchestra.run.trial094.stdout:Install 42 Packages 2026-02-25T09:15:02.197 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:15:02.198 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 114 M 2026-02-25T09:15:02.198 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 961 M 2026-02-25T09:15:02.198 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:15:02.925 INFO:teuthology.orchestra.run.trial094.stdout:(1/42): ceph-20.3.0-5536.ge6eec768.el9.x86_64.r 22 kB/s | 6.5 kB 00:00 2026-02-25T09:15:03.784 INFO:teuthology.orchestra.run.trial094.stdout:(2/42): ceph-mgr-20.3.0-5536.ge6eec768.el9.x86_ 825 kB/s | 947 kB 00:01 2026-02-25T09:15:03.917 INFO:teuthology.orchestra.run.trial094.stdout:(3/42): ceph-osd-20.3.0-5536.ge6eec768.el9.x86_ 1.2 MB/s | 163 kB 00:00 2026-02-25T09:15:07.407 INFO:teuthology.orchestra.run.trial094.stdout:(4/42): ceph-mds-20.3.0-5536.ge6eec768.el9.x86_ 434 kB/s | 2.0 MB 00:04 2026-02-25T09:15:07.607 INFO:teuthology.orchestra.run.trial094.stdout:(5/42): ceph-mon-20.3.0-5536.ge6eec768.el9.x86_ 1.0 MB/s | 4.9 MB 00:04 2026-02-25T09:15:08.018 INFO:teuthology.orchestra.run.trial094.stdout:(6/42): ceph-mgr-modules-core-20.3.0-5536.ge6ee 727 kB/s | 299 kB 00:00 2026-02-25T09:15:08.300 INFO:teuthology.orchestra.run.trial094.stdout:(7/42): ledmon-libs-1.1.0-3.el9.x86_64.rpm 143 kB/s | 40 kB 00:00 2026-02-25T09:15:08.491 INFO:teuthology.orchestra.run.trial094.stdout:(8/42): libconfig-1.7.2-9.el9.x86_64.rpm 378 kB/s | 72 kB 00:00 2026-02-25T09:15:08.719 INFO:teuthology.orchestra.run.trial094.stdout:(9/42): python3-cffi-1.14.5-5.el9.x86_64.rpm 1.1 MB/s | 253 kB 00:00 2026-02-25T09:15:08.998 INFO:teuthology.orchestra.run.trial094.stdout:(10/42): python3-cryptography-36.0.1-5.el9.x86_ 4.5 MB/s | 1.2 MB 00:00 2026-02-25T09:15:09.079 INFO:teuthology.orchestra.run.trial094.stdout:(11/42): python3-ply-3.11-14.el9.noarch.rpm 1.3 MB/s | 106 kB 00:00 2026-02-25T09:15:09.161 INFO:teuthology.orchestra.run.trial094.stdout:(12/42): python3-pycparser-2.20-6.el9.noarch.rp 1.6 MB/s | 135 kB 00:00 2026-02-25T09:15:09.244 INFO:teuthology.orchestra.run.trial094.stdout:(13/42): python3-pyparsing-2.4.7-9.el9.noarch.r 1.8 MB/s | 150 kB 00:00 2026-02-25T09:15:09.326 INFO:teuthology.orchestra.run.trial094.stdout:(14/42): python3-requests-2.25.1-10.el9.noarch. 1.5 MB/s | 126 kB 00:00 2026-02-25T09:15:09.411 INFO:teuthology.orchestra.run.trial094.stdout:(15/42): python3-urllib3-1.26.5-7.el9.noarch.rp 2.5 MB/s | 218 kB 00:00 2026-02-25T09:15:09.496 INFO:teuthology.orchestra.run.trial094.stdout:(16/42): unzip-6.0-59.el9.x86_64.rpm 2.1 MB/s | 182 kB 00:00 2026-02-25T09:15:09.582 INFO:teuthology.orchestra.run.trial094.stdout:(17/42): zip-3.0-35.el9.x86_64.rpm 3.0 MB/s | 266 kB 00:00 2026-02-25T09:15:09.603 INFO:teuthology.orchestra.run.trial094.stdout:(18/42): libstoragemgmt-1.10.1-1.el9.x86_64.rpm 11 MB/s | 246 kB 00:00 2026-02-25T09:15:09.610 INFO:teuthology.orchestra.run.trial094.stdout:(19/42): lua-5.4.4-4.el9.x86_64.rpm 26 MB/s | 188 kB 00:00 2026-02-25T09:15:09.617 INFO:teuthology.orchestra.run.trial094.stdout:(20/42): python3-libstoragemgmt-1.10.1-1.el9.x8 26 MB/s | 177 kB 00:00 2026-02-25T09:15:09.623 INFO:teuthology.orchestra.run.trial094.stdout:(21/42): python3-packaging-20.9-5.el9.noarch.rp 14 MB/s | 77 kB 00:00 2026-02-25T09:15:09.627 INFO:teuthology.orchestra.run.trial094.stdout:(22/42): python3-toml-0.10.2-6.el9.noarch.rpm 8.7 MB/s | 42 kB 00:00 2026-02-25T09:15:09.632 INFO:teuthology.orchestra.run.trial094.stdout:(23/42): lua-devel-5.4.4-4.el9.x86_64.rpm 5.1 MB/s | 22 kB 00:00 2026-02-25T09:15:09.793 INFO:teuthology.orchestra.run.trial094.stdout:(24/42): luarocks-3.9.2-5.el9.noarch.rpm 940 kB/s | 151 kB 00:00 2026-02-25T09:15:09.818 INFO:teuthology.orchestra.run.trial094.stdout:(25/42): python3-autocommand-2.2.2-8.el9.noarch 1.2 MB/s | 29 kB 00:00 2026-02-25T09:15:09.843 INFO:teuthology.orchestra.run.trial094.stdout:(26/42): python3-backports-tarfile-1.2.0-1.el9. 2.4 MB/s | 60 kB 00:00 2026-02-25T09:15:09.868 INFO:teuthology.orchestra.run.trial094.stdout:(27/42): python3-bcrypt-3.2.2-1.el9.x86_64.rpm 1.7 MB/s | 43 kB 00:00 2026-02-25T09:15:09.917 INFO:teuthology.orchestra.run.trial094.stdout:(28/42): python3-cheroot-10.0.1-4.el9.noarch.rp 3.4 MB/s | 173 kB 00:00 2026-02-25T09:15:09.971 INFO:teuthology.orchestra.run.trial094.stdout:(29/42): python3-cherrypy-18.6.1-2.el9.noarch.r 6.5 MB/s | 358 kB 00:00 2026-02-25T09:15:09.996 INFO:teuthology.orchestra.run.trial094.stdout:(30/42): python3-jaraco-8.2.1-3.el9.noarch.rpm 436 kB/s | 11 kB 00:00 2026-02-25T09:15:10.021 INFO:teuthology.orchestra.run.trial094.stdout:(31/42): python3-jaraco-classes-3.2.1-5.el9.noa 721 kB/s | 18 kB 00:00 2026-02-25T09:15:10.045 INFO:teuthology.orchestra.run.trial094.stdout:(32/42): python3-jaraco-collections-3.0.0-8.el9 945 kB/s | 23 kB 00:00 2026-02-25T09:15:10.071 INFO:teuthology.orchestra.run.trial094.stdout:(33/42): python3-jaraco-context-6.0.1-3.el9.noa 753 kB/s | 20 kB 00:00 2026-02-25T09:15:10.096 INFO:teuthology.orchestra.run.trial094.stdout:(34/42): python3-jaraco-functools-3.5.0-2.el9.n 793 kB/s | 19 kB 00:00 2026-02-25T09:15:10.121 INFO:teuthology.orchestra.run.trial094.stdout:(35/42): python3-jaraco-text-4.0.0-2.el9.noarch 1.0 MB/s | 26 kB 00:00 2026-02-25T09:15:10.146 INFO:teuthology.orchestra.run.trial094.stdout:(36/42): python3-more-itertools-8.12.0-2.el9.no 3.0 MB/s | 79 kB 00:00 2026-02-25T09:15:10.171 INFO:teuthology.orchestra.run.trial094.stdout:(37/42): python3-portend-3.1.0-2.el9.noarch.rpm 660 kB/s | 16 kB 00:00 2026-02-25T09:15:10.197 INFO:teuthology.orchestra.run.trial094.stdout:(38/42): python3-pyOpenSSL-21.0.0-1.el9.noarch. 3.4 MB/s | 90 kB 00:00 2026-02-25T09:15:10.222 INFO:teuthology.orchestra.run.trial094.stdout:(39/42): python3-tempora-5.0.0-2.el9.noarch.rpm 1.4 MB/s | 36 kB 00:00 2026-02-25T09:15:10.246 INFO:teuthology.orchestra.run.trial094.stdout:(40/42): python3-zc-lockfile-2.0-10.el9.noarch. 816 kB/s | 20 kB 00:00 2026-02-25T09:15:23.019 INFO:teuthology.orchestra.run.trial094.stdout:(41/42): ceph-osd-classic-20.3.0-5536.ge6eec768 791 kB/s | 15 MB 00:19 2026-02-25T09:16:43.903 INFO:teuthology.orchestra.run.trial151.stdout:(42/42): ceph-osd-crimson-20.3.0-5536.ge6eec768 741 kB/s | 87 MB 01:59 2026-02-25T09:16:43.906 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:16:43.906 INFO:teuthology.orchestra.run.trial151.stdout:Total 920 kB/s | 114 MB 02:07 2026-02-25T09:16:44.020 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:16:44.040 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:16:44.040 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:16:44.437 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:16:44.438 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:16:44.861 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:16:44.863 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/42 2026-02-25T09:16:44.869 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/42 2026-02-25T09:16:44.873 INFO:teuthology.orchestra.run.trial151.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/42 2026-02-25T09:16:44.875 INFO:teuthology.orchestra.run.trial151.stdout: Installing : unzip-6.0-59.el9.x86_64 4/42 2026-02-25T09:16:44.881 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 5/42 2026-02-25T09:16:44.905 INFO:teuthology.orchestra.run.trial151.stdout: Installing : zip-3.0-35.el9.x86_64 6/42 2026-02-25T09:16:44.908 INFO:teuthology.orchestra.run.trial151.stdout: Installing : luarocks-3.9.2-5.el9.noarch 7/42 2026-02-25T09:16:44.913 INFO:teuthology.orchestra.run.trial151.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 8/42 2026-02-25T09:16:44.918 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 9/42 2026-02-25T09:16:44.924 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 10/42 2026-02-25T09:16:44.928 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 11/42 2026-02-25T09:16:44.933 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 12/42 2026-02-25T09:16:44.939 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 13/42 2026-02-25T09:16:44.942 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 14/42 2026-02-25T09:16:44.948 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 15/42 2026-02-25T09:16:44.951 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 16/42 2026-02-25T09:16:44.958 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 17/42 2026-02-25T09:16:44.962 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 18/42 2026-02-25T09:16:44.984 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 19/42 2026-02-25T09:16:44.997 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 20/42 2026-02-25T09:16:45.002 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 21/42 2026-02-25T09:16:45.012 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 22/42 2026-02-25T09:16:45.018 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-25T09:16:45.032 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/42 2026-02-25T09:16:45.047 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/42 2026-02-25T09:16:45.123 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/42 2026-02-25T09:16:45.131 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 27/42 2026-02-25T09:16:45.155 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/42 2026-02-25T09:16:45.184 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 29/42 2026-02-25T09:16:45.190 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/42 2026-02-25T09:16:45.241 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/42 2026-02-25T09:16:45.280 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5536.ge6eec768.el9. 32/42 2026-02-25T09:16:45.281 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-mgr-2:20.3.0-5536.ge6eec768.el9.x86_64 33/42 2026-02-25T09:16:45.303 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5536.ge6eec768.el9.x86_64 33/42 2026-02-25T09:16:45.303 INFO:teuthology.orchestra.run.trial151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T09:16:45.304 INFO:teuthology.orchestra.run.trial151.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-25T09:16:45.304 INFO:teuthology.orchestra.run.trial151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T09:16:45.304 INFO:teuthology.orchestra.run.trial151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T09:16:45.304 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:45.309 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 34/42 2026-02-25T09:16:45.324 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-25T09:16:45.325 INFO:teuthology.orchestra.run.trial151.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-25T09:16:45.325 INFO:teuthology.orchestra.run.trial151.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-25T09:16:45.325 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:45.333 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-25T09:16:45.355 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-25T09:16:45.355 INFO:teuthology.orchestra.run.trial151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-25T09:16:45.355 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:45.953 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 36/42 2026-02-25T09:16:51.302 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-osd-classic-2:20.3.0-5536.ge6eec768.el9.x86_6 37/42 2026-02-25T09:16:51.312 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-osd-crimson-2:20.3.0-5536.ge6eec768.el9.x86_6 38/42 2026-02-25T09:16:51.313 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-osd-2:20.3.0-5536.ge6eec768.el9.x86_64 39/42 2026-02-25T09:16:51.336 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-osd-2:20.3.0-5536.ge6eec768.el9.x86_64 39/42 2026-02-25T09:16:51.336 INFO:teuthology.orchestra.run.trial151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T09:16:51.336 INFO:teuthology.orchestra.run.trial151.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-25T09:16:51.336 INFO:teuthology.orchestra.run.trial151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T09:16:51.336 INFO:teuthology.orchestra.run.trial151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T09:16:51.336 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:51.540 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-mon-2:20.3.0-5536.ge6eec768.el9.x86_64 40/42 2026-02-25T09:16:51.561 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-mon-2:20.3.0-5536.ge6eec768.el9.x86_64 40/42 2026-02-25T09:16:51.561 INFO:teuthology.orchestra.run.trial151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T09:16:51.561 INFO:teuthology.orchestra.run.trial151.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-25T09:16:51.561 INFO:teuthology.orchestra.run.trial151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T09:16:51.561 INFO:teuthology.orchestra.run.trial151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T09:16:51.562 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:51.644 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-mds-2:20.3.0-5536.ge6eec768.el9.x86_64 41/42 2026-02-25T09:16:51.664 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-mds-2:20.3.0-5536.ge6eec768.el9.x86_64 41/42 2026-02-25T09:16:51.664 INFO:teuthology.orchestra.run.trial151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T09:16:51.664 INFO:teuthology.orchestra.run.trial151.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-25T09:16:51.664 INFO:teuthology.orchestra.run.trial151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T09:16:51.664 INFO:teuthology.orchestra.run.trial151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T09:16:51.664 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:51.665 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-2:20.3.0-5536.ge6eec768.el9.x86_64 42/42 2026-02-25T09:16:51.671 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5536.ge6eec768.el9.x86_6 42/42 2026-02-25T09:16:51.694 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5536.ge6eec768.el9.x86_6 42/42 2026-02-25T09:16:52.667 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-2:20.3.0-5536.ge6eec768.el9.x86_64 42/42 2026-02-25T09:16:52.667 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-2:20.3.0-5536.ge6eec768.el9.x86_64 1/42 2026-02-25T09:16:52.668 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-mds-2:20.3.0-5536.ge6eec768.el9.x86_64 2/42 2026-02-25T09:16:52.668 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-mgr-2:20.3.0-5536.ge6eec768.el9.x86_64 3/42 2026-02-25T09:16:52.668 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-mon-2:20.3.0-5536.ge6eec768.el9.x86_64 4/42 2026-02-25T09:16:52.668 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-osd-2:20.3.0-5536.ge6eec768.el9.x86_64 5/42 2026-02-25T09:16:52.668 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-osd-classic-2:20.3.0-5536.ge6eec768.el9.x86_6 6/42 2026-02-25T09:16:52.668 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5536.ge6eec768.el9.x86_6 7/42 2026-02-25T09:16:52.668 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5536.ge6eec768.el9. 8/42 2026-02-25T09:16:52.668 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 9/42 2026-02-25T09:16:52.668 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 10/42 2026-02-25T09:16:52.668 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/42 2026-02-25T09:16:52.668 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 12/42 2026-02-25T09:16:52.668 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/42 2026-02-25T09:16:52.669 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/42 2026-02-25T09:16:52.669 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 15/42 2026-02-25T09:16:52.669 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 16/42 2026-02-25T09:16:52.669 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 17/42 2026-02-25T09:16:52.669 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : unzip-6.0-59.el9.x86_64 18/42 2026-02-25T09:16:52.669 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : zip-3.0-35.el9.x86_64 19/42 2026-02-25T09:16:52.669 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 20/42 2026-02-25T09:16:52.669 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : lua-5.4.4-4.el9.x86_64 21/42 2026-02-25T09:16:52.669 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 22/42 2026-02-25T09:16:52.669 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-25T09:16:52.669 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 24/42 2026-02-25T09:16:52.669 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 25/42 2026-02-25T09:16:52.669 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 26/42 2026-02-25T09:16:52.670 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 27/42 2026-02-25T09:16:52.670 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 28/42 2026-02-25T09:16:52.670 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 29/42 2026-02-25T09:16:52.670 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 30/42 2026-02-25T09:16:52.670 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 31/42 2026-02-25T09:16:52.670 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 32/42 2026-02-25T09:16:52.670 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 33/42 2026-02-25T09:16:52.670 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 34/42 2026-02-25T09:16:52.670 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 35/42 2026-02-25T09:16:52.670 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 36/42 2026-02-25T09:16:52.670 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 37/42 2026-02-25T09:16:52.670 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 38/42 2026-02-25T09:16:52.670 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 39/42 2026-02-25T09:16:52.671 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 40/42 2026-02-25T09:16:52.671 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 41/42 2026-02-25T09:16:52.731 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 42/42 2026-02-25T09:16:52.731 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:52.731 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:16:52.731 INFO:teuthology.orchestra.run.trial151.stdout: ceph-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:16:52.731 INFO:teuthology.orchestra.run.trial151.stdout: ceph-mds-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:16:52.732 INFO:teuthology.orchestra.run.trial151.stdout: ceph-mgr-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:16:52.732 INFO:teuthology.orchestra.run.trial151.stdout: ceph-mgr-modules-core-2:20.3.0-5536.ge6eec768.el9.noarch 2026-02-25T09:16:52.732 INFO:teuthology.orchestra.run.trial151.stdout: ceph-mon-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:16:52.732 INFO:teuthology.orchestra.run.trial151.stdout: ceph-osd-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:16:52.732 INFO:teuthology.orchestra.run.trial151.stdout: ceph-osd-classic-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:16:52.732 INFO:teuthology.orchestra.run.trial151.stdout: ceph-osd-crimson-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:16:52.732 INFO:teuthology.orchestra.run.trial151.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-25T09:16:52.732 INFO:teuthology.orchestra.run.trial151.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-25T09:16:52.732 INFO:teuthology.orchestra.run.trial151.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T09:16:52.732 INFO:teuthology.orchestra.run.trial151.stdout: lua-5.4.4-4.el9.x86_64 2026-02-25T09:16:52.733 INFO:teuthology.orchestra.run.trial151.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-25T09:16:52.733 INFO:teuthology.orchestra.run.trial151.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-25T09:16:52.733 INFO:teuthology.orchestra.run.trial151.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-25T09:16:52.733 INFO:teuthology.orchestra.run.trial151.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-25T09:16:52.733 INFO:teuthology.orchestra.run.trial151.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-25T09:16:52.733 INFO:teuthology.orchestra.run.trial151.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-25T09:16:52.733 INFO:teuthology.orchestra.run.trial151.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-25T09:16:52.733 INFO:teuthology.orchestra.run.trial151.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-25T09:16:52.733 INFO:teuthology.orchestra.run.trial151.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-25T09:16:52.733 INFO:teuthology.orchestra.run.trial151.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-25T09:16:52.733 INFO:teuthology.orchestra.run.trial151.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-25T09:16:52.733 INFO:teuthology.orchestra.run.trial151.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-25T09:16:52.734 INFO:teuthology.orchestra.run.trial151.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-25T09:16:52.734 INFO:teuthology.orchestra.run.trial151.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-25T09:16:52.734 INFO:teuthology.orchestra.run.trial151.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-25T09:16:52.734 INFO:teuthology.orchestra.run.trial151.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T09:16:52.734 INFO:teuthology.orchestra.run.trial151.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-25T09:16:52.734 INFO:teuthology.orchestra.run.trial151.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-25T09:16:52.734 INFO:teuthology.orchestra.run.trial151.stdout: python3-ply-3.11-14.el9.noarch 2026-02-25T09:16:52.734 INFO:teuthology.orchestra.run.trial151.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-25T09:16:52.734 INFO:teuthology.orchestra.run.trial151.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-25T09:16:52.734 INFO:teuthology.orchestra.run.trial151.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-25T09:16:52.734 INFO:teuthology.orchestra.run.trial151.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-25T09:16:52.734 INFO:teuthology.orchestra.run.trial151.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-25T09:16:52.734 INFO:teuthology.orchestra.run.trial151.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-25T09:16:52.734 INFO:teuthology.orchestra.run.trial151.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-25T09:16:52.735 INFO:teuthology.orchestra.run.trial151.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-25T09:16:52.735 INFO:teuthology.orchestra.run.trial151.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-25T09:16:52.735 INFO:teuthology.orchestra.run.trial151.stdout: unzip-6.0-59.el9.x86_64 2026-02-25T09:16:52.735 INFO:teuthology.orchestra.run.trial151.stdout: zip-3.0-35.el9.x86_64 2026-02-25T09:16:52.735 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:52.735 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:16:52.775 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-base 2026-02-25T09:16:53.092 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:04:41 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:16:53.157 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-base-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:16:53.175 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:16:53.176 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:16:53.176 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:16:53.198 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install cephadm 2026-02-25T09:16:53.508 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:04:41 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:16:53.591 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:16:53.591 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:16:53.591 INFO:teuthology.orchestra.run.trial151.stdout: Package Arch Version Repository Size 2026-02-25T09:16:53.591 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:16:53.591 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:16:53.591 INFO:teuthology.orchestra.run.trial151.stdout: cephadm noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 1.0 M 2026-02-25T09:16:53.591 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:53.592 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:16:53.592 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:16:53.592 INFO:teuthology.orchestra.run.trial151.stdout:Install 1 Package 2026-02-25T09:16:53.592 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:53.592 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 1.0 M 2026-02-25T09:16:53.592 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 1.0 M 2026-02-25T09:16:53.592 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:16:54.881 INFO:teuthology.orchestra.run.trial151.stdout:cephadm-20.3.0-5536.ge6eec768.el9.noarch.rpm 808 kB/s | 1.0 MB 00:01 2026-02-25T09:16:54.882 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:16:54.882 INFO:teuthology.orchestra.run.trial151.stdout:Total 808 kB/s | 1.0 MB 00:01 2026-02-25T09:16:54.882 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:16:54.885 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:16:54.885 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:16:54.889 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:16:54.889 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:16:54.908 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:16:54.969 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 1/1 2026-02-25T09:16:54.984 INFO:teuthology.orchestra.run.trial151.stdout: Installing : cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 1/1 2026-02-25T09:16:55.055 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 1/1 2026-02-25T09:16:55.114 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 1/1 2026-02-25T09:16:55.115 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:55.115 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:16:55.115 INFO:teuthology.orchestra.run.trial151.stdout: cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 2026-02-25T09:16:55.115 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:55.115 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:16:55.143 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T09:16:55.465 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:04:43 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:16:55.547 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:16:55.548 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:16:55.548 INFO:teuthology.orchestra.run.trial151.stdout: Package Arch Version Repo Size 2026-02-25T09:16:55.548 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:16:55.548 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:16:55.548 INFO:teuthology.orchestra.run.trial151.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 149 k 2026-02-25T09:16:55.549 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:55.549 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:16:55.549 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:16:55.549 INFO:teuthology.orchestra.run.trial151.stdout:Install 1 Package 2026-02-25T09:16:55.549 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:55.549 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 149 k 2026-02-25T09:16:55.549 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 455 k 2026-02-25T09:16:55.549 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:16:56.112 INFO:teuthology.orchestra.run.trial151.stdout:ceph-immutable-object-cache-20.3.0-5536.ge6eec7 265 kB/s | 149 kB 00:00 2026-02-25T09:16:56.112 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:16:56.112 INFO:teuthology.orchestra.run.trial151.stdout:Total 265 kB/s | 149 kB 00:00 2026-02-25T09:16:56.112 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:16:56.117 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:16:56.117 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:16:56.143 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:16:56.143 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:16:56.192 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:16:56.194 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5536.ge6eec768. 1/1 2026-02-25T09:16:56.217 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5536.ge6eec768. 1/1 2026-02-25T09:16:56.217 INFO:teuthology.orchestra.run.trial151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T09:16:56.218 INFO:teuthology.orchestra.run.trial151.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-25T09:16:56.218 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:56.533 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5536.ge6eec768. 1/1 2026-02-25T09:16:56.533 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:56.533 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:16:56.533 INFO:teuthology.orchestra.run.trial151.stdout: ceph-immutable-object-cache-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:16:56.533 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:56.533 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:16:56.561 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-mgr 2026-02-25T09:16:56.877 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:04:44 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:16:56.942 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-mgr-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:16:56.960 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:16:56.960 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:16:56.960 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:16:56.982 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T09:16:57.293 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:04:45 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:16:57.376 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:16:57.376 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:16:57.376 INFO:teuthology.orchestra.run.trial151.stdout: Package Arch Version Repository Size 2026-02-25T09:16:57.376 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:16:57.377 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:16:57.377 INFO:teuthology.orchestra.run.trial151.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 15 M 2026-02-25T09:16:57.377 INFO:teuthology.orchestra.run.trial151.stdout:Installing dependencies: 2026-02-25T09:16:57.377 INFO:teuthology.orchestra.run.trial151.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-25T09:16:57.377 INFO:teuthology.orchestra.run.trial151.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 57 k 2026-02-25T09:16:57.377 INFO:teuthology.orchestra.run.trial151.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 17 k 2026-02-25T09:16:57.377 INFO:teuthology.orchestra.run.trial151.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-25T09:16:57.377 INFO:teuthology.orchestra.run.trial151.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-25T09:16:57.378 INFO:teuthology.orchestra.run.trial151.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-25T09:16:57.378 INFO:teuthology.orchestra.run.trial151.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-25T09:16:57.378 INFO:teuthology.orchestra.run.trial151.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-25T09:16:57.378 INFO:teuthology.orchestra.run.trial151.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-25T09:16:57.378 INFO:teuthology.orchestra.run.trial151.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-25T09:16:57.378 INFO:teuthology.orchestra.run.trial151.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-25T09:16:57.378 INFO:teuthology.orchestra.run.trial151.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-25T09:16:57.378 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:57.378 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:16:57.378 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:16:57.378 INFO:teuthology.orchestra.run.trial151.stdout:Install 13 Packages 2026-02-25T09:16:57.378 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:16:57.379 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 19 M 2026-02-25T09:16:57.379 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 187 M 2026-02-25T09:16:57.379 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:16:58.501 INFO:teuthology.orchestra.run.trial151.stdout:(1/13): ceph-prometheus-alerts-20.3.0-5536.ge6e 50 kB/s | 17 kB 00:00 2026-02-25T09:16:58.548 INFO:teuthology.orchestra.run.trial151.stdout:(2/13): python3-jmespath-1.0.1-1.el9.noarch.rpm 1.0 MB/s | 48 kB 00:00 2026-02-25T09:16:58.572 INFO:teuthology.orchestra.run.trial151.stdout:(3/13): ceph-grafana-dashboards-20.3.0-5536.ge6 137 kB/s | 57 kB 00:00 2026-02-25T09:16:58.584 INFO:teuthology.orchestra.run.trial151.stdout:(4/13): python3-protobuf-3.14.0-17.el9.noarch.r 7.2 MB/s | 267 kB 00:00 2026-02-25T09:16:58.810 INFO:teuthology.orchestra.run.trial151.stdout:(5/13): protobuf-compiler-3.14.0-17.el9.x86_64. 3.5 MB/s | 862 kB 00:00 2026-02-25T09:16:59.095 INFO:teuthology.orchestra.run.trial151.stdout:(6/13): grpc-data-1.46.7-10.el9.noarch.rpm 68 kB/s | 19 kB 00:00 2026-02-25T09:16:59.167 INFO:teuthology.orchestra.run.trial151.stdout:(7/13): abseil-cpp-20211102.0-4.el9.x86_64.rpm 946 kB/s | 551 kB 00:00 2026-02-25T09:16:59.313 INFO:teuthology.orchestra.run.trial151.stdout:(8/13): python3-grpcio-1.46.7-10.el9.x86_64.rpm 9.3 MB/s | 2.0 MB 00:00 2026-02-25T09:16:59.314 INFO:teuthology.orchestra.run.trial151.stdout:(9/13): python3-grpcio-tools-1.46.7-10.el9.x86_ 984 kB/s | 144 kB 00:00 2026-02-25T09:16:59.386 INFO:teuthology.orchestra.run.trial151.stdout:(10/13): python3-repoze-lru-0.7-16.el9.noarch.r 426 kB/s | 31 kB 00:00 2026-02-25T09:16:59.390 INFO:teuthology.orchestra.run.trial151.stdout:(11/13): python3-routes-2.5.1-5.el9.noarch.rpm 2.4 MB/s | 188 kB 00:00 2026-02-25T09:16:59.458 INFO:teuthology.orchestra.run.trial151.stdout:(12/13): python3-xmltodict-0.12.0-15.el9.noarch 307 kB/s | 22 kB 00:00 2026-02-25T09:17:05.388 INFO:teuthology.orchestra.run.trial094.stdout:(42/42): ceph-osd-crimson-20.3.0-5536.ge6eec768 753 kB/s | 87 MB 01:57 2026-02-25T09:17:05.389 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:05.389 INFO:teuthology.orchestra.run.trial094.stdout:Total 951 kB/s | 114 MB 02:03 2026-02-25T09:17:05.508 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:17:05.527 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:17:05.527 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:17:05.920 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:17:05.920 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:17:06.343 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:17:06.345 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-more-itertools-8.12.0-2.el9.noarch 1/42 2026-02-25T09:17:06.350 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-jaraco-8.2.1-3.el9.noarch 2/42 2026-02-25T09:17:06.355 INFO:teuthology.orchestra.run.trial094.stdout: Installing : lua-5.4.4-4.el9.x86_64 3/42 2026-02-25T09:17:06.357 INFO:teuthology.orchestra.run.trial094.stdout: Installing : unzip-6.0-59.el9.x86_64 4/42 2026-02-25T09:17:06.363 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ledmon-libs-1.1.0-3.el9.x86_64 5/42 2026-02-25T09:17:06.385 INFO:teuthology.orchestra.run.trial151.stdout:(13/13): ceph-mgr-dashboard-20.3.0-5536.ge6eec7 1.8 MB/s | 15 MB 00:08 2026-02-25T09:17:06.387 INFO:teuthology.orchestra.run.trial094.stdout: Installing : zip-3.0-35.el9.x86_64 6/42 2026-02-25T09:17:06.388 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:06.388 INFO:teuthology.orchestra.run.trial151.stdout:Total 2.1 MB/s | 19 MB 00:09 2026-02-25T09:17:06.390 INFO:teuthology.orchestra.run.trial094.stdout: Installing : luarocks-3.9.2-5.el9.noarch 7/42 2026-02-25T09:17:06.395 INFO:teuthology.orchestra.run.trial094.stdout: Installing : lua-devel-5.4.4-4.el9.x86_64 8/42 2026-02-25T09:17:06.399 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-jaraco-classes-3.2.1-5.el9.noarch 9/42 2026-02-25T09:17:06.405 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-zc-lockfile-2.0-10.el9.noarch 10/42 2026-02-25T09:17:06.409 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-backports-tarfile-1.2.0-1.el9.noarch 11/42 2026-02-25T09:17:06.414 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:17:06.415 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-jaraco-context-6.0.1-3.el9.noarch 12/42 2026-02-25T09:17:06.420 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-autocommand-2.2.2-8.el9.noarch 13/42 2026-02-25T09:17:06.424 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-toml-0.10.2-6.el9.noarch 14/42 2026-02-25T09:17:06.424 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:17:06.424 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:17:06.429 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-jaraco-functools-3.5.0-2.el9.noarch 15/42 2026-02-25T09:17:06.433 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-jaraco-text-4.0.0-2.el9.noarch 16/42 2026-02-25T09:17:06.439 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-jaraco-collections-3.0.0-8.el9.noarch 17/42 2026-02-25T09:17:06.443 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-tempora-5.0.0-2.el9.noarch 18/42 2026-02-25T09:17:06.466 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-portend-3.1.0-2.el9.noarch 19/42 2026-02-25T09:17:06.478 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-urllib3-1.26.5-7.el9.noarch 20/42 2026-02-25T09:17:06.483 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-requests-2.25.1-10.el9.noarch 21/42 2026-02-25T09:17:06.493 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-pyparsing-2.4.7-9.el9.noarch 22/42 2026-02-25T09:17:06.499 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-25T09:17:06.512 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-ply-3.11-14.el9.noarch 24/42 2026-02-25T09:17:06.528 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-pycparser-2.20-6.el9.noarch 25/42 2026-02-25T09:17:06.532 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:17:06.532 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:17:06.602 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-cffi-1.14.5-5.el9.x86_64 26/42 2026-02-25T09:17:06.611 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-cryptography-36.0.1-5.el9.x86_64 27/42 2026-02-25T09:17:06.634 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-pyOpenSSL-21.0.0-1.el9.noarch 28/42 2026-02-25T09:17:06.655 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:17:06.660 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 1/13 2026-02-25T09:17:06.663 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-cheroot-10.0.1-4.el9.noarch 29/42 2026-02-25T09:17:06.668 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-cherrypy-18.6.1-2.el9.noarch 30/42 2026-02-25T09:17:06.671 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 2/13 2026-02-25T09:17:06.673 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 3/13 2026-02-25T09:17:06.702 INFO:teuthology.orchestra.run.trial151.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 4/13 2026-02-25T09:17:06.720 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-bcrypt-3.2.2-1.el9.x86_64 31/42 2026-02-25T09:17:06.747 INFO:teuthology.orchestra.run.trial151.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 5/13 2026-02-25T09:17:06.756 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 6/13 2026-02-25T09:17:06.758 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mgr-modules-core-2:20.3.0-5536.ge6eec768.el9. 32/42 2026-02-25T09:17:06.759 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mgr-2:20.3.0-5536.ge6eec768.el9.x86_64 33/42 2026-02-25T09:17:06.782 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mgr-2:20.3.0-5536.ge6eec768.el9.x86_64 33/42 2026-02-25T09:17:06.782 INFO:teuthology.orchestra.run.trial094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T09:17:06.782 INFO:teuthology.orchestra.run.trial094.stdout:Invalid unit name "ceph-mgr@*.service" escaped as "ceph-mgr@\x2a.service". 2026-02-25T09:17:06.782 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T09:17:06.782 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mgr.target → /usr/lib/systemd/system/ceph-mgr.target. 2026-02-25T09:17:06.783 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:06.784 INFO:teuthology.orchestra.run.trial151.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 7/13 2026-02-25T09:17:06.786 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libconfig-1.7.2-9.el9.x86_64 34/42 2026-02-25T09:17:06.795 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 8/13 2026-02-25T09:17:06.802 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/13 2026-02-25T09:17:06.802 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-25T09:17:06.802 INFO:teuthology.orchestra.run.trial094.stdout:Creating group 'libstoragemgmt' with GID 991. 2026-02-25T09:17:06.802 INFO:teuthology.orchestra.run.trial094.stdout:Creating user 'libstoragemgmt' (daemon account for libstoragemgmt) with UID 991 and GID 991. 2026-02-25T09:17:06.802 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:06.804 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 10/13 2026-02-25T09:17:06.810 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-25T09:17:06.813 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5536.ge6eec768.el9 11/13 2026-02-25T09:17:06.832 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: libstoragemgmt-1.10.1-1.el9.x86_64 35/42 2026-02-25T09:17:06.832 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/libstoragemgmt.service → /usr/lib/systemd/system/libstoragemgmt.service. 2026-02-25T09:17:06.832 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:07.435 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-libstoragemgmt-1.10.1-1.el9.x86_64 36/42 2026-02-25T09:17:08.374 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5536.ge6eec768.el 12/13 2026-02-25T09:17:08.381 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5536.ge6eec768.el9.noa 13/13 2026-02-25T09:17:08.619 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5536.ge6eec768.el9.noa 13/13 2026-02-25T09:17:08.619 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5536.ge6eec768.el 1/13 2026-02-25T09:17:08.619 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5536.ge6eec768.el9.noa 2/13 2026-02-25T09:17:08.620 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5536.ge6eec768.el9 3/13 2026-02-25T09:17:08.620 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 4/13 2026-02-25T09:17:08.620 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 5/13 2026-02-25T09:17:08.620 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 6/13 2026-02-25T09:17:08.620 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 7/13 2026-02-25T09:17:08.620 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/13 2026-02-25T09:17:08.620 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/13 2026-02-25T09:17:08.620 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/13 2026-02-25T09:17:08.620 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/13 2026-02-25T09:17:08.620 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/13 2026-02-25T09:17:08.679 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 13/13 2026-02-25T09:17:08.679 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:08.680 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:17:08.680 INFO:teuthology.orchestra.run.trial151.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-25T09:17:08.680 INFO:teuthology.orchestra.run.trial151.stdout: ceph-grafana-dashboards-2:20.3.0-5536.ge6eec768.el9.noarch 2026-02-25T09:17:08.680 INFO:teuthology.orchestra.run.trial151.stdout: ceph-mgr-dashboard-2:20.3.0-5536.ge6eec768.el9.noarch 2026-02-25T09:17:08.680 INFO:teuthology.orchestra.run.trial151.stdout: ceph-prometheus-alerts-2:20.3.0-5536.ge6eec768.el9.noarch 2026-02-25T09:17:08.680 INFO:teuthology.orchestra.run.trial151.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-25T09:17:08.680 INFO:teuthology.orchestra.run.trial151.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-25T09:17:08.680 INFO:teuthology.orchestra.run.trial151.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-25T09:17:08.680 INFO:teuthology.orchestra.run.trial151.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-25T09:17:08.680 INFO:teuthology.orchestra.run.trial151.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-25T09:17:08.680 INFO:teuthology.orchestra.run.trial151.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-25T09:17:08.681 INFO:teuthology.orchestra.run.trial151.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-25T09:17:08.681 INFO:teuthology.orchestra.run.trial151.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-25T09:17:08.681 INFO:teuthology.orchestra.run.trial151.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-25T09:17:08.681 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:08.681 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:08.713 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T09:17:09.033 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:04:57 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:09.117 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:09.118 INFO:teuthology.orchestra.run.trial151.stdout:======================================================================================= 2026-02-25T09:17:09.119 INFO:teuthology.orchestra.run.trial151.stdout: Package Arch Version Repository Size 2026-02-25T09:17:09.119 INFO:teuthology.orchestra.run.trial151.stdout:======================================================================================= 2026-02-25T09:17:09.119 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:17:09.119 INFO:teuthology.orchestra.run.trial151.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 7.4 M 2026-02-25T09:17:09.119 INFO:teuthology.orchestra.run.trial151.stdout:Installing dependencies: 2026-02-25T09:17:09.119 INFO:teuthology.orchestra.run.trial151.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-25T09:17:09.119 INFO:teuthology.orchestra.run.trial151.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-25T09:17:09.119 INFO:teuthology.orchestra.run.trial151.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-25T09:17:09.119 INFO:teuthology.orchestra.run.trial151.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-25T09:17:09.119 INFO:teuthology.orchestra.run.trial151.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-25T09:17:09.120 INFO:teuthology.orchestra.run.trial151.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-25T09:17:09.120 INFO:teuthology.orchestra.run.trial151.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-25T09:17:09.120 INFO:teuthology.orchestra.run.trial151.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-25T09:17:09.120 INFO:teuthology.orchestra.run.trial151.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-25T09:17:09.120 INFO:teuthology.orchestra.run.trial151.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-25T09:17:09.120 INFO:teuthology.orchestra.run.trial151.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-25T09:17:09.120 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:09.120 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:17:09.120 INFO:teuthology.orchestra.run.trial151.stdout:======================================================================================= 2026-02-25T09:17:09.120 INFO:teuthology.orchestra.run.trial151.stdout:Install 12 Packages 2026-02-25T09:17:09.120 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:09.121 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 43 M 2026-02-25T09:17:09.121 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 234 M 2026-02-25T09:17:09.121 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:17:09.574 INFO:teuthology.orchestra.run.trial151.stdout:(1/12): libquadmath-11.5.0-14.el9.x86_64.rpm 2.3 MB/s | 184 kB 00:00 2026-02-25T09:17:09.608 INFO:teuthology.orchestra.run.trial151.stdout:(2/12): libgfortran-11.5.0-14.el9.x86_64.rpm 6.8 MB/s | 794 kB 00:00 2026-02-25T09:17:10.131 INFO:teuthology.orchestra.run.trial151.stdout:(3/12): flexiblas-3.0.4-9.el9.x86_64.rpm 53 kB/s | 30 kB 00:00 2026-02-25T09:17:10.289 INFO:teuthology.orchestra.run.trial151.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-9.el9.x 94 kB/s | 15 kB 00:00 2026-02-25T09:17:10.525 INFO:teuthology.orchestra.run.trial151.stdout:(5/12): openblas-0.3.29-1.el9.x86_64.rpm 178 kB/s | 42 kB 00:00 2026-02-25T09:17:10.761 INFO:teuthology.orchestra.run.trial151.stdout:(6/12): flexiblas-netlib-3.0.4-9.el9.x86_64.rpm 2.6 MB/s | 3.0 MB 00:01 2026-02-25T09:17:10.924 INFO:teuthology.orchestra.run.trial151.stdout:(7/12): python3-devel-3.9.25-3.el9.x86_64.rpm 1.5 MB/s | 244 kB 00:00 2026-02-25T09:17:11.163 INFO:teuthology.orchestra.run.trial151.stdout:(8/12): openblas-openmp-0.3.29-1.el9.x86_64.rpm 8.3 MB/s | 5.3 MB 00:00 2026-02-25T09:17:11.247 INFO:teuthology.orchestra.run.trial151.stdout:(9/12): python3-numpy-1.23.5-2.el9.x86_64.rpm 19 MB/s | 6.1 MB 00:00 2026-02-25T09:17:11.329 INFO:teuthology.orchestra.run.trial151.stdout:(10/12): python3-numpy-f2py-1.23.5-2.el9.x86_64 2.6 MB/s | 442 kB 00:00 2026-02-25T09:17:11.828 INFO:teuthology.orchestra.run.trial151.stdout:(11/12): python3-scipy-1.9.3-2.el9.x86_64.rpm 33 MB/s | 19 MB 00:00 2026-02-25T09:17:12.794 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-osd-classic-2:20.3.0-5536.ge6eec768.el9.x86_6 37/42 2026-02-25T09:17:12.804 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-osd-crimson-2:20.3.0-5536.ge6eec768.el9.x86_6 38/42 2026-02-25T09:17:12.805 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-osd-2:20.3.0-5536.ge6eec768.el9.x86_64 39/42 2026-02-25T09:17:12.829 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-osd-2:20.3.0-5536.ge6eec768.el9.x86_64 39/42 2026-02-25T09:17:12.829 INFO:teuthology.orchestra.run.trial094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T09:17:12.829 INFO:teuthology.orchestra.run.trial094.stdout:Invalid unit name "ceph-osd@*.service" escaped as "ceph-osd@\x2a.service". 2026-02-25T09:17:12.829 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T09:17:12.829 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-osd.target → /usr/lib/systemd/system/ceph-osd.target. 2026-02-25T09:17:12.830 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:13.037 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mon-2:20.3.0-5536.ge6eec768.el9.x86_64 40/42 2026-02-25T09:17:13.057 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mon-2:20.3.0-5536.ge6eec768.el9.x86_64 40/42 2026-02-25T09:17:13.057 INFO:teuthology.orchestra.run.trial094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T09:17:13.057 INFO:teuthology.orchestra.run.trial094.stdout:Invalid unit name "ceph-mon@*.service" escaped as "ceph-mon@\x2a.service". 2026-02-25T09:17:13.057 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T09:17:13.058 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mon.target → /usr/lib/systemd/system/ceph-mon.target. 2026-02-25T09:17:13.058 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:13.141 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mds-2:20.3.0-5536.ge6eec768.el9.x86_64 41/42 2026-02-25T09:17:13.162 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mds-2:20.3.0-5536.ge6eec768.el9.x86_64 41/42 2026-02-25T09:17:13.162 INFO:teuthology.orchestra.run.trial094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T09:17:13.162 INFO:teuthology.orchestra.run.trial094.stdout:Invalid unit name "ceph-mds@*.service" escaped as "ceph-mds@\x2a.service". 2026-02-25T09:17:13.162 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T09:17:13.162 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-mds.target → /usr/lib/systemd/system/ceph-mds.target. 2026-02-25T09:17:13.162 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:13.163 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-2:20.3.0-5536.ge6eec768.el9.x86_64 42/42 2026-02-25T09:17:13.168 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-osd-classic-2:20.3.0-5536.ge6eec768.el9.x86_6 42/42 2026-02-25T09:17:13.191 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-osd-crimson-2:20.3.0-5536.ge6eec768.el9.x86_6 42/42 2026-02-25T09:17:14.105 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-2:20.3.0-5536.ge6eec768.el9.x86_64 42/42 2026-02-25T09:17:14.106 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-2:20.3.0-5536.ge6eec768.el9.x86_64 1/42 2026-02-25T09:17:14.106 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mds-2:20.3.0-5536.ge6eec768.el9.x86_64 2/42 2026-02-25T09:17:14.106 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mgr-2:20.3.0-5536.ge6eec768.el9.x86_64 3/42 2026-02-25T09:17:14.106 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mon-2:20.3.0-5536.ge6eec768.el9.x86_64 4/42 2026-02-25T09:17:14.106 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-osd-2:20.3.0-5536.ge6eec768.el9.x86_64 5/42 2026-02-25T09:17:14.106 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-osd-classic-2:20.3.0-5536.ge6eec768.el9.x86_6 6/42 2026-02-25T09:17:14.106 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-osd-crimson-2:20.3.0-5536.ge6eec768.el9.x86_6 7/42 2026-02-25T09:17:14.106 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mgr-modules-core-2:20.3.0-5536.ge6eec768.el9. 8/42 2026-02-25T09:17:14.106 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ledmon-libs-1.1.0-3.el9.x86_64 9/42 2026-02-25T09:17:14.106 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libconfig-1.7.2-9.el9.x86_64 10/42 2026-02-25T09:17:14.106 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-cffi-1.14.5-5.el9.x86_64 11/42 2026-02-25T09:17:14.106 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-cryptography-36.0.1-5.el9.x86_64 12/42 2026-02-25T09:17:14.107 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-ply-3.11-14.el9.noarch 13/42 2026-02-25T09:17:14.107 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-pycparser-2.20-6.el9.noarch 14/42 2026-02-25T09:17:14.107 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-pyparsing-2.4.7-9.el9.noarch 15/42 2026-02-25T09:17:14.107 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-requests-2.25.1-10.el9.noarch 16/42 2026-02-25T09:17:14.107 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-urllib3-1.26.5-7.el9.noarch 17/42 2026-02-25T09:17:14.107 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : unzip-6.0-59.el9.x86_64 18/42 2026-02-25T09:17:14.107 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : zip-3.0-35.el9.x86_64 19/42 2026-02-25T09:17:14.107 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libstoragemgmt-1.10.1-1.el9.x86_64 20/42 2026-02-25T09:17:14.107 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : lua-5.4.4-4.el9.x86_64 21/42 2026-02-25T09:17:14.107 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-libstoragemgmt-1.10.1-1.el9.x86_64 22/42 2026-02-25T09:17:14.107 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-packaging-20.9-5.el9.noarch 23/42 2026-02-25T09:17:14.107 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-toml-0.10.2-6.el9.noarch 24/42 2026-02-25T09:17:14.107 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : lua-devel-5.4.4-4.el9.x86_64 25/42 2026-02-25T09:17:14.108 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : luarocks-3.9.2-5.el9.noarch 26/42 2026-02-25T09:17:14.108 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-autocommand-2.2.2-8.el9.noarch 27/42 2026-02-25T09:17:14.108 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-backports-tarfile-1.2.0-1.el9.noarch 28/42 2026-02-25T09:17:14.108 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-bcrypt-3.2.2-1.el9.x86_64 29/42 2026-02-25T09:17:14.108 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-cheroot-10.0.1-4.el9.noarch 30/42 2026-02-25T09:17:14.108 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-cherrypy-18.6.1-2.el9.noarch 31/42 2026-02-25T09:17:14.108 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-jaraco-8.2.1-3.el9.noarch 32/42 2026-02-25T09:17:14.108 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-jaraco-classes-3.2.1-5.el9.noarch 33/42 2026-02-25T09:17:14.108 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-jaraco-collections-3.0.0-8.el9.noarch 34/42 2026-02-25T09:17:14.108 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-jaraco-context-6.0.1-3.el9.noarch 35/42 2026-02-25T09:17:14.108 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-jaraco-functools-3.5.0-2.el9.noarch 36/42 2026-02-25T09:17:14.108 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-jaraco-text-4.0.0-2.el9.noarch 37/42 2026-02-25T09:17:14.108 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-more-itertools-8.12.0-2.el9.noarch 38/42 2026-02-25T09:17:14.109 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-portend-3.1.0-2.el9.noarch 39/42 2026-02-25T09:17:14.109 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-pyOpenSSL-21.0.0-1.el9.noarch 40/42 2026-02-25T09:17:14.109 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-tempora-5.0.0-2.el9.noarch 41/42 2026-02-25T09:17:14.166 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-zc-lockfile-2.0-10.el9.noarch 42/42 2026-02-25T09:17:14.166 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:14.166 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:17:14.167 INFO:teuthology.orchestra.run.trial094.stdout: ceph-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:14.167 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mds-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:14.167 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:14.167 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-modules-core-2:20.3.0-5536.ge6eec768.el9.noarch 2026-02-25T09:17:14.167 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mon-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:14.167 INFO:teuthology.orchestra.run.trial094.stdout: ceph-osd-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:14.167 INFO:teuthology.orchestra.run.trial094.stdout: ceph-osd-classic-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:14.167 INFO:teuthology.orchestra.run.trial094.stdout: ceph-osd-crimson-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:14.167 INFO:teuthology.orchestra.run.trial094.stdout: ledmon-libs-1.1.0-3.el9.x86_64 2026-02-25T09:17:14.167 INFO:teuthology.orchestra.run.trial094.stdout: libconfig-1.7.2-9.el9.x86_64 2026-02-25T09:17:14.167 INFO:teuthology.orchestra.run.trial094.stdout: libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T09:17:14.168 INFO:teuthology.orchestra.run.trial094.stdout: lua-5.4.4-4.el9.x86_64 2026-02-25T09:17:14.168 INFO:teuthology.orchestra.run.trial094.stdout: lua-devel-5.4.4-4.el9.x86_64 2026-02-25T09:17:14.168 INFO:teuthology.orchestra.run.trial094.stdout: luarocks-3.9.2-5.el9.noarch 2026-02-25T09:17:14.168 INFO:teuthology.orchestra.run.trial094.stdout: python3-autocommand-2.2.2-8.el9.noarch 2026-02-25T09:17:14.168 INFO:teuthology.orchestra.run.trial094.stdout: python3-backports-tarfile-1.2.0-1.el9.noarch 2026-02-25T09:17:14.168 INFO:teuthology.orchestra.run.trial094.stdout: python3-bcrypt-3.2.2-1.el9.x86_64 2026-02-25T09:17:14.168 INFO:teuthology.orchestra.run.trial094.stdout: python3-cffi-1.14.5-5.el9.x86_64 2026-02-25T09:17:14.168 INFO:teuthology.orchestra.run.trial094.stdout: python3-cheroot-10.0.1-4.el9.noarch 2026-02-25T09:17:14.168 INFO:teuthology.orchestra.run.trial094.stdout: python3-cherrypy-18.6.1-2.el9.noarch 2026-02-25T09:17:14.168 INFO:teuthology.orchestra.run.trial094.stdout: python3-cryptography-36.0.1-5.el9.x86_64 2026-02-25T09:17:14.168 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-8.2.1-3.el9.noarch 2026-02-25T09:17:14.168 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-classes-3.2.1-5.el9.noarch 2026-02-25T09:17:14.168 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-collections-3.0.0-8.el9.noarch 2026-02-25T09:17:14.169 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-context-6.0.1-3.el9.noarch 2026-02-25T09:17:14.169 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-functools-3.5.0-2.el9.noarch 2026-02-25T09:17:14.169 INFO:teuthology.orchestra.run.trial094.stdout: python3-jaraco-text-4.0.0-2.el9.noarch 2026-02-25T09:17:14.169 INFO:teuthology.orchestra.run.trial094.stdout: python3-libstoragemgmt-1.10.1-1.el9.x86_64 2026-02-25T09:17:14.169 INFO:teuthology.orchestra.run.trial094.stdout: python3-more-itertools-8.12.0-2.el9.noarch 2026-02-25T09:17:14.169 INFO:teuthology.orchestra.run.trial094.stdout: python3-packaging-20.9-5.el9.noarch 2026-02-25T09:17:14.169 INFO:teuthology.orchestra.run.trial094.stdout: python3-ply-3.11-14.el9.noarch 2026-02-25T09:17:14.169 INFO:teuthology.orchestra.run.trial094.stdout: python3-portend-3.1.0-2.el9.noarch 2026-02-25T09:17:14.169 INFO:teuthology.orchestra.run.trial094.stdout: python3-pyOpenSSL-21.0.0-1.el9.noarch 2026-02-25T09:17:14.170 INFO:teuthology.orchestra.run.trial094.stdout: python3-pycparser-2.20-6.el9.noarch 2026-02-25T09:17:14.170 INFO:teuthology.orchestra.run.trial094.stdout: python3-pyparsing-2.4.7-9.el9.noarch 2026-02-25T09:17:14.170 INFO:teuthology.orchestra.run.trial094.stdout: python3-requests-2.25.1-10.el9.noarch 2026-02-25T09:17:14.170 INFO:teuthology.orchestra.run.trial094.stdout: python3-tempora-5.0.0-2.el9.noarch 2026-02-25T09:17:14.170 INFO:teuthology.orchestra.run.trial094.stdout: python3-toml-0.10.2-6.el9.noarch 2026-02-25T09:17:14.170 INFO:teuthology.orchestra.run.trial094.stdout: python3-urllib3-1.26.5-7.el9.noarch 2026-02-25T09:17:14.170 INFO:teuthology.orchestra.run.trial094.stdout: python3-zc-lockfile-2.0-10.el9.noarch 2026-02-25T09:17:14.171 INFO:teuthology.orchestra.run.trial094.stdout: unzip-6.0-59.el9.x86_64 2026-02-25T09:17:14.171 INFO:teuthology.orchestra.run.trial094.stdout: zip-3.0-35.el9.x86_64 2026-02-25T09:17:14.171 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:14.171 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:17:14.209 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-base 2026-02-25T09:17:14.527 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:04:57 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:17:14.592 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-base-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:17:14.610 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:17:14.610 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:17:14.610 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:17:14.632 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install cephadm 2026-02-25T09:17:14.946 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:04:57 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:17:15.029 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:17:15.029 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:15.029 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repository Size 2026-02-25T09:17:15.030 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:15.030 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:17:15.030 INFO:teuthology.orchestra.run.trial094.stdout: cephadm noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 1.0 M 2026-02-25T09:17:15.030 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:15.030 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:17:15.030 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:15.031 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T09:17:15.031 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:15.031 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 1.0 M 2026-02-25T09:17:15.031 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 1.0 M 2026-02-25T09:17:15.031 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:17:16.189 INFO:teuthology.orchestra.run.trial094.stdout:cephadm-20.3.0-5536.ge6eec768.el9.noarch.rpm 899 kB/s | 1.0 MB 00:01 2026-02-25T09:17:16.190 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:16.190 INFO:teuthology.orchestra.run.trial094.stdout:Total 898 kB/s | 1.0 MB 00:01 2026-02-25T09:17:16.190 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:17:16.192 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:17:16.193 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:17:16.196 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:17:16.197 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:17:16.217 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:17:16.279 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 1/1 2026-02-25T09:17:16.294 INFO:teuthology.orchestra.run.trial094.stdout: Installing : cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 1/1 2026-02-25T09:17:16.366 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 1/1 2026-02-25T09:17:16.426 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 1/1 2026-02-25T09:17:16.426 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:16.426 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:17:16.427 INFO:teuthology.orchestra.run.trial094.stdout: cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 2026-02-25T09:17:16.427 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:16.427 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:17:16.454 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T09:17:16.771 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:04:59 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:17:16.855 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:17:16.855 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:16.855 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T09:17:16.855 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:16.855 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:17:16.855 INFO:teuthology.orchestra.run.trial094.stdout: ceph-immutable-object-cache x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 149 k 2026-02-25T09:17:16.856 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:16.856 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:17:16.856 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:16.856 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T09:17:16.856 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:16.856 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 149 k 2026-02-25T09:17:16.856 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 455 k 2026-02-25T09:17:16.856 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:17:16.879 INFO:teuthology.orchestra.run.trial151.stdout:(12/12): ceph-mgr-diskprediction-local-20.3.0-5 1.0 MB/s | 7.4 MB 00:07 2026-02-25T09:17:16.882 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:16.882 INFO:teuthology.orchestra.run.trial151.stdout:Total 5.5 MB/s | 43 MB 00:07 2026-02-25T09:17:17.041 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:17:17.054 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:17:17.054 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:17:17.257 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:17:17.257 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:17:17.358 INFO:teuthology.orchestra.run.trial094.stdout:ceph-immutable-object-cache-20.3.0-5536.ge6eec7 298 kB/s | 149 kB 00:00 2026-02-25T09:17:17.358 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:17.359 INFO:teuthology.orchestra.run.trial094.stdout:Total 297 kB/s | 149 kB 00:00 2026-02-25T09:17:17.359 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:17:17.363 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:17:17.363 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:17:17.389 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:17:17.389 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:17:17.439 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:17:17.440 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-immutable-object-cache-2:20.3.0-5536.ge6eec768. 1/1 2026-02-25T09:17:17.461 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-immutable-object-cache-2:20.3.0-5536.ge6eec768. 1/1 2026-02-25T09:17:17.462 INFO:teuthology.orchestra.run.trial094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T09:17:17.462 INFO:teuthology.orchestra.run.trial094.stdout:Invalid unit name "ceph-immutable-object-cache@*.service" escaped as "ceph-immutable-object-cache@\x2a.service". 2026-02-25T09:17:17.462 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:17.474 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:17:17.477 INFO:teuthology.orchestra.run.trial151.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 1/12 2026-02-25T09:17:17.485 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 2/12 2026-02-25T09:17:17.519 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 3/12 2026-02-25T09:17:17.523 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 4/12 2026-02-25T09:17:17.582 INFO:teuthology.orchestra.run.trial151.stdout: Installing : openblas-0.3.29-1.el9.x86_64 5/12 2026-02-25T09:17:17.584 INFO:teuthology.orchestra.run.trial151.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 6/12 2026-02-25T09:17:17.606 INFO:teuthology.orchestra.run.trial151.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 7/12 2026-02-25T09:17:17.775 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-immutable-object-cache-2:20.3.0-5536.ge6eec768. 1/1 2026-02-25T09:17:17.775 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:17.775 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:17:17.775 INFO:teuthology.orchestra.run.trial094.stdout: ceph-immutable-object-cache-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:17.775 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:17.775 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:17:17.802 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mgr 2026-02-25T09:17:17.929 INFO:teuthology.orchestra.run.trial151.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 8/12 2026-02-25T09:17:18.001 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 9/12 2026-02-25T09:17:18.123 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:01 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:17:18.188 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-mgr-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:17:18.206 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:17:18.207 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:17:18.207 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:17:18.229 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T09:17:18.543 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:01 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:17:18.627 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:17:18.628 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:18.628 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repository Size 2026-02-25T09:17:18.628 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:18.628 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:17:18.628 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-dashboard noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 15 M 2026-02-25T09:17:18.628 INFO:teuthology.orchestra.run.trial094.stdout:Installing dependencies: 2026-02-25T09:17:18.628 INFO:teuthology.orchestra.run.trial094.stdout: abseil-cpp x86_64 20211102.0-4.el9 epel 551 k 2026-02-25T09:17:18.628 INFO:teuthology.orchestra.run.trial094.stdout: ceph-grafana-dashboards noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 57 k 2026-02-25T09:17:18.628 INFO:teuthology.orchestra.run.trial094.stdout: ceph-prometheus-alerts noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 17 k 2026-02-25T09:17:18.628 INFO:teuthology.orchestra.run.trial094.stdout: grpc-data noarch 1.46.7-10.el9 epel 19 k 2026-02-25T09:17:18.629 INFO:teuthology.orchestra.run.trial094.stdout: protobuf-compiler x86_64 3.14.0-17.el9 crb 862 k 2026-02-25T09:17:18.629 INFO:teuthology.orchestra.run.trial094.stdout: python3-grpcio x86_64 1.46.7-10.el9 epel 2.0 M 2026-02-25T09:17:18.629 INFO:teuthology.orchestra.run.trial094.stdout: python3-grpcio-tools x86_64 1.46.7-10.el9 epel 144 k 2026-02-25T09:17:18.629 INFO:teuthology.orchestra.run.trial094.stdout: python3-jmespath noarch 1.0.1-1.el9 appstream 48 k 2026-02-25T09:17:18.629 INFO:teuthology.orchestra.run.trial094.stdout: python3-protobuf noarch 3.14.0-17.el9 appstream 267 k 2026-02-25T09:17:18.629 INFO:teuthology.orchestra.run.trial094.stdout: python3-repoze-lru noarch 0.7-16.el9 epel 31 k 2026-02-25T09:17:18.629 INFO:teuthology.orchestra.run.trial094.stdout: python3-routes noarch 2.5.1-5.el9 epel 188 k 2026-02-25T09:17:18.629 INFO:teuthology.orchestra.run.trial094.stdout: python3-xmltodict noarch 0.12.0-15.el9 epel 22 k 2026-02-25T09:17:18.629 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:18.629 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:17:18.629 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:18.629 INFO:teuthology.orchestra.run.trial094.stdout:Install 13 Packages 2026-02-25T09:17:18.630 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:18.630 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 19 M 2026-02-25T09:17:18.630 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 187 M 2026-02-25T09:17:18.630 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:17:18.668 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 10/12 2026-02-25T09:17:19.126 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 11/12 2026-02-25T09:17:19.127 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5536.ge6eec 12/12 2026-02-25T09:17:19.404 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5536.ge6eec 12/12 2026-02-25T09:17:19.404 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5536.ge6eec 1/12 2026-02-25T09:17:19.405 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 2/12 2026-02-25T09:17:19.405 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 3/12 2026-02-25T09:17:19.405 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 4/12 2026-02-25T09:17:19.405 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 5/12 2026-02-25T09:17:19.405 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 6/12 2026-02-25T09:17:19.405 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 7/12 2026-02-25T09:17:19.405 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 8/12 2026-02-25T09:17:19.405 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 9/12 2026-02-25T09:17:19.405 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 10/12 2026-02-25T09:17:19.405 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 11/12 2026-02-25T09:17:19.474 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 12/12 2026-02-25T09:17:19.474 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:19.474 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:17:19.475 INFO:teuthology.orchestra.run.trial151.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5536.ge6eec768.el9.noarch 2026-02-25T09:17:19.475 INFO:teuthology.orchestra.run.trial151.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-25T09:17:19.475 INFO:teuthology.orchestra.run.trial151.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-25T09:17:19.475 INFO:teuthology.orchestra.run.trial151.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-25T09:17:19.475 INFO:teuthology.orchestra.run.trial151.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-25T09:17:19.475 INFO:teuthology.orchestra.run.trial151.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-25T09:17:19.475 INFO:teuthology.orchestra.run.trial151.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-25T09:17:19.475 INFO:teuthology.orchestra.run.trial151.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-25T09:17:19.475 INFO:teuthology.orchestra.run.trial151.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-25T09:17:19.475 INFO:teuthology.orchestra.run.trial151.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-25T09:17:19.475 INFO:teuthology.orchestra.run.trial151.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-25T09:17:19.476 INFO:teuthology.orchestra.run.trial151.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-25T09:17:19.476 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:19.476 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:19.494 INFO:teuthology.orchestra.run.trial094.stdout:(1/13): ceph-prometheus-alerts-20.3.0-5536.ge6e 48 kB/s | 17 kB 00:00 2026-02-25T09:17:19.498 INFO:teuthology.orchestra.run.trial094.stdout:[MIRROR] python3-jmespath-1.0.1-1.el9.noarch.rpm: Curl error (7): Couldn't connect to server for http://mirror.us.mirhosting.net/centos/9-stream/AppStream/x86_64/os/Packages/python3-jmespath-1.0.1-1.el9.noarch.rpm [Failed to connect to mirror.us.mirhosting.net port 80: Connection refused] 2026-02-25T09:17:19.502 INFO:teuthology.orchestra.run.trial094.stdout:[MIRROR] python3-jmespath-1.0.1-1.el9.noarch.rpm: Curl error (7): Couldn't connect to server for https://mirror.us.mirhosting.net/centos/9-stream/AppStream/x86_64/os/Packages/python3-jmespath-1.0.1-1.el9.noarch.rpm [Failed to connect to mirror.us.mirhosting.net port 443: Connection refused] 2026-02-25T09:17:19.512 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-mgr-rook 2026-02-25T09:17:19.565 INFO:teuthology.orchestra.run.trial094.stdout:(2/13): ceph-grafana-dashboards-20.3.0-5536.ge6 133 kB/s | 57 kB 00:00 2026-02-25T09:17:19.568 INFO:teuthology.orchestra.run.trial094.stdout:[MIRROR] python3-protobuf-3.14.0-17.el9.noarch.rpm: Curl error (7): Couldn't connect to server for http://mirror.us.mirhosting.net/centos/9-stream/AppStream/x86_64/os/Packages/python3-protobuf-3.14.0-17.el9.noarch.rpm [Failed to connect to mirror.us.mirhosting.net port 80: Connection refused] 2026-02-25T09:17:19.571 INFO:teuthology.orchestra.run.trial094.stdout:[MIRROR] python3-protobuf-3.14.0-17.el9.noarch.rpm: Curl error (7): Couldn't connect to server for https://mirror.us.mirhosting.net/centos/9-stream/AppStream/x86_64/os/Packages/python3-protobuf-3.14.0-17.el9.noarch.rpm [Failed to connect to mirror.us.mirhosting.net port 443: Connection refused] 2026-02-25T09:17:19.815 INFO:teuthology.orchestra.run.trial094.stdout:(3/13): python3-jmespath-1.0.1-1.el9.noarch.rpm 148 kB/s | 48 kB 00:00 2026-02-25T09:17:19.831 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:07 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:19.916 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:19.917 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:19.917 INFO:teuthology.orchestra.run.trial151.stdout: Package Arch Version Repository Size 2026-02-25T09:17:19.917 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:19.917 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:17:19.917 INFO:teuthology.orchestra.run.trial151.stdout: ceph-mgr-rook noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 50 k 2026-02-25T09:17:19.917 INFO:teuthology.orchestra.run.trial151.stdout:Installing dependencies: 2026-02-25T09:17:19.918 INFO:teuthology.orchestra.run.trial151.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-25T09:17:19.918 INFO:teuthology.orchestra.run.trial151.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-25T09:17:19.918 INFO:teuthology.orchestra.run.trial151.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-25T09:17:19.918 INFO:teuthology.orchestra.run.trial151.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-25T09:17:19.918 INFO:teuthology.orchestra.run.trial151.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-25T09:17:19.918 INFO:teuthology.orchestra.run.trial151.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-25T09:17:19.918 INFO:teuthology.orchestra.run.trial151.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-25T09:17:19.918 INFO:teuthology.orchestra.run.trial151.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-25T09:17:19.918 INFO:teuthology.orchestra.run.trial151.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-25T09:17:19.918 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:19.918 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:17:19.918 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:19.919 INFO:teuthology.orchestra.run.trial151.stdout:Install 10 Packages 2026-02-25T09:17:19.919 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:19.919 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 2.0 M 2026-02-25T09:17:19.919 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 25 M 2026-02-25T09:17:19.919 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:17:20.015 INFO:teuthology.orchestra.run.trial094.stdout:(4/13): protobuf-compiler-3.14.0-17.el9.x86_64. 4.2 MB/s | 862 kB 00:00 2026-02-25T09:17:20.082 INFO:teuthology.orchestra.run.trial094.stdout:(5/13): python3-protobuf-3.14.0-17.el9.noarch.r 517 kB/s | 267 kB 00:00 2026-02-25T09:17:20.114 INFO:teuthology.orchestra.run.trial151.stdout:(1/10): python3-pyasn1-modules-0.4.8-7.el9.noar 5.3 MB/s | 277 kB 00:00 2026-02-25T09:17:20.122 INFO:teuthology.orchestra.run.trial151.stdout:(2/10): python3-pyasn1-0.4.8-7.el9.noarch.rpm 2.6 MB/s | 157 kB 00:00 2026-02-25T09:17:20.137 INFO:teuthology.orchestra.run.trial151.stdout:(3/10): python3-requests-oauthlib-1.3.0-12.el9. 2.3 MB/s | 54 kB 00:00 2026-02-25T09:17:20.186 INFO:teuthology.orchestra.run.trial094.stdout:(6/13): grpc-data-1.46.7-10.el9.noarch.rpm 187 kB/s | 19 kB 00:00 2026-02-25T09:17:20.220 INFO:teuthology.orchestra.run.trial151.stdout:(4/10): python3-cachetools-4.2.4-1.el9.noarch.r 327 kB/s | 32 kB 00:00 2026-02-25T09:17:20.221 INFO:teuthology.orchestra.run.trial151.stdout:(5/10): python3-certifi-2023.05.07-4.el9.noarch 168 kB/s | 14 kB 00:00 2026-02-25T09:17:20.290 INFO:teuthology.orchestra.run.trial094.stdout:(7/13): abseil-cpp-20211102.0-4.el9.x86_64.rpm 2.0 MB/s | 551 kB 00:00 2026-02-25T09:17:20.331 INFO:teuthology.orchestra.run.trial094.stdout:(8/13): python3-grpcio-tools-1.46.7-10.el9.x86_ 3.4 MB/s | 144 kB 00:00 2026-02-25T09:17:20.349 INFO:teuthology.orchestra.run.trial151.stdout:(6/10): python3-google-auth-2.45.0-1.el9.noarch 1.9 MB/s | 254 kB 00:00 2026-02-25T09:17:20.367 INFO:teuthology.orchestra.run.trial094.stdout:(9/13): python3-repoze-lru-0.7-16.el9.noarch.rp 849 kB/s | 31 kB 00:00 2026-02-25T09:17:20.382 INFO:teuthology.orchestra.run.trial151.stdout:(7/10): python3-rsa-4.9-2.el9.noarch.rpm 1.7 MB/s | 59 kB 00:00 2026-02-25T09:17:20.390 INFO:teuthology.orchestra.run.trial151.stdout:(8/10): python3-kubernetes-26.1.0-3.el9.noarch. 6.0 MB/s | 1.0 MB 00:00 2026-02-25T09:17:20.410 INFO:teuthology.orchestra.run.trial094.stdout:(10/13): python3-routes-2.5.1-5.el9.noarch.rpm 4.3 MB/s | 188 kB 00:00 2026-02-25T09:17:20.416 INFO:teuthology.orchestra.run.trial151.stdout:(9/10): python3-websocket-client-1.2.3-2.el9.no 2.6 MB/s | 90 kB 00:00 2026-02-25T09:17:20.438 INFO:teuthology.orchestra.run.trial094.stdout:(11/13): python3-grpcio-1.46.7-10.el9.x86_64.rp 8.1 MB/s | 2.0 MB 00:00 2026-02-25T09:17:20.446 INFO:teuthology.orchestra.run.trial094.stdout:(12/13): python3-xmltodict-0.12.0-15.el9.noarch 615 kB/s | 22 kB 00:00 2026-02-25T09:17:20.485 INFO:teuthology.orchestra.run.trial151.stdout:(10/10): ceph-mgr-rook-20.3.0-5536.ge6eec768.el 119 kB/s | 50 kB 00:00 2026-02-25T09:17:20.487 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:20.487 INFO:teuthology.orchestra.run.trial151.stdout:Total 3.5 MB/s | 2.0 MB 00:00 2026-02-25T09:17:20.516 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:17:20.525 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:17:20.525 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:17:20.566 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:17:20.566 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:17:20.642 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:17:20.693 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 1/10 2026-02-25T09:17:20.706 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 2/10 2026-02-25T09:17:20.721 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/10 2026-02-25T09:17:20.726 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/10 2026-02-25T09:17:20.733 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/10 2026-02-25T09:17:20.773 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/10 2026-02-25T09:17:20.785 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-25T09:17:21.103 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 8/10 2026-02-25T09:17:21.118 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 9/10 2026-02-25T09:17:21.119 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-mgr-rook-2:20.3.0-5536.ge6eec768.el9.noarch 10/10 2026-02-25T09:17:21.233 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5536.ge6eec768.el9.noarch 10/10 2026-02-25T09:17:21.234 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5536.ge6eec768.el9.noarch 1/10 2026-02-25T09:17:21.234 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 2/10 2026-02-25T09:17:21.234 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 3/10 2026-02-25T09:17:21.234 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 4/10 2026-02-25T09:17:21.234 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 5/10 2026-02-25T09:17:21.234 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 6/10 2026-02-25T09:17:21.234 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-25T09:17:21.234 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 8/10 2026-02-25T09:17:21.234 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 9/10 2026-02-25T09:17:21.295 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 10/10 2026-02-25T09:17:21.295 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:21.295 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:17:21.295 INFO:teuthology.orchestra.run.trial151.stdout: ceph-mgr-rook-2:20.3.0-5536.ge6eec768.el9.noarch 2026-02-25T09:17:21.295 INFO:teuthology.orchestra.run.trial151.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-25T09:17:21.295 INFO:teuthology.orchestra.run.trial151.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-25T09:17:21.296 INFO:teuthology.orchestra.run.trial151.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-25T09:17:21.296 INFO:teuthology.orchestra.run.trial151.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-25T09:17:21.296 INFO:teuthology.orchestra.run.trial151.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-25T09:17:21.296 INFO:teuthology.orchestra.run.trial151.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-25T09:17:21.296 INFO:teuthology.orchestra.run.trial151.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-25T09:17:21.296 INFO:teuthology.orchestra.run.trial151.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-25T09:17:21.296 INFO:teuthology.orchestra.run.trial151.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-25T09:17:21.296 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:21.296 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:21.327 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T09:17:21.645 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:09 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:21.730 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:21.731 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:21.731 INFO:teuthology.orchestra.run.trial151.stdout: Package Arch Version Repository Size 2026-02-25T09:17:21.731 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:21.731 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:17:21.731 INFO:teuthology.orchestra.run.trial151.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 184 k 2026-02-25T09:17:21.731 INFO:teuthology.orchestra.run.trial151.stdout:Installing dependencies: 2026-02-25T09:17:21.732 INFO:teuthology.orchestra.run.trial151.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-25T09:17:21.732 INFO:teuthology.orchestra.run.trial151.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-25T09:17:21.732 INFO:teuthology.orchestra.run.trial151.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-25T09:17:21.732 INFO:teuthology.orchestra.run.trial151.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-25T09:17:21.732 INFO:teuthology.orchestra.run.trial151.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-25T09:17:21.732 INFO:teuthology.orchestra.run.trial151.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-25T09:17:21.732 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:21.732 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:17:21.732 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:21.732 INFO:teuthology.orchestra.run.trial151.stdout:Install 7 Packages 2026-02-25T09:17:21.732 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:21.732 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 7.1 M 2026-02-25T09:17:21.733 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 34 M 2026-02-25T09:17:21.733 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:17:22.690 INFO:teuthology.orchestra.run.trial151.stdout:(1/7): ceph-mgr-cephadm-20.3.0-5536.ge6eec768.e 322 kB/s | 184 kB 00:00 2026-02-25T09:17:22.795 INFO:teuthology.orchestra.run.trial151.stdout:(2/7): python3-jinja2-2.11.3-8.el9.noarch.rpm 367 kB/s | 249 kB 00:00 2026-02-25T09:17:22.838 INFO:teuthology.orchestra.run.trial151.stdout:(3/7): python3-asyncssh-2.13.2-5.el9.noarch.rpm 13 MB/s | 548 kB 00:00 2026-02-25T09:17:22.841 INFO:teuthology.orchestra.run.trial151.stdout:(4/7): python3-natsort-7.1.1-5.el9.noarch.rpm 17 MB/s | 58 kB 00:00 2026-02-25T09:17:22.845 INFO:teuthology.orchestra.run.trial151.stdout:(5/7): python3-typing-extensions-4.15.0-1.el9.n 24 MB/s | 86 kB 00:00 2026-02-25T09:17:23.110 INFO:teuthology.orchestra.run.trial151.stdout:(6/7): python3-markupsafe-1.1.1-12.el9.x86_64.r 83 kB/s | 35 kB 00:00 2026-02-25T09:17:23.123 INFO:teuthology.orchestra.run.trial151.stdout:(7/7): python3-babel-2.9.1-2.el9.noarch.rpm 5.9 MB/s | 6.0 MB 00:01 2026-02-25T09:17:23.125 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:23.125 INFO:teuthology.orchestra.run.trial151.stdout:Total 5.1 MB/s | 7.1 MB 00:01 2026-02-25T09:17:23.164 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:17:23.171 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:17:23.171 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:17:23.245 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:17:23.245 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:17:23.337 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:17:23.386 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 1/7 2026-02-25T09:17:23.395 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/7 2026-02-25T09:17:23.401 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/7 2026-02-25T09:17:23.620 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-25T09:17:23.643 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 5/7 2026-02-25T09:17:23.667 INFO:teuthology.orchestra.run.trial151.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 6/7 2026-02-25T09:17:23.669 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 7/7 2026-02-25T09:17:23.782 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 7/7 2026-02-25T09:17:23.782 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 1/7 2026-02-25T09:17:23.782 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/7 2026-02-25T09:17:23.782 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 3/7 2026-02-25T09:17:23.782 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-25T09:17:23.782 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 5/7 2026-02-25T09:17:23.782 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 6/7 2026-02-25T09:17:23.844 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 7/7 2026-02-25T09:17:23.844 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:23.844 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:17:23.845 INFO:teuthology.orchestra.run.trial151.stdout: ceph-mgr-cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 2026-02-25T09:17:23.845 INFO:teuthology.orchestra.run.trial151.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-25T09:17:23.845 INFO:teuthology.orchestra.run.trial151.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-25T09:17:23.845 INFO:teuthology.orchestra.run.trial151.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-25T09:17:23.845 INFO:teuthology.orchestra.run.trial151.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-25T09:17:23.845 INFO:teuthology.orchestra.run.trial151.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-25T09:17:23.845 INFO:teuthology.orchestra.run.trial151.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-25T09:17:23.845 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:23.845 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:23.876 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-osd 2026-02-25T09:17:24.200 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:12 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:24.265 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-osd-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:17:24.284 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:24.284 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:17:24.284 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:24.308 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-osd-classic 2026-02-25T09:17:24.623 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:12 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:24.689 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-osd-classic-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:17:24.708 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:24.708 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:17:24.709 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:24.729 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-fuse 2026-02-25T09:17:25.046 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:13 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:25.130 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:25.130 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:25.130 INFO:teuthology.orchestra.run.trial151.stdout: Package Arch Version Repo Size 2026-02-25T09:17:25.130 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:25.131 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:17:25.131 INFO:teuthology.orchestra.run.trial151.stdout: ceph-fuse x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 914 k 2026-02-25T09:17:25.131 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:25.131 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:17:25.131 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:25.131 INFO:teuthology.orchestra.run.trial151.stdout:Install 1 Package 2026-02-25T09:17:25.131 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:25.131 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 914 k 2026-02-25T09:17:25.131 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 2.9 M 2026-02-25T09:17:25.131 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:17:26.219 INFO:teuthology.orchestra.run.trial151.stdout:ceph-fuse-20.3.0-5536.ge6eec768.el9.x86_64.rpm 840 kB/s | 914 kB 00:01 2026-02-25T09:17:26.219 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:26.219 INFO:teuthology.orchestra.run.trial151.stdout:Total 840 kB/s | 914 kB 00:01 2026-02-25T09:17:26.220 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:17:26.224 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:17:26.224 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:17:26.257 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:17:26.257 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:17:26.347 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:17:26.364 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:26.590 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:26.652 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:26.652 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:26.652 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:17:26.652 INFO:teuthology.orchestra.run.trial151.stdout: ceph-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:26.652 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:26.652 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:26.679 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-volume 2026-02-25T09:17:27.002 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:15 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:27.087 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:27.087 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:27.088 INFO:teuthology.orchestra.run.trial151.stdout: Package Arch Version Repository Size 2026-02-25T09:17:27.088 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:27.088 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:17:27.088 INFO:teuthology.orchestra.run.trial151.stdout: ceph-volume noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 301 k 2026-02-25T09:17:27.088 INFO:teuthology.orchestra.run.trial151.stdout:Installing dependencies: 2026-02-25T09:17:27.088 INFO:teuthology.orchestra.run.trial151.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-25T09:17:27.088 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:27.088 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:17:27.088 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:27.088 INFO:teuthology.orchestra.run.trial151.stdout:Install 2 Packages 2026-02-25T09:17:27.088 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:27.089 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 652 k 2026-02-25T09:17:27.089 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 2.2 M 2026-02-25T09:17:27.089 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:17:27.154 INFO:teuthology.orchestra.run.trial151.stdout:[MIRROR] cryptsetup-2.8.1-3.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for http://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/cryptsetup-2.8.1-3.el9.x86_64.rpm [Failed to connect to mirror.us.mirhosting.net port 80: Connection refused] 2026-02-25T09:17:27.158 INFO:teuthology.orchestra.run.trial151.stdout:[MIRROR] cryptsetup-2.8.1-3.el9.x86_64.rpm: Curl error (7): Couldn't connect to server for https://mirror.us.mirhosting.net/centos/9-stream/BaseOS/x86_64/os/Packages/cryptsetup-2.8.1-3.el9.x86_64.rpm [Failed to connect to mirror.us.mirhosting.net port 443: Connection refused] 2026-02-25T09:17:27.293 INFO:teuthology.orchestra.run.trial151.stdout:(1/2): cryptsetup-2.8.1-3.el9.x86_64.rpm 2.4 MB/s | 351 kB 00:00 2026-02-25T09:17:27.793 INFO:teuthology.orchestra.run.trial151.stdout:(2/2): ceph-volume-20.3.0-5536.ge6eec768.el9.no 469 kB/s | 301 kB 00:00 2026-02-25T09:17:27.794 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:27.794 INFO:teuthology.orchestra.run.trial151.stdout:Total 924 kB/s | 652 kB 00:00 2026-02-25T09:17:27.805 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:17:27.811 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:17:27.811 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:17:27.857 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:17:27.857 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:17:27.937 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:17:28.001 INFO:teuthology.orchestra.run.trial151.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 1/2 2026-02-25T09:17:28.003 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-volume-2:20.3.0-5536.ge6eec768.el9.noarch 2/2 2026-02-25T09:17:28.018 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-volume-2:20.3.0-5536.ge6eec768.el9.noarch 2/2 2026-02-25T09:17:28.019 INFO:teuthology.orchestra.run.trial151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T09:17:28.019 INFO:teuthology.orchestra.run.trial151.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-25T09:17:28.019 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:28.251 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-volume-2:20.3.0-5536.ge6eec768.el9.noarch 1/2 2026-02-25T09:17:28.314 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 2/2 2026-02-25T09:17:28.314 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:28.314 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:17:28.314 INFO:teuthology.orchestra.run.trial151.stdout: ceph-volume-2:20.3.0-5536.ge6eec768.el9.noarch cryptsetup-2.8.1-3.el9.x86_64 2026-02-25T09:17:28.314 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:28.315 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:28.342 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install librados-devel 2026-02-25T09:17:28.659 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:16 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:28.743 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:28.744 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:28.744 INFO:teuthology.orchestra.run.trial151.stdout: Package Arch Version Repo Size 2026-02-25T09:17:28.744 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:28.744 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:17:28.745 INFO:teuthology.orchestra.run.trial151.stdout: librados-devel x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 124 k 2026-02-25T09:17:28.745 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:28.745 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:17:28.745 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:28.745 INFO:teuthology.orchestra.run.trial151.stdout:Install 1 Package 2026-02-25T09:17:28.745 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:28.745 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 124 k 2026-02-25T09:17:28.745 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 434 k 2026-02-25T09:17:28.745 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:17:29.229 INFO:teuthology.orchestra.run.trial151.stdout:librados-devel-20.3.0-5536.ge6eec768.el9.x86_64 255 kB/s | 124 kB 00:00 2026-02-25T09:17:29.230 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:29.230 INFO:teuthology.orchestra.run.trial151.stdout:Total 254 kB/s | 124 kB 00:00 2026-02-25T09:17:29.230 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:17:29.233 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:17:29.233 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:17:29.265 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:17:29.265 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:17:29.319 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:17:29.336 INFO:teuthology.orchestra.run.trial151.stdout: Installing : librados-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:29.431 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: librados-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:29.494 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : librados-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:29.494 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:29.494 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:17:29.494 INFO:teuthology.orchestra.run.trial151.stdout: librados-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:29.495 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:29.495 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:29.521 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install libcephfs2 2026-02-25T09:17:29.837 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:17 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:29.902 INFO:teuthology.orchestra.run.trial151.stdout:Package libcephfs2-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:17:29.921 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:29.922 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:17:29.922 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:29.943 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install libcephfs-devel 2026-02-25T09:17:30.265 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:18 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:30.350 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:30.351 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:30.351 INFO:teuthology.orchestra.run.trial151.stdout: Package Arch Version Repo Size 2026-02-25T09:17:30.351 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:30.351 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:17:30.351 INFO:teuthology.orchestra.run.trial151.stdout: libcephfs-devel x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 37 k 2026-02-25T09:17:30.351 INFO:teuthology.orchestra.run.trial151.stdout:Installing dependencies: 2026-02-25T09:17:30.351 INFO:teuthology.orchestra.run.trial151.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 26 k 2026-02-25T09:17:30.351 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:30.351 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:17:30.351 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:30.352 INFO:teuthology.orchestra.run.trial151.stdout:Install 2 Packages 2026-02-25T09:17:30.352 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:30.352 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 64 k 2026-02-25T09:17:30.352 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 231 k 2026-02-25T09:17:30.352 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:17:30.696 INFO:teuthology.orchestra.run.trial151.stdout:(1/2): libcephfs-devel-20.3.0-5536.ge6eec768.el 108 kB/s | 37 kB 00:00 2026-02-25T09:17:30.699 INFO:teuthology.orchestra.run.trial151.stdout:(2/2): libcephfs-proxy2-20.3.0-5536.ge6eec768.e 75 kB/s | 26 kB 00:00 2026-02-25T09:17:30.703 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:30.703 INFO:teuthology.orchestra.run.trial151.stdout:Total 180 kB/s | 64 kB 00:00 2026-02-25T09:17:30.703 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:17:30.706 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:17:30.707 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:17:30.739 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:17:30.739 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:17:30.788 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:17:30.789 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libcephfs-proxy2-2:20.3.0-5536.ge6eec768.el9.x86_64 1/2 2026-02-25T09:17:30.803 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5536.ge6eec768.el9.x86_64 1/2 2026-02-25T09:17:30.820 INFO:teuthology.orchestra.run.trial151.stdout: Installing : libcephfs-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 2/2 2026-02-25T09:17:30.883 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 2/2 2026-02-25T09:17:30.883 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libcephfs-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 1/2 2026-02-25T09:17:30.945 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5536.ge6eec768.el9.x86_64 2/2 2026-02-25T09:17:30.945 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:30.946 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:17:30.946 INFO:teuthology.orchestra.run.trial151.stdout: libcephfs-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:30.946 INFO:teuthology.orchestra.run.trial151.stdout: libcephfs-proxy2-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:30.946 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:30.946 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:30.973 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install librados2 2026-02-25T09:17:31.295 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:19 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:31.360 INFO:teuthology.orchestra.run.trial151.stdout:Package librados2-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:17:31.379 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:31.380 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:17:31.380 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:31.403 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install librbd1 2026-02-25T09:17:31.718 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:19 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:31.785 INFO:teuthology.orchestra.run.trial151.stdout:Package librbd1-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:17:31.803 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:31.804 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:17:31.804 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:31.825 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install python3-rados 2026-02-25T09:17:32.147 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:20 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:32.214 INFO:teuthology.orchestra.run.trial151.stdout:Package python3-rados-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:17:32.232 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:32.233 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:17:32.233 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:32.255 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install python3-rgw 2026-02-25T09:17:32.572 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:20 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:32.637 INFO:teuthology.orchestra.run.trial151.stdout:Package python3-rgw-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:17:32.656 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:32.657 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:17:32.657 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:32.678 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install python3-cephfs 2026-02-25T09:17:32.929 INFO:teuthology.orchestra.run.trial094.stdout:(13/13): ceph-mgr-dashboard-20.3.0-5536.ge6eec7 1.1 MB/s | 15 MB 00:13 2026-02-25T09:17:32.930 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:32.931 INFO:teuthology.orchestra.run.trial094.stdout:Total 1.3 MB/s | 19 MB 00:14 2026-02-25T09:17:32.959 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:17:32.968 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:17:32.969 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:17:32.996 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:20 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:33.061 INFO:teuthology.orchestra.run.trial151.stdout:Package python3-cephfs-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:17:33.076 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:17:33.077 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:17:33.080 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:33.080 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:17:33.080 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:33.102 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install python3-rbd 2026-02-25T09:17:33.201 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:17:33.206 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-xmltodict-0.12.0-15.el9.noarch 1/13 2026-02-25T09:17:33.217 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-repoze-lru-0.7-16.el9.noarch 2/13 2026-02-25T09:17:33.220 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-routes-2.5.1-5.el9.noarch 3/13 2026-02-25T09:17:33.247 INFO:teuthology.orchestra.run.trial094.stdout: Installing : grpc-data-1.46.7-10.el9.noarch 4/13 2026-02-25T09:17:33.290 INFO:teuthology.orchestra.run.trial094.stdout: Installing : abseil-cpp-20211102.0-4.el9.x86_64 5/13 2026-02-25T09:17:33.300 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-grpcio-1.46.7-10.el9.x86_64 6/13 2026-02-25T09:17:33.328 INFO:teuthology.orchestra.run.trial094.stdout: Installing : protobuf-compiler-3.14.0-17.el9.x86_64 7/13 2026-02-25T09:17:33.337 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-protobuf-3.14.0-17.el9.noarch 8/13 2026-02-25T09:17:33.344 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-grpcio-tools-1.46.7-10.el9.x86_64 9/13 2026-02-25T09:17:33.347 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-jmespath-1.0.1-1.el9.noarch 10/13 2026-02-25T09:17:33.356 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-prometheus-alerts-2:20.3.0-5536.ge6eec768.el9 11/13 2026-02-25T09:17:33.421 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:21 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:33.487 INFO:teuthology.orchestra.run.trial151.stdout:Package python3-rbd-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:17:33.506 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:33.507 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:17:33.507 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:33.529 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install rbd-fuse 2026-02-25T09:17:33.842 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:21 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:33.927 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:33.928 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:33.928 INFO:teuthology.orchestra.run.trial151.stdout: Package Architecture Version Repository Size 2026-02-25T09:17:33.928 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:33.928 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:17:33.928 INFO:teuthology.orchestra.run.trial151.stdout: rbd-fuse x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 87 k 2026-02-25T09:17:33.928 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:33.928 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:17:33.928 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:33.928 INFO:teuthology.orchestra.run.trial151.stdout:Install 1 Package 2026-02-25T09:17:33.928 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:33.929 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 87 k 2026-02-25T09:17:33.929 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 236 k 2026-02-25T09:17:33.929 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:17:34.348 INFO:teuthology.orchestra.run.trial151.stdout:rbd-fuse-20.3.0-5536.ge6eec768.el9.x86_64.rpm 206 kB/s | 87 kB 00:00 2026-02-25T09:17:34.348 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:34.349 INFO:teuthology.orchestra.run.trial151.stdout:Total 206 kB/s | 87 kB 00:00 2026-02-25T09:17:34.349 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:17:34.353 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:17:34.353 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:17:34.384 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:17:34.384 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:17:34.435 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:17:34.452 INFO:teuthology.orchestra.run.trial151.stdout: Installing : rbd-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:34.541 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:34.604 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : rbd-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:34.605 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:34.605 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:17:34.605 INFO:teuthology.orchestra.run.trial151.stdout: rbd-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:34.605 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:34.605 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:34.634 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install rbd-mirror 2026-02-25T09:17:34.929 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-grafana-dashboards-2:20.3.0-5536.ge6eec768.el 12/13 2026-02-25T09:17:34.935 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mgr-dashboard-2:20.3.0-5536.ge6eec768.el9.noa 13/13 2026-02-25T09:17:34.959 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:22 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:35.043 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:35.044 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:35.044 INFO:teuthology.orchestra.run.trial151.stdout: Package Arch Version Repo Size 2026-02-25T09:17:35.044 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:35.044 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:17:35.045 INFO:teuthology.orchestra.run.trial151.stdout: rbd-mirror x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 2.6 M 2026-02-25T09:17:35.045 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:35.045 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:17:35.045 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:35.045 INFO:teuthology.orchestra.run.trial151.stdout:Install 1 Package 2026-02-25T09:17:35.045 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:35.045 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 2.6 M 2026-02-25T09:17:35.045 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 11 M 2026-02-25T09:17:35.045 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:17:35.164 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mgr-dashboard-2:20.3.0-5536.ge6eec768.el9.noa 13/13 2026-02-25T09:17:35.164 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-grafana-dashboards-2:20.3.0-5536.ge6eec768.el 1/13 2026-02-25T09:17:35.165 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mgr-dashboard-2:20.3.0-5536.ge6eec768.el9.noa 2/13 2026-02-25T09:17:35.165 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-prometheus-alerts-2:20.3.0-5536.ge6eec768.el9 3/13 2026-02-25T09:17:35.165 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-jmespath-1.0.1-1.el9.noarch 4/13 2026-02-25T09:17:35.165 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-protobuf-3.14.0-17.el9.noarch 5/13 2026-02-25T09:17:35.165 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : protobuf-compiler-3.14.0-17.el9.x86_64 6/13 2026-02-25T09:17:35.165 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : abseil-cpp-20211102.0-4.el9.x86_64 7/13 2026-02-25T09:17:35.165 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : grpc-data-1.46.7-10.el9.noarch 8/13 2026-02-25T09:17:35.165 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-grpcio-1.46.7-10.el9.x86_64 9/13 2026-02-25T09:17:35.165 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-grpcio-tools-1.46.7-10.el9.x86_64 10/13 2026-02-25T09:17:35.165 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-repoze-lru-0.7-16.el9.noarch 11/13 2026-02-25T09:17:35.165 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-routes-2.5.1-5.el9.noarch 12/13 2026-02-25T09:17:35.224 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-xmltodict-0.12.0-15.el9.noarch 13/13 2026-02-25T09:17:35.224 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:35.224 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:17:35.224 INFO:teuthology.orchestra.run.trial094.stdout: abseil-cpp-20211102.0-4.el9.x86_64 2026-02-25T09:17:35.224 INFO:teuthology.orchestra.run.trial094.stdout: ceph-grafana-dashboards-2:20.3.0-5536.ge6eec768.el9.noarch 2026-02-25T09:17:35.225 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-dashboard-2:20.3.0-5536.ge6eec768.el9.noarch 2026-02-25T09:17:35.225 INFO:teuthology.orchestra.run.trial094.stdout: ceph-prometheus-alerts-2:20.3.0-5536.ge6eec768.el9.noarch 2026-02-25T09:17:35.225 INFO:teuthology.orchestra.run.trial094.stdout: grpc-data-1.46.7-10.el9.noarch 2026-02-25T09:17:35.225 INFO:teuthology.orchestra.run.trial094.stdout: protobuf-compiler-3.14.0-17.el9.x86_64 2026-02-25T09:17:35.225 INFO:teuthology.orchestra.run.trial094.stdout: python3-grpcio-1.46.7-10.el9.x86_64 2026-02-25T09:17:35.225 INFO:teuthology.orchestra.run.trial094.stdout: python3-grpcio-tools-1.46.7-10.el9.x86_64 2026-02-25T09:17:35.225 INFO:teuthology.orchestra.run.trial094.stdout: python3-jmespath-1.0.1-1.el9.noarch 2026-02-25T09:17:35.225 INFO:teuthology.orchestra.run.trial094.stdout: python3-protobuf-3.14.0-17.el9.noarch 2026-02-25T09:17:35.225 INFO:teuthology.orchestra.run.trial094.stdout: python3-repoze-lru-0.7-16.el9.noarch 2026-02-25T09:17:35.225 INFO:teuthology.orchestra.run.trial094.stdout: python3-routes-2.5.1-5.el9.noarch 2026-02-25T09:17:35.226 INFO:teuthology.orchestra.run.trial094.stdout: python3-xmltodict-0.12.0-15.el9.noarch 2026-02-25T09:17:35.226 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:35.226 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:17:35.257 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T09:17:35.582 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:18 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:17:35.668 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:17:35.668 INFO:teuthology.orchestra.run.trial094.stdout:======================================================================================= 2026-02-25T09:17:35.668 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repository Size 2026-02-25T09:17:35.668 INFO:teuthology.orchestra.run.trial094.stdout:======================================================================================= 2026-02-25T09:17:35.669 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:17:35.669 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-diskprediction-local noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 7.4 M 2026-02-25T09:17:35.669 INFO:teuthology.orchestra.run.trial094.stdout:Installing dependencies: 2026-02-25T09:17:35.669 INFO:teuthology.orchestra.run.trial094.stdout: flexiblas x86_64 3.0.4-9.el9 appstream 30 k 2026-02-25T09:17:35.669 INFO:teuthology.orchestra.run.trial094.stdout: flexiblas-netlib x86_64 3.0.4-9.el9 appstream 3.0 M 2026-02-25T09:17:35.669 INFO:teuthology.orchestra.run.trial094.stdout: flexiblas-openblas-openmp x86_64 3.0.4-9.el9 appstream 15 k 2026-02-25T09:17:35.669 INFO:teuthology.orchestra.run.trial094.stdout: libgfortran x86_64 11.5.0-14.el9 baseos 794 k 2026-02-25T09:17:35.669 INFO:teuthology.orchestra.run.trial094.stdout: libquadmath x86_64 11.5.0-14.el9 baseos 184 k 2026-02-25T09:17:35.669 INFO:teuthology.orchestra.run.trial094.stdout: openblas x86_64 0.3.29-1.el9 appstream 42 k 2026-02-25T09:17:35.669 INFO:teuthology.orchestra.run.trial094.stdout: openblas-openmp x86_64 0.3.29-1.el9 appstream 5.3 M 2026-02-25T09:17:35.669 INFO:teuthology.orchestra.run.trial094.stdout: python3-devel x86_64 3.9.25-3.el9 appstream 244 k 2026-02-25T09:17:35.669 INFO:teuthology.orchestra.run.trial094.stdout: python3-numpy x86_64 1:1.23.5-2.el9 appstream 6.1 M 2026-02-25T09:17:35.670 INFO:teuthology.orchestra.run.trial094.stdout: python3-numpy-f2py x86_64 1:1.23.5-2.el9 appstream 442 k 2026-02-25T09:17:35.670 INFO:teuthology.orchestra.run.trial094.stdout: python3-scipy x86_64 1.9.3-2.el9 appstream 19 M 2026-02-25T09:17:35.670 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:35.670 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:17:35.670 INFO:teuthology.orchestra.run.trial094.stdout:======================================================================================= 2026-02-25T09:17:35.670 INFO:teuthology.orchestra.run.trial094.stdout:Install 12 Packages 2026-02-25T09:17:35.670 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:35.671 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 43 M 2026-02-25T09:17:35.671 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 234 M 2026-02-25T09:17:35.671 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:17:36.078 INFO:teuthology.orchestra.run.trial094.stdout:(1/12): libquadmath-11.5.0-14.el9.x86_64.rpm 790 kB/s | 184 kB 00:00 2026-02-25T09:17:36.165 INFO:teuthology.orchestra.run.trial094.stdout:(2/12): libgfortran-11.5.0-14.el9.x86_64.rpm 2.4 MB/s | 794 kB 00:00 2026-02-25T09:17:36.612 INFO:teuthology.orchestra.run.trial094.stdout:(3/12): flexiblas-3.0.4-9.el9.x86_64.rpm 55 kB/s | 30 kB 00:00 2026-02-25T09:17:37.075 INFO:teuthology.orchestra.run.trial094.stdout:(4/12): flexiblas-openblas-openmp-3.0.4-9.el9.x 32 kB/s | 15 kB 00:00 2026-02-25T09:17:37.163 INFO:teuthology.orchestra.run.trial094.stdout:(5/12): flexiblas-netlib-3.0.4-9.el9.x86_64.rpm 3.0 MB/s | 3.0 MB 00:00 2026-02-25T09:17:37.652 INFO:teuthology.orchestra.run.trial094.stdout:(6/12): openblas-0.3.29-1.el9.x86_64.rpm 73 kB/s | 42 kB 00:00 2026-02-25T09:17:38.037 INFO:teuthology.orchestra.run.trial151.stdout:rbd-mirror-20.3.0-5536.ge6eec768.el9.x86_64.rpm 887 kB/s | 2.6 MB 00:02 2026-02-25T09:17:38.037 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:38.037 INFO:teuthology.orchestra.run.trial151.stdout:Total 887 kB/s | 2.6 MB 00:02 2026-02-25T09:17:38.038 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:17:38.043 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:17:38.043 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:17:38.080 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:17:38.080 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:17:38.215 INFO:teuthology.orchestra.run.trial094.stdout:(7/12): openblas-openmp-0.3.29-1.el9.x86_64.rpm 5.0 MB/s | 5.3 MB 00:01 2026-02-25T09:17:38.243 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:17:38.244 INFO:teuthology.orchestra.run.trial151.stdout: Installing : rbd-mirror-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:38.267 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:38.268 INFO:teuthology.orchestra.run.trial151.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T09:17:38.268 INFO:teuthology.orchestra.run.trial151.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-25T09:17:38.268 INFO:teuthology.orchestra.run.trial151.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T09:17:38.268 INFO:teuthology.orchestra.run.trial151.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T09:17:38.268 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:38.385 INFO:teuthology.orchestra.run.trial094.stdout:(8/12): python3-devel-3.9.25-3.el9.x86_64.rpm 333 kB/s | 244 kB 00:00 2026-02-25T09:17:38.607 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : rbd-mirror-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:38.607 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:38.607 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:17:38.607 INFO:teuthology.orchestra.run.trial151.stdout: rbd-mirror-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:38.607 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:38.607 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:38.635 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install rbd-nbd 2026-02-25T09:17:38.957 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:26 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:39.041 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:39.041 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:39.041 INFO:teuthology.orchestra.run.trial151.stdout: Package Architecture Version Repository Size 2026-02-25T09:17:39.041 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:39.042 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:17:39.042 INFO:teuthology.orchestra.run.trial151.stdout: rbd-nbd x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 166 k 2026-02-25T09:17:39.042 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:39.042 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:17:39.042 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:39.042 INFO:teuthology.orchestra.run.trial151.stdout:Install 1 Package 2026-02-25T09:17:39.042 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:39.042 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 166 k 2026-02-25T09:17:39.042 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 496 k 2026-02-25T09:17:39.042 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:17:39.205 INFO:teuthology.orchestra.run.trial094.stdout:(9/12): python3-numpy-f2py-1.23.5-2.el9.x86_64. 539 kB/s | 442 kB 00:00 2026-02-25T09:17:39.337 INFO:teuthology.orchestra.run.trial094.stdout:(10/12): python3-numpy-1.23.5-2.el9.x86_64.rpm 5.5 MB/s | 6.1 MB 00:01 2026-02-25T09:17:39.602 INFO:teuthology.orchestra.run.trial151.stdout:rbd-nbd-20.3.0-5536.ge6eec768.el9.x86_64.rpm 297 kB/s | 166 kB 00:00 2026-02-25T09:17:39.604 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:39.604 INFO:teuthology.orchestra.run.trial151.stdout:Total 296 kB/s | 166 kB 00:00 2026-02-25T09:17:39.604 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:17:39.609 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:17:39.609 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:17:39.640 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:17:39.640 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:17:39.697 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:17:39.714 INFO:teuthology.orchestra.run.trial151.stdout: Installing : rbd-nbd-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:39.800 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:39.862 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : rbd-nbd-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:39.862 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:39.862 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:17:39.863 INFO:teuthology.orchestra.run.trial151.stdout: rbd-nbd-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:39.863 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:39.863 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:39.891 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-osd-crimson 2026-02-25T09:17:40.213 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:28 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:40.278 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-osd-crimson-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:17:40.297 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:40.297 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:17:40.297 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:17:40.321 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-25T09:17:40.613 INFO:teuthology.orchestra.run.trial094.stdout:(11/12): python3-scipy-1.9.3-2.el9.x86_64.rpm 14 MB/s | 19 MB 00:01 2026-02-25T09:17:40.640 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:05:28 ago on Wed 25 Feb 2026 09:12:12 AM UTC. 2026-02-25T09:17:40.725 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:17:40.725 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:40.725 INFO:teuthology.orchestra.run.trial151.stdout: Package Arch Version Repo Size 2026-02-25T09:17:40.725 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:40.725 INFO:teuthology.orchestra.run.trial151.stdout:Installing: 2026-02-25T09:17:40.725 INFO:teuthology.orchestra.run.trial151.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 429 M 2026-02-25T09:17:40.725 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:40.726 INFO:teuthology.orchestra.run.trial151.stdout:Transaction Summary 2026-02-25T09:17:40.726 INFO:teuthology.orchestra.run.trial151.stdout:================================================================================ 2026-02-25T09:17:40.726 INFO:teuthology.orchestra.run.trial151.stdout:Install 1 Package 2026-02-25T09:17:40.726 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:17:40.726 INFO:teuthology.orchestra.run.trial151.stdout:Total download size: 429 M 2026-02-25T09:17:40.726 INFO:teuthology.orchestra.run.trial151.stdout:Installed size: 2.6 G 2026-02-25T09:17:40.726 INFO:teuthology.orchestra.run.trial151.stdout:Downloading Packages: 2026-02-25T09:17:44.962 INFO:teuthology.orchestra.run.trial094.stdout:(12/12): ceph-mgr-diskprediction-local-20.3.0-5 832 kB/s | 7.4 MB 00:09 2026-02-25T09:17:44.963 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:44.963 INFO:teuthology.orchestra.run.trial094.stdout:Total 4.6 MB/s | 43 MB 00:09 2026-02-25T09:17:45.123 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:17:45.137 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:17:45.137 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:17:45.340 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:17:45.340 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:17:45.561 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:17:45.564 INFO:teuthology.orchestra.run.trial094.stdout: Installing : flexiblas-3.0.4-9.el9.x86_64 1/12 2026-02-25T09:17:45.570 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libquadmath-11.5.0-14.el9.x86_64 2/12 2026-02-25T09:17:45.604 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libgfortran-11.5.0-14.el9.x86_64 3/12 2026-02-25T09:17:45.609 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-devel-3.9.25-3.el9.x86_64 4/12 2026-02-25T09:17:45.667 INFO:teuthology.orchestra.run.trial094.stdout: Installing : openblas-0.3.29-1.el9.x86_64 5/12 2026-02-25T09:17:45.670 INFO:teuthology.orchestra.run.trial094.stdout: Installing : openblas-openmp-0.3.29-1.el9.x86_64 6/12 2026-02-25T09:17:45.691 INFO:teuthology.orchestra.run.trial094.stdout: Installing : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 7/12 2026-02-25T09:17:46.011 INFO:teuthology.orchestra.run.trial094.stdout: Installing : flexiblas-netlib-3.0.4-9.el9.x86_64 8/12 2026-02-25T09:17:46.083 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-numpy-1:1.23.5-2.el9.x86_64 9/12 2026-02-25T09:17:46.743 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 10/12 2026-02-25T09:17:47.212 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-scipy-1.9.3-2.el9.x86_64 11/12 2026-02-25T09:17:47.213 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mgr-diskprediction-local-2:20.3.0-5536.ge6eec 12/12 2026-02-25T09:17:47.500 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mgr-diskprediction-local-2:20.3.0-5536.ge6eec 12/12 2026-02-25T09:17:47.500 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mgr-diskprediction-local-2:20.3.0-5536.ge6eec 1/12 2026-02-25T09:17:47.501 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libgfortran-11.5.0-14.el9.x86_64 2/12 2026-02-25T09:17:47.501 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libquadmath-11.5.0-14.el9.x86_64 3/12 2026-02-25T09:17:47.501 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : flexiblas-3.0.4-9.el9.x86_64 4/12 2026-02-25T09:17:47.501 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : flexiblas-netlib-3.0.4-9.el9.x86_64 5/12 2026-02-25T09:17:47.501 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 6/12 2026-02-25T09:17:47.501 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : openblas-0.3.29-1.el9.x86_64 7/12 2026-02-25T09:17:47.501 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : openblas-openmp-0.3.29-1.el9.x86_64 8/12 2026-02-25T09:17:47.501 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-devel-3.9.25-3.el9.x86_64 9/12 2026-02-25T09:17:47.501 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-numpy-1:1.23.5-2.el9.x86_64 10/12 2026-02-25T09:17:47.501 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-numpy-f2py-1:1.23.5-2.el9.x86_64 11/12 2026-02-25T09:17:47.563 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-scipy-1.9.3-2.el9.x86_64 12/12 2026-02-25T09:17:47.563 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:47.563 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:17:47.563 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-diskprediction-local-2:20.3.0-5536.ge6eec768.el9.noarch 2026-02-25T09:17:47.564 INFO:teuthology.orchestra.run.trial094.stdout: flexiblas-3.0.4-9.el9.x86_64 2026-02-25T09:17:47.564 INFO:teuthology.orchestra.run.trial094.stdout: flexiblas-netlib-3.0.4-9.el9.x86_64 2026-02-25T09:17:47.564 INFO:teuthology.orchestra.run.trial094.stdout: flexiblas-openblas-openmp-3.0.4-9.el9.x86_64 2026-02-25T09:17:47.564 INFO:teuthology.orchestra.run.trial094.stdout: libgfortran-11.5.0-14.el9.x86_64 2026-02-25T09:17:47.564 INFO:teuthology.orchestra.run.trial094.stdout: libquadmath-11.5.0-14.el9.x86_64 2026-02-25T09:17:47.564 INFO:teuthology.orchestra.run.trial094.stdout: openblas-0.3.29-1.el9.x86_64 2026-02-25T09:17:47.564 INFO:teuthology.orchestra.run.trial094.stdout: openblas-openmp-0.3.29-1.el9.x86_64 2026-02-25T09:17:47.564 INFO:teuthology.orchestra.run.trial094.stdout: python3-devel-3.9.25-3.el9.x86_64 2026-02-25T09:17:47.564 INFO:teuthology.orchestra.run.trial094.stdout: python3-numpy-1:1.23.5-2.el9.x86_64 2026-02-25T09:17:47.564 INFO:teuthology.orchestra.run.trial094.stdout: python3-numpy-f2py-1:1.23.5-2.el9.x86_64 2026-02-25T09:17:47.565 INFO:teuthology.orchestra.run.trial094.stdout: python3-scipy-1.9.3-2.el9.x86_64 2026-02-25T09:17:47.565 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:47.565 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:17:47.603 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mgr-rook 2026-02-25T09:17:47.925 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:30 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:17:48.010 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:17:48.011 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:48.011 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repository Size 2026-02-25T09:17:48.011 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:48.011 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:17:48.011 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-rook noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 50 k 2026-02-25T09:17:48.012 INFO:teuthology.orchestra.run.trial094.stdout:Installing dependencies: 2026-02-25T09:17:48.012 INFO:teuthology.orchestra.run.trial094.stdout: python3-cachetools noarch 4.2.4-1.el9 epel 32 k 2026-02-25T09:17:48.012 INFO:teuthology.orchestra.run.trial094.stdout: python3-certifi noarch 2023.05.07-4.el9 epel 14 k 2026-02-25T09:17:48.012 INFO:teuthology.orchestra.run.trial094.stdout: python3-google-auth noarch 1:2.45.0-1.el9 epel 254 k 2026-02-25T09:17:48.012 INFO:teuthology.orchestra.run.trial094.stdout: python3-kubernetes noarch 1:26.1.0-3.el9 epel 1.0 M 2026-02-25T09:17:48.012 INFO:teuthology.orchestra.run.trial094.stdout: python3-pyasn1 noarch 0.4.8-7.el9 appstream 157 k 2026-02-25T09:17:48.012 INFO:teuthology.orchestra.run.trial094.stdout: python3-pyasn1-modules noarch 0.4.8-7.el9 appstream 277 k 2026-02-25T09:17:48.012 INFO:teuthology.orchestra.run.trial094.stdout: python3-requests-oauthlib noarch 1.3.0-12.el9 appstream 54 k 2026-02-25T09:17:48.012 INFO:teuthology.orchestra.run.trial094.stdout: python3-rsa noarch 4.9-2.el9 epel 59 k 2026-02-25T09:17:48.012 INFO:teuthology.orchestra.run.trial094.stdout: python3-websocket-client noarch 1.2.3-2.el9 epel 90 k 2026-02-25T09:17:48.013 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:48.013 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:17:48.013 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:48.013 INFO:teuthology.orchestra.run.trial094.stdout:Install 10 Packages 2026-02-25T09:17:48.013 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:48.013 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 2.0 M 2026-02-25T09:17:48.013 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 25 M 2026-02-25T09:17:48.013 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:17:48.850 INFO:teuthology.orchestra.run.trial094.stdout:(1/10): ceph-mgr-rook-20.3.0-5536.ge6eec768.el9 120 kB/s | 50 kB 00:00 2026-02-25T09:17:49.150 INFO:teuthology.orchestra.run.trial094.stdout:(2/10): python3-pyasn1-0.4.8-7.el9.noarch.rpm 219 kB/s | 157 kB 00:00 2026-02-25T09:17:49.211 INFO:teuthology.orchestra.run.trial094.stdout:(3/10): python3-pyasn1-modules-0.4.8-7.el9.noar 356 kB/s | 277 kB 00:00 2026-02-25T09:17:49.268 INFO:teuthology.orchestra.run.trial094.stdout:(4/10): python3-cachetools-4.2.4-1.el9.noarch.r 275 kB/s | 32 kB 00:00 2026-02-25T09:17:49.308 INFO:teuthology.orchestra.run.trial094.stdout:(5/10): python3-certifi-2023.05.07-4.el9.noarch 145 kB/s | 14 kB 00:00 2026-02-25T09:17:49.404 INFO:teuthology.orchestra.run.trial094.stdout:(6/10): python3-google-auth-2.45.0-1.el9.noarch 1.8 MB/s | 254 kB 00:00 2026-02-25T09:17:49.424 INFO:teuthology.orchestra.run.trial094.stdout:(7/10): python3-requests-oauthlib-1.3.0-12.el9. 93 kB/s | 54 kB 00:00 2026-02-25T09:17:49.441 INFO:teuthology.orchestra.run.trial094.stdout:(8/10): python3-rsa-4.9-2.el9.noarch.rpm 1.5 MB/s | 59 kB 00:00 2026-02-25T09:17:49.520 INFO:teuthology.orchestra.run.trial094.stdout:(9/10): python3-kubernetes-26.1.0-3.el9.noarch. 4.8 MB/s | 1.0 MB 00:00 2026-02-25T09:17:49.583 INFO:teuthology.orchestra.run.trial094.stdout:(10/10): python3-websocket-client-1.2.3-2.el9.n 563 kB/s | 90 kB 00:00 2026-02-25T09:17:49.585 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:49.585 INFO:teuthology.orchestra.run.trial094.stdout:Total 1.3 MB/s | 2.0 MB 00:01 2026-02-25T09:17:49.610 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:17:49.618 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:17:49.618 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:17:49.658 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:17:49.658 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:17:49.734 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:17:49.785 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-pyasn1-0.4.8-7.el9.noarch 1/10 2026-02-25T09:17:49.798 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-pyasn1-modules-0.4.8-7.el9.noarch 2/10 2026-02-25T09:17:49.812 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-rsa-4.9-2.el9.noarch 3/10 2026-02-25T09:17:49.817 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-websocket-client-1.2.3-2.el9.noarch 4/10 2026-02-25T09:17:49.825 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-certifi-2023.05.07-4.el9.noarch 5/10 2026-02-25T09:17:49.864 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-cachetools-4.2.4-1.el9.noarch 6/10 2026-02-25T09:17:49.876 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-25T09:17:50.196 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-requests-oauthlib-1.3.0-12.el9.noarch 8/10 2026-02-25T09:17:50.211 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-kubernetes-1:26.1.0-3.el9.noarch 9/10 2026-02-25T09:17:50.212 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mgr-rook-2:20.3.0-5536.ge6eec768.el9.noarch 10/10 2026-02-25T09:17:50.327 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mgr-rook-2:20.3.0-5536.ge6eec768.el9.noarch 10/10 2026-02-25T09:17:50.327 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mgr-rook-2:20.3.0-5536.ge6eec768.el9.noarch 1/10 2026-02-25T09:17:50.327 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-pyasn1-0.4.8-7.el9.noarch 2/10 2026-02-25T09:17:50.327 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-pyasn1-modules-0.4.8-7.el9.noarch 3/10 2026-02-25T09:17:50.327 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-requests-oauthlib-1.3.0-12.el9.noarch 4/10 2026-02-25T09:17:50.327 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-cachetools-4.2.4-1.el9.noarch 5/10 2026-02-25T09:17:50.327 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-certifi-2023.05.07-4.el9.noarch 6/10 2026-02-25T09:17:50.327 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-google-auth-1:2.45.0-1.el9.noarch 7/10 2026-02-25T09:17:50.327 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-kubernetes-1:26.1.0-3.el9.noarch 8/10 2026-02-25T09:17:50.327 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-rsa-4.9-2.el9.noarch 9/10 2026-02-25T09:17:50.389 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-websocket-client-1.2.3-2.el9.noarch 10/10 2026-02-25T09:17:50.389 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:50.389 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:17:50.389 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-rook-2:20.3.0-5536.ge6eec768.el9.noarch 2026-02-25T09:17:50.389 INFO:teuthology.orchestra.run.trial094.stdout: python3-cachetools-4.2.4-1.el9.noarch 2026-02-25T09:17:50.389 INFO:teuthology.orchestra.run.trial094.stdout: python3-certifi-2023.05.07-4.el9.noarch 2026-02-25T09:17:50.389 INFO:teuthology.orchestra.run.trial094.stdout: python3-google-auth-1:2.45.0-1.el9.noarch 2026-02-25T09:17:50.389 INFO:teuthology.orchestra.run.trial094.stdout: python3-kubernetes-1:26.1.0-3.el9.noarch 2026-02-25T09:17:50.389 INFO:teuthology.orchestra.run.trial094.stdout: python3-pyasn1-0.4.8-7.el9.noarch 2026-02-25T09:17:50.389 INFO:teuthology.orchestra.run.trial094.stdout: python3-pyasn1-modules-0.4.8-7.el9.noarch 2026-02-25T09:17:50.389 INFO:teuthology.orchestra.run.trial094.stdout: python3-requests-oauthlib-1.3.0-12.el9.noarch 2026-02-25T09:17:50.389 INFO:teuthology.orchestra.run.trial094.stdout: python3-rsa-4.9-2.el9.noarch 2026-02-25T09:17:50.390 INFO:teuthology.orchestra.run.trial094.stdout: python3-websocket-client-1.2.3-2.el9.noarch 2026-02-25T09:17:50.390 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:50.390 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:17:50.418 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T09:17:50.735 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:33 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:17:50.818 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:17:50.819 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:50.819 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repository Size 2026-02-25T09:17:50.819 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:50.820 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:17:50.820 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-cephadm noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 184 k 2026-02-25T09:17:50.820 INFO:teuthology.orchestra.run.trial094.stdout:Installing dependencies: 2026-02-25T09:17:50.820 INFO:teuthology.orchestra.run.trial094.stdout: python3-asyncssh noarch 2.13.2-5.el9 epel 548 k 2026-02-25T09:17:50.820 INFO:teuthology.orchestra.run.trial094.stdout: python3-babel noarch 2.9.1-2.el9 appstream 6.0 M 2026-02-25T09:17:50.820 INFO:teuthology.orchestra.run.trial094.stdout: python3-jinja2 noarch 2.11.3-8.el9 appstream 249 k 2026-02-25T09:17:50.820 INFO:teuthology.orchestra.run.trial094.stdout: python3-markupsafe x86_64 1.1.1-12.el9 appstream 35 k 2026-02-25T09:17:50.820 INFO:teuthology.orchestra.run.trial094.stdout: python3-natsort noarch 7.1.1-5.el9 epel 58 k 2026-02-25T09:17:50.820 INFO:teuthology.orchestra.run.trial094.stdout: python3-typing-extensions noarch 4.15.0-1.el9 epel 86 k 2026-02-25T09:17:50.820 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:50.820 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:17:50.820 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:50.821 INFO:teuthology.orchestra.run.trial094.stdout:Install 7 Packages 2026-02-25T09:17:50.821 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:50.821 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 7.1 M 2026-02-25T09:17:50.821 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 34 M 2026-02-25T09:17:50.821 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:17:51.635 INFO:teuthology.orchestra.run.trial094.stdout:(1/7): ceph-mgr-cephadm-20.3.0-5536.ge6eec768.e 298 kB/s | 184 kB 00:00 2026-02-25T09:17:51.664 INFO:teuthology.orchestra.run.trial094.stdout:(2/7): python3-jinja2-2.11.3-8.el9.noarch.rpm 384 kB/s | 249 kB 00:00 2026-02-25T09:17:51.987 INFO:teuthology.orchestra.run.trial094.stdout:(3/7): python3-babel-2.9.1-2.el9.noarch.rpm 6.1 MB/s | 6.0 MB 00:00 2026-02-25T09:17:52.020 INFO:teuthology.orchestra.run.trial094.stdout:(4/7): python3-asyncssh-2.13.2-5.el9.noarch.rpm 1.5 MB/s | 548 kB 00:00 2026-02-25T09:17:52.034 INFO:teuthology.orchestra.run.trial094.stdout:(5/7): python3-markupsafe-1.1.1-12.el9.x86_64.r 87 kB/s | 35 kB 00:00 2026-02-25T09:17:52.066 INFO:teuthology.orchestra.run.trial094.stdout:(6/7): python3-typing-extensions-4.15.0-1.el9.n 1.9 MB/s | 86 kB 00:00 2026-02-25T09:17:52.201 INFO:teuthology.orchestra.run.trial094.stdout:(7/7): python3-natsort-7.1.1-5.el9.noarch.rpm 270 kB/s | 58 kB 00:00 2026-02-25T09:17:52.206 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:52.206 INFO:teuthology.orchestra.run.trial094.stdout:Total 5.1 MB/s | 7.1 MB 00:01 2026-02-25T09:17:52.244 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:17:52.251 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:17:52.251 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:17:52.324 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:17:52.324 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:17:52.414 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:17:52.463 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-typing-extensions-4.15.0-1.el9.noarch 1/7 2026-02-25T09:17:52.473 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-asyncssh-2.13.2-5.el9.noarch 2/7 2026-02-25T09:17:52.479 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-natsort-7.1.1-5.el9.noarch 3/7 2026-02-25T09:17:52.698 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-25T09:17:52.722 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-babel-2.9.1-2.el9.noarch 5/7 2026-02-25T09:17:52.746 INFO:teuthology.orchestra.run.trial094.stdout: Installing : python3-jinja2-2.11.3-8.el9.noarch 6/7 2026-02-25T09:17:52.747 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-mgr-cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 7/7 2026-02-25T09:17:52.863 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-mgr-cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 7/7 2026-02-25T09:17:52.863 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-mgr-cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 1/7 2026-02-25T09:17:52.863 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-babel-2.9.1-2.el9.noarch 2/7 2026-02-25T09:17:52.863 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-jinja2-2.11.3-8.el9.noarch 3/7 2026-02-25T09:17:52.863 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-markupsafe-1.1.1-12.el9.x86_64 4/7 2026-02-25T09:17:52.863 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-asyncssh-2.13.2-5.el9.noarch 5/7 2026-02-25T09:17:52.864 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-natsort-7.1.1-5.el9.noarch 6/7 2026-02-25T09:17:52.924 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : python3-typing-extensions-4.15.0-1.el9.noarch 7/7 2026-02-25T09:17:52.925 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:52.925 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:17:52.925 INFO:teuthology.orchestra.run.trial094.stdout: ceph-mgr-cephadm-2:20.3.0-5536.ge6eec768.el9.noarch 2026-02-25T09:17:52.925 INFO:teuthology.orchestra.run.trial094.stdout: python3-asyncssh-2.13.2-5.el9.noarch 2026-02-25T09:17:52.925 INFO:teuthology.orchestra.run.trial094.stdout: python3-babel-2.9.1-2.el9.noarch 2026-02-25T09:17:52.925 INFO:teuthology.orchestra.run.trial094.stdout: python3-jinja2-2.11.3-8.el9.noarch 2026-02-25T09:17:52.925 INFO:teuthology.orchestra.run.trial094.stdout: python3-markupsafe-1.1.1-12.el9.x86_64 2026-02-25T09:17:52.925 INFO:teuthology.orchestra.run.trial094.stdout: python3-natsort-7.1.1-5.el9.noarch 2026-02-25T09:17:52.925 INFO:teuthology.orchestra.run.trial094.stdout: python3-typing-extensions-4.15.0-1.el9.noarch 2026-02-25T09:17:52.926 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:52.926 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:17:52.956 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-osd 2026-02-25T09:17:53.275 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:36 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:17:53.340 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-osd-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:17:53.358 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:17:53.359 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:17:53.359 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:17:53.382 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-osd-classic 2026-02-25T09:17:53.698 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:36 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:17:53.765 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-osd-classic-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:17:53.784 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:17:53.784 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:17:53.784 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:17:53.806 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-fuse 2026-02-25T09:17:54.122 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:37 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:17:54.207 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:17:54.208 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:54.208 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T09:17:54.208 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:54.208 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:17:54.208 INFO:teuthology.orchestra.run.trial094.stdout: ceph-fuse x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 914 k 2026-02-25T09:17:54.209 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:54.209 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:17:54.209 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:54.209 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T09:17:54.209 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:54.209 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 914 k 2026-02-25T09:17:54.209 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 2.9 M 2026-02-25T09:17:54.209 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:17:55.357 INFO:teuthology.orchestra.run.trial094.stdout:ceph-fuse-20.3.0-5536.ge6eec768.el9.x86_64.rpm 796 kB/s | 914 kB 00:01 2026-02-25T09:17:55.358 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:55.358 INFO:teuthology.orchestra.run.trial094.stdout:Total 795 kB/s | 914 kB 00:01 2026-02-25T09:17:55.358 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:17:55.363 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:17:55.363 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:17:55.395 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:17:55.395 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:17:55.485 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:17:55.503 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:55.748 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:55.809 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:55.809 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:55.809 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:17:55.809 INFO:teuthology.orchestra.run.trial094.stdout: ceph-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:55.809 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:55.810 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:17:55.836 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-volume 2026-02-25T09:17:56.160 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:39 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:17:56.244 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:17:56.245 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:56.245 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repository Size 2026-02-25T09:17:56.245 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:56.245 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:17:56.245 INFO:teuthology.orchestra.run.trial094.stdout: ceph-volume noarch 2:20.3.0-5536.ge6eec768.el9 ceph-noarch 301 k 2026-02-25T09:17:56.245 INFO:teuthology.orchestra.run.trial094.stdout:Installing dependencies: 2026-02-25T09:17:56.246 INFO:teuthology.orchestra.run.trial094.stdout: cryptsetup x86_64 2.8.1-3.el9 baseos 351 k 2026-02-25T09:17:56.246 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:56.246 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:17:56.246 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:56.246 INFO:teuthology.orchestra.run.trial094.stdout:Install 2 Packages 2026-02-25T09:17:56.246 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:56.246 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 652 k 2026-02-25T09:17:56.246 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 2.2 M 2026-02-25T09:17:56.246 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:17:56.363 INFO:teuthology.orchestra.run.trial094.stdout:(1/2): cryptsetup-2.8.1-3.el9.x86_64.rpm 5.8 MB/s | 351 kB 00:00 2026-02-25T09:17:57.271 INFO:teuthology.orchestra.run.trial094.stdout:(2/2): ceph-volume-20.3.0-5536.ge6eec768.el9.no 312 kB/s | 301 kB 00:00 2026-02-25T09:17:57.273 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:57.273 INFO:teuthology.orchestra.run.trial094.stdout:Total 635 kB/s | 652 kB 00:01 2026-02-25T09:17:57.281 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:17:57.287 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:17:57.288 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:17:57.331 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:17:57.331 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:17:57.412 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:17:57.475 INFO:teuthology.orchestra.run.trial094.stdout: Installing : cryptsetup-2.8.1-3.el9.x86_64 1/2 2026-02-25T09:17:57.477 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-volume-2:20.3.0-5536.ge6eec768.el9.noarch 2/2 2026-02-25T09:17:57.491 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-volume-2:20.3.0-5536.ge6eec768.el9.noarch 2/2 2026-02-25T09:17:57.491 INFO:teuthology.orchestra.run.trial094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T09:17:57.492 INFO:teuthology.orchestra.run.trial094.stdout:Invalid unit name "ceph-volume@*.service" escaped as "ceph-volume@\x2a.service". 2026-02-25T09:17:57.492 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:57.723 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-volume-2:20.3.0-5536.ge6eec768.el9.noarch 1/2 2026-02-25T09:17:57.785 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : cryptsetup-2.8.1-3.el9.x86_64 2/2 2026-02-25T09:17:57.786 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:57.786 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:17:57.786 INFO:teuthology.orchestra.run.trial094.stdout: ceph-volume-2:20.3.0-5536.ge6eec768.el9.noarch cryptsetup-2.8.1-3.el9.x86_64 2026-02-25T09:17:57.786 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:57.786 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:17:57.816 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install librados-devel 2026-02-25T09:17:58.135 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:41 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:17:58.219 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:17:58.219 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:58.219 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T09:17:58.219 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:58.219 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:17:58.220 INFO:teuthology.orchestra.run.trial094.stdout: librados-devel x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 124 k 2026-02-25T09:17:58.220 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:58.220 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:17:58.220 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:58.220 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T09:17:58.220 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:58.220 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 124 k 2026-02-25T09:17:58.220 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 434 k 2026-02-25T09:17:58.220 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:17:58.779 INFO:teuthology.orchestra.run.trial094.stdout:librados-devel-20.3.0-5536.ge6eec768.el9.x86_64 221 kB/s | 124 kB 00:00 2026-02-25T09:17:58.781 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:17:58.781 INFO:teuthology.orchestra.run.trial094.stdout:Total 220 kB/s | 124 kB 00:00 2026-02-25T09:17:58.781 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:17:58.784 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:17:58.784 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:17:58.815 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:17:58.815 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:17:58.870 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:17:58.887 INFO:teuthology.orchestra.run.trial094.stdout: Installing : librados-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:58.980 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: librados-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:59.042 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : librados-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:17:59.042 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:59.043 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:17:59.043 INFO:teuthology.orchestra.run.trial094.stdout: librados-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:17:59.043 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:59.043 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:17:59.070 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install libcephfs2 2026-02-25T09:17:59.396 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:42 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:17:59.462 INFO:teuthology.orchestra.run.trial094.stdout:Package libcephfs2-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:17:59.480 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:17:59.481 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:17:59.481 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:17:59.503 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install libcephfs-devel 2026-02-25T09:17:59.826 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:42 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:17:59.910 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:17:59.911 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:59.912 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T09:17:59.912 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:59.912 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:17:59.912 INFO:teuthology.orchestra.run.trial094.stdout: libcephfs-devel x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 37 k 2026-02-25T09:17:59.912 INFO:teuthology.orchestra.run.trial094.stdout:Installing dependencies: 2026-02-25T09:17:59.912 INFO:teuthology.orchestra.run.trial094.stdout: libcephfs-proxy2 x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 26 k 2026-02-25T09:17:59.912 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:59.912 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:17:59.912 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:17:59.912 INFO:teuthology.orchestra.run.trial094.stdout:Install 2 Packages 2026-02-25T09:17:59.913 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:17:59.913 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 64 k 2026-02-25T09:17:59.913 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 231 k 2026-02-25T09:17:59.913 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:18:00.252 INFO:teuthology.orchestra.run.trial094.stdout:(1/2): libcephfs-devel-20.3.0-5536.ge6eec768.el 110 kB/s | 37 kB 00:00 2026-02-25T09:18:00.255 INFO:teuthology.orchestra.run.trial094.stdout:(2/2): libcephfs-proxy2-20.3.0-5536.ge6eec768.e 76 kB/s | 26 kB 00:00 2026-02-25T09:18:00.257 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:18:00.257 INFO:teuthology.orchestra.run.trial094.stdout:Total 184 kB/s | 64 kB 00:00 2026-02-25T09:18:00.258 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:18:00.260 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:18:00.261 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:18:00.294 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:18:00.294 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:18:00.344 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:18:00.345 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libcephfs-proxy2-2:20.3.0-5536.ge6eec768.el9.x86_64 1/2 2026-02-25T09:18:00.360 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: libcephfs-proxy2-2:20.3.0-5536.ge6eec768.el9.x86_64 1/2 2026-02-25T09:18:00.377 INFO:teuthology.orchestra.run.trial094.stdout: Installing : libcephfs-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 2/2 2026-02-25T09:18:00.441 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: libcephfs-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 2/2 2026-02-25T09:18:00.441 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libcephfs-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 1/2 2026-02-25T09:18:00.502 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : libcephfs-proxy2-2:20.3.0-5536.ge6eec768.el9.x86_64 2/2 2026-02-25T09:18:00.503 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:00.503 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:18:00.503 INFO:teuthology.orchestra.run.trial094.stdout: libcephfs-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:18:00.503 INFO:teuthology.orchestra.run.trial094.stdout: libcephfs-proxy2-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:18:00.503 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:00.503 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:18:00.531 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install librados2 2026-02-25T09:18:00.850 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:43 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:18:00.916 INFO:teuthology.orchestra.run.trial094.stdout:Package librados2-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:18:00.935 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:18:00.936 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:18:00.936 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:18:00.957 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install librbd1 2026-02-25T09:18:01.275 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:44 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:18:01.340 INFO:teuthology.orchestra.run.trial094.stdout:Package librbd1-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:18:01.359 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:18:01.360 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:18:01.360 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:18:01.382 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install python3-rados 2026-02-25T09:18:01.702 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:44 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:18:01.768 INFO:teuthology.orchestra.run.trial094.stdout:Package python3-rados-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:18:01.787 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:18:01.788 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:18:01.788 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:18:01.809 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install python3-rgw 2026-02-25T09:18:02.126 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:45 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:18:02.192 INFO:teuthology.orchestra.run.trial094.stdout:Package python3-rgw-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:18:02.211 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:18:02.211 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:18:02.212 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:18:02.233 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install python3-cephfs 2026-02-25T09:18:02.556 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:45 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:18:02.621 INFO:teuthology.orchestra.run.trial094.stdout:Package python3-cephfs-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:18:02.640 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:18:02.640 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:18:02.641 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:18:02.662 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install python3-rbd 2026-02-25T09:18:02.985 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:45 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:18:03.052 INFO:teuthology.orchestra.run.trial094.stdout:Package python3-rbd-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:18:03.071 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:18:03.072 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:18:03.072 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:18:03.094 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install rbd-fuse 2026-02-25T09:18:03.418 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:46 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:18:03.503 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:18:03.503 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:18:03.503 INFO:teuthology.orchestra.run.trial094.stdout: Package Architecture Version Repository Size 2026-02-25T09:18:03.503 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:18:03.503 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:18:03.503 INFO:teuthology.orchestra.run.trial094.stdout: rbd-fuse x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 87 k 2026-02-25T09:18:03.504 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:03.504 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:18:03.504 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:18:03.504 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T09:18:03.504 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:03.504 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 87 k 2026-02-25T09:18:03.504 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 236 k 2026-02-25T09:18:03.504 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:18:03.912 INFO:teuthology.orchestra.run.trial094.stdout:rbd-fuse-20.3.0-5536.ge6eec768.el9.x86_64.rpm 212 kB/s | 87 kB 00:00 2026-02-25T09:18:03.912 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:18:03.912 INFO:teuthology.orchestra.run.trial094.stdout:Total 212 kB/s | 87 kB 00:00 2026-02-25T09:18:03.912 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:18:03.916 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:18:03.917 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:18:03.947 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:18:03.947 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:18:03.999 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:18:04.016 INFO:teuthology.orchestra.run.trial094.stdout: Installing : rbd-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:18:04.104 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: rbd-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:18:04.167 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : rbd-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:18:04.168 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:04.168 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:18:04.168 INFO:teuthology.orchestra.run.trial094.stdout: rbd-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:18:04.168 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:04.168 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:18:04.198 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install rbd-mirror 2026-02-25T09:18:04.519 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:47 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:18:04.603 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:18:04.603 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:18:04.604 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T09:18:04.604 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:18:04.604 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:18:04.604 INFO:teuthology.orchestra.run.trial094.stdout: rbd-mirror x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 2.6 M 2026-02-25T09:18:04.604 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:04.604 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:18:04.604 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:18:04.604 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T09:18:04.604 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:04.604 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 2.6 M 2026-02-25T09:18:04.605 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 11 M 2026-02-25T09:18:04.605 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:18:07.864 INFO:teuthology.orchestra.run.trial094.stdout:rbd-mirror-20.3.0-5536.ge6eec768.el9.x86_64.rpm 814 kB/s | 2.6 MB 00:03 2026-02-25T09:18:07.866 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:18:07.866 INFO:teuthology.orchestra.run.trial094.stdout:Total 814 kB/s | 2.6 MB 00:03 2026-02-25T09:18:07.867 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:18:07.872 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:18:07.872 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:18:07.910 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:18:07.910 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:18:08.080 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:18:08.081 INFO:teuthology.orchestra.run.trial094.stdout: Installing : rbd-mirror-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:18:08.104 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: rbd-mirror-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:18:08.104 INFO:teuthology.orchestra.run.trial094.stdout:Glob pattern passed to enable, but globs are not supported for this. 2026-02-25T09:18:08.104 INFO:teuthology.orchestra.run.trial094.stdout:Invalid unit name "ceph-rbd-mirror@*.service" escaped as "ceph-rbd-mirror@\x2a.service". 2026-02-25T09:18:08.104 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/multi-user.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T09:18:08.104 INFO:teuthology.orchestra.run.trial094.stdout:Created symlink /etc/systemd/system/ceph.target.wants/ceph-rbd-mirror.target → /usr/lib/systemd/system/ceph-rbd-mirror.target. 2026-02-25T09:18:08.104 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:08.434 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : rbd-mirror-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:18:08.434 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:08.434 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:18:08.434 INFO:teuthology.orchestra.run.trial094.stdout: rbd-mirror-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:18:08.435 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:08.435 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:18:08.462 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install rbd-nbd 2026-02-25T09:18:08.778 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:51 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:18:08.865 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:18:08.866 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:18:08.866 INFO:teuthology.orchestra.run.trial094.stdout: Package Architecture Version Repository Size 2026-02-25T09:18:08.866 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:18:08.866 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:18:08.866 INFO:teuthology.orchestra.run.trial094.stdout: rbd-nbd x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 166 k 2026-02-25T09:18:08.867 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:08.867 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:18:08.867 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:18:08.867 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T09:18:08.867 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:08.867 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 166 k 2026-02-25T09:18:08.867 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 496 k 2026-02-25T09:18:08.867 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:18:09.492 INFO:teuthology.orchestra.run.trial094.stdout:rbd-nbd-20.3.0-5536.ge6eec768.el9.x86_64.rpm 266 kB/s | 166 kB 00:00 2026-02-25T09:18:09.493 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:18:09.494 INFO:teuthology.orchestra.run.trial094.stdout:Total 265 kB/s | 166 kB 00:00 2026-02-25T09:18:09.494 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:18:09.499 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:18:09.499 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:18:09.530 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:18:09.530 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:18:09.586 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:18:09.604 INFO:teuthology.orchestra.run.trial094.stdout: Installing : rbd-nbd-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:18:09.700 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: rbd-nbd-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:18:09.762 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : rbd-nbd-2:20.3.0-5536.ge6eec768.el9.x86_64 1/1 2026-02-25T09:18:09.763 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:09.763 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:18:09.763 INFO:teuthology.orchestra.run.trial094.stdout: rbd-nbd-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:18:09.763 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:09.763 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:18:09.791 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-osd-crimson 2026-02-25T09:18:10.113 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:53 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:18:10.178 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-osd-crimson-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:18:10.196 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:18:10.197 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:18:10.197 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:18:10.218 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-osd-crimson-debuginfo 2026-02-25T09:18:10.538 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:05:53 ago on Wed 25 Feb 2026 09:12:17 AM UTC. 2026-02-25T09:18:10.623 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:18:10.624 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:18:10.624 INFO:teuthology.orchestra.run.trial094.stdout: Package Arch Version Repo Size 2026-02-25T09:18:10.624 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:18:10.624 INFO:teuthology.orchestra.run.trial094.stdout:Installing: 2026-02-25T09:18:10.624 INFO:teuthology.orchestra.run.trial094.stdout: ceph-osd-crimson-debuginfo x86_64 2:20.3.0-5536.ge6eec768.el9 ceph 429 M 2026-02-25T09:18:10.624 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:10.624 INFO:teuthology.orchestra.run.trial094.stdout:Transaction Summary 2026-02-25T09:18:10.624 INFO:teuthology.orchestra.run.trial094.stdout:================================================================================ 2026-02-25T09:18:10.625 INFO:teuthology.orchestra.run.trial094.stdout:Install 1 Package 2026-02-25T09:18:10.625 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:18:10.625 INFO:teuthology.orchestra.run.trial094.stdout:Total download size: 429 M 2026-02-25T09:18:10.625 INFO:teuthology.orchestra.run.trial094.stdout:Installed size: 2.6 G 2026-02-25T09:18:10.625 INFO:teuthology.orchestra.run.trial094.stdout:Downloading Packages: 2026-02-25T09:21:55.701 INFO:teuthology.orchestra.run.trial151.stdout:ceph-osd-crimson-debuginfo-20.3.0-5536.ge6eec76 1.7 MB/s | 429 MB 04:14 2026-02-25T09:21:55.702 INFO:teuthology.orchestra.run.trial151.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:21:55.702 INFO:teuthology.orchestra.run.trial151.stdout:Total 1.7 MB/s | 429 MB 04:14 2026-02-25T09:21:55.702 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction check 2026-02-25T09:21:55.704 INFO:teuthology.orchestra.run.trial151.stdout:Transaction check succeeded. 2026-02-25T09:21:55.704 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction test 2026-02-25T09:21:56.981 INFO:teuthology.orchestra.run.trial151.stdout:Transaction test succeeded. 2026-02-25T09:21:56.981 INFO:teuthology.orchestra.run.trial151.stdout:Running transaction 2026-02-25T09:22:00.989 INFO:teuthology.orchestra.run.trial094.stdout:ceph-osd-crimson-debuginfo-20.3.0-5536.ge6eec76 1.9 MB/s | 429 MB 03:50 2026-02-25T09:22:00.989 INFO:teuthology.orchestra.run.trial094.stdout:-------------------------------------------------------------------------------- 2026-02-25T09:22:00.990 INFO:teuthology.orchestra.run.trial094.stdout:Total 1.9 MB/s | 429 MB 03:50 2026-02-25T09:22:00.990 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction check 2026-02-25T09:22:00.991 INFO:teuthology.orchestra.run.trial094.stdout:Transaction check succeeded. 2026-02-25T09:22:00.992 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction test 2026-02-25T09:22:02.272 INFO:teuthology.orchestra.run.trial094.stdout:Transaction test succeeded. 2026-02-25T09:22:02.272 INFO:teuthology.orchestra.run.trial094.stdout:Running transaction 2026-02-25T09:22:21.389 INFO:teuthology.orchestra.run.trial151.stdout: Preparing : 1/1 2026-02-25T09:22:21.406 INFO:teuthology.orchestra.run.trial151.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5536.ge6eec768.e 1/1 2026-02-25T09:22:23.337 INFO:teuthology.orchestra.run.trial151.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5536.ge6eec768.e 1/1 2026-02-25T09:22:23.400 INFO:teuthology.orchestra.run.trial151.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5536.ge6eec768.e 1/1 2026-02-25T09:22:23.400 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:22:23.400 INFO:teuthology.orchestra.run.trial151.stdout:Installed: 2026-02-25T09:22:23.400 INFO:teuthology.orchestra.run.trial151.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:22:23.400 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:22:23.400 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:23.469 DEBUG:teuthology.parallel:result is None 2026-02-25T09:22:25.639 INFO:teuthology.orchestra.run.trial094.stdout: Preparing : 1/1 2026-02-25T09:22:25.656 INFO:teuthology.orchestra.run.trial094.stdout: Installing : ceph-osd-crimson-debuginfo-2:20.3.0-5536.ge6eec768.e 1/1 2026-02-25T09:22:27.491 INFO:teuthology.orchestra.run.trial094.stdout: Running scriptlet: ceph-osd-crimson-debuginfo-2:20.3.0-5536.ge6eec768.e 1/1 2026-02-25T09:22:27.552 INFO:teuthology.orchestra.run.trial094.stdout: Verifying : ceph-osd-crimson-debuginfo-2:20.3.0-5536.ge6eec768.e 1/1 2026-02-25T09:22:27.552 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:22:27.553 INFO:teuthology.orchestra.run.trial094.stdout:Installed: 2026-02-25T09:22:27.553 INFO:teuthology.orchestra.run.trial094.stdout: ceph-osd-crimson-debuginfo-2:20.3.0-5536.ge6eec768.el9.x86_64 2026-02-25T09:22:27.553 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:22:27.553 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:22:27.626 DEBUG:teuthology.parallel:result is None 2026-02-25T09:22:27.627 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=e6eec7683022e6824acc16f4c26421a3804e27e6 2026-02-25T09:22:27.796 DEBUG:teuthology.orchestra.run.trial094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T09:22:27.817 INFO:teuthology.orchestra.run.trial094.stdout:20.3.0-5536.ge6eec768.el9 2026-02-25T09:22:27.817 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5536.ge6eec768.el9 2026-02-25T09:22:27.818 INFO:teuthology.task.install:The correct ceph version 20.3.0-5536.ge6eec768 is installed. 2026-02-25T09:22:27.821 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=e6eec7683022e6824acc16f4c26421a3804e27e6 2026-02-25T09:22:27.957 DEBUG:teuthology.orchestra.run.trial151:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T09:22:27.979 INFO:teuthology.orchestra.run.trial151.stdout:20.3.0-5536.ge6eec768.el9 2026-02-25T09:22:27.979 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5536.ge6eec768.el9 2026-02-25T09:22:27.979 INFO:teuthology.task.install:The correct ceph version 20.3.0-5536.ge6eec768 is installed. 2026-02-25T09:22:27.981 INFO:teuthology.task.install.util:Shipping valgrind.supp... 2026-02-25T09:22:27.981 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:22:27.981 DEBUG:teuthology.orchestra.run.trial094:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-25T09:22:28.007 DEBUG:teuthology.orchestra.run.trial151:> set -ex 2026-02-25T09:22:28.007 DEBUG:teuthology.orchestra.run.trial151:> sudo dd of=/home/ubuntu/cephtest/valgrind.supp 2026-02-25T09:22:28.044 INFO:teuthology.task.install.util:Shipping 'daemon-helper'... 2026-02-25T09:22:28.045 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:22:28.045 DEBUG:teuthology.orchestra.run.trial094:> sudo dd of=/usr/bin/daemon-helper 2026-02-25T09:22:28.072 DEBUG:teuthology.orchestra.run.trial094:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-25T09:22:28.136 DEBUG:teuthology.orchestra.run.trial151:> set -ex 2026-02-25T09:22:28.136 DEBUG:teuthology.orchestra.run.trial151:> sudo dd of=/usr/bin/daemon-helper 2026-02-25T09:22:28.161 DEBUG:teuthology.orchestra.run.trial151:> sudo chmod a=rx -- /usr/bin/daemon-helper 2026-02-25T09:22:28.224 INFO:teuthology.task.install.util:Shipping 'adjust-ulimits'... 2026-02-25T09:22:28.224 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:22:28.224 DEBUG:teuthology.orchestra.run.trial094:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-25T09:22:28.249 DEBUG:teuthology.orchestra.run.trial094:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-25T09:22:28.313 DEBUG:teuthology.orchestra.run.trial151:> set -ex 2026-02-25T09:22:28.313 DEBUG:teuthology.orchestra.run.trial151:> sudo dd of=/usr/bin/adjust-ulimits 2026-02-25T09:22:28.339 DEBUG:teuthology.orchestra.run.trial151:> sudo chmod a=rx -- /usr/bin/adjust-ulimits 2026-02-25T09:22:28.403 INFO:teuthology.task.install.util:Shipping 'stdin-killer'... 2026-02-25T09:22:28.403 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:22:28.403 DEBUG:teuthology.orchestra.run.trial094:> sudo dd of=/usr/bin/stdin-killer 2026-02-25T09:22:28.428 DEBUG:teuthology.orchestra.run.trial094:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-25T09:22:28.493 DEBUG:teuthology.orchestra.run.trial151:> set -ex 2026-02-25T09:22:28.493 DEBUG:teuthology.orchestra.run.trial151:> sudo dd of=/usr/bin/stdin-killer 2026-02-25T09:22:28.518 DEBUG:teuthology.orchestra.run.trial151:> sudo chmod a=rx -- /usr/bin/stdin-killer 2026-02-25T09:22:28.581 INFO:teuthology.run_tasks:Running task pexec... 2026-02-25T09:22:28.594 INFO:teuthology.task.pexec:Executing custom commands... 2026-02-25T09:22:28.594 DEBUG:teuthology.orchestra.run.trial094:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-25T09:22:28.594 DEBUG:teuthology.orchestra.run.trial151:> TESTDIR=/home/ubuntu/cephtest bash -s 2026-02-25T09:22:28.596 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial094.front.sepia.ceph.com 2026-02-25T09:22:28.596 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-25T09:22:28.624 INFO:teuthology.task.pexec:Running commands on host ubuntu@trial151.front.sepia.ceph.com 2026-02-25T09:22:28.624 INFO:teuthology.task.pexec:sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson 2026-02-25T09:22:28.647 DEBUG:teuthology.parallel:result is None 2026-02-25T09:22:28.675 DEBUG:teuthology.parallel:result is None 2026-02-25T09:22:28.675 INFO:teuthology.run_tasks:Running task install... 2026-02-25T09:22:28.686 DEBUG:teuthology.task.install:project ceph 2026-02-25T09:22:28.686 DEBUG:teuthology.task.install:INSTALL overrides: {'ceph': {'flavor': 'debug', 'sha1': 'e6eec7683022e6824acc16f4c26421a3804e27e6'}} 2026-02-25T09:22:28.686 DEBUG:teuthology.task.install:config {'flavor': 'debug', 'sha1': 'e6eec7683022e6824acc16f4c26421a3804e27e6'} 2026-02-25T09:22:28.686 INFO:teuthology.task.install:Using flavor: debug 2026-02-25T09:22:28.689 DEBUG:teuthology.task.install:Package list is: {'deb': ['ceph', 'cephadm', 'ceph-mds', 'ceph-mgr', 'ceph-osd', 'ceph-osd-classic', 'ceph-common', 'ceph-fuse', 'ceph-test', 'ceph-volume', 'radosgw', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'libcephfs2', 'libcephfs-dev', 'librados2', 'librbd1', 'rbd-fuse'], 'rpm': ['ceph-radosgw', 'ceph-test', 'ceph', 'ceph-base', 'cephadm', 'ceph-immutable-object-cache', 'ceph-mgr', 'ceph-mgr-dashboard', 'ceph-mgr-diskprediction-local', 'ceph-mgr-rook', 'ceph-mgr-cephadm', 'ceph-osd', 'ceph-osd-classic', 'ceph-fuse', 'ceph-volume', 'librados-devel', 'libcephfs2', 'libcephfs-devel', 'librados2', 'librbd1', 'python3-rados', 'python3-rgw', 'python3-cephfs', 'python3-rbd', 'rbd-fuse', 'rbd-mirror', 'rbd-nbd']} 2026-02-25T09:22:28.690 INFO:teuthology.task.install:extra packages: [] 2026-02-25T09:22:28.690 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e6eec7683022e6824acc16f4c26421a3804e27e6', 'tag': None, 'wait_for_package': False} 2026-02-25T09:22:28.690 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=e6eec7683022e6824acc16f4c26421a3804e27e6 2026-02-25T09:22:28.691 DEBUG:teuthology.task.install.rpm:_update_package_list_and_install: config is {'branch': None, 'cleanup': None, 'debuginfo': None, 'downgrade_packages': [], 'exclude_packages': [], 'extra_packages': [], 'extra_system_packages': [], 'extras': None, 'enable_coprs': [], 'flavor': 'debug', 'install_ceph_packages': True, 'packages': {}, 'project': 'ceph', 'repos_only': False, 'sha1': 'e6eec7683022e6824acc16f4c26421a3804e27e6', 'tag': None, 'wait_for_package': False} 2026-02-25T09:22:28.691 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=e6eec7683022e6824acc16f4c26421a3804e27e6 2026-02-25T09:22:28.867 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/e6eec7683022e6824acc16f4c26421a3804e27e6/centos/9/flavors/debug/ 2026-02-25T09:22:28.867 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5536.ge6eec768 2026-02-25T09:22:28.876 INFO:teuthology.task.install.rpm:Pulling from https://3.chacra.ceph.com/r/ceph/main/e6eec7683022e6824acc16f4c26421a3804e27e6/centos/9/flavors/debug/ 2026-02-25T09:22:28.876 INFO:teuthology.task.install.rpm:Package version is 20.3.0-5536.ge6eec768 2026-02-25T09:22:29.242 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/e6eec7683022e6824acc16f4c26421a3804e27e6/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/e6eec7683022e6824acc16f4c26421a3804e27e6/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/e6eec7683022e6824acc16f4c26421a3804e27e6/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T09:22:29.242 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:22:29.242 DEBUG:teuthology.orchestra.run.trial094:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T09:22:29.268 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-25T09:22:29.269 DEBUG:teuthology.orchestra.run.trial094:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e6eec7683022e6824acc16f4c26421a3804e27e6/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T09:22:29.284 INFO:teuthology.packaging:Writing yum repo: [ceph] name=ceph packages for $basearch baseurl=https://3.chacra.ceph.com/r/ceph/main/e6eec7683022e6824acc16f4c26421a3804e27e6/centos/9/flavors/debug/$basearch enabled=1 gpgcheck=0 type=rpm-md [ceph-noarch] name=ceph noarch packages baseurl=https://3.chacra.ceph.com/r/ceph/main/e6eec7683022e6824acc16f4c26421a3804e27e6/centos/9/flavors/debug/noarch enabled=1 gpgcheck=0 type=rpm-md [ceph-source] name=ceph source packages baseurl=https://3.chacra.ceph.com/r/ceph/main/e6eec7683022e6824acc16f4c26421a3804e27e6/centos/9/flavors/debug/SRPMS enabled=1 gpgcheck=0 type=rpm-md 2026-02-25T09:22:29.284 DEBUG:teuthology.orchestra.run.trial151:> set -ex 2026-02-25T09:22:29.285 DEBUG:teuthology.orchestra.run.trial151:> sudo dd of=/etc/yum.repos.d/ceph.repo 2026-02-25T09:22:29.309 INFO:teuthology.task.install.rpm:Installing packages: ceph-radosgw, ceph-test, ceph, ceph-base, cephadm, ceph-immutable-object-cache, ceph-mgr, ceph-mgr-dashboard, ceph-mgr-diskprediction-local, ceph-mgr-rook, ceph-mgr-cephadm, ceph-osd, ceph-osd-classic, ceph-fuse, ceph-volume, librados-devel, libcephfs2, libcephfs-devel, librados2, librbd1, python3-rados, python3-rgw, python3-cephfs, python3-rbd, rbd-fuse, rbd-mirror, rbd-nbd on remote rpm x86_64 2026-02-25T09:22:29.310 DEBUG:teuthology.orchestra.run.trial151:> if test -f /etc/yum.repos.d/ceph.repo ; then sudo sed -i -e ':a;N;$!ba;s/enabled=1\ngpg/enabled=1\npriority=1\ngpg/g' -e 's;ref/[a-zA-Z0-9_-]*/;sha1/e6eec7683022e6824acc16f4c26421a3804e27e6/;g' /etc/yum.repos.d/ceph.repo ; fi 2026-02-25T09:22:29.333 DEBUG:teuthology.orchestra.run.trial094:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T09:22:29.374 DEBUG:teuthology.orchestra.run.trial151:> sudo touch -a /etc/yum/pluginconf.d/priorities.conf ; test -e /etc/yum/pluginconf.d/priorities.conf.orig || sudo cp -af /etc/yum/pluginconf.d/priorities.conf /etc/yum/pluginconf.d/priorities.conf.orig 2026-02-25T09:22:29.399 DEBUG:teuthology.orchestra.run.trial094:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T09:22:29.439 DEBUG:teuthology.orchestra.run.trial151:> grep check_obsoletes /etc/yum/pluginconf.d/priorities.conf && sudo sed -i 's/check_obsoletes.*0/check_obsoletes = 1/g' /etc/yum/pluginconf.d/priorities.conf || echo 'check_obsoletes = 1' | sudo tee -a /etc/yum/pluginconf.d/priorities.conf 2026-02-25T09:22:29.455 INFO:teuthology.orchestra.run.trial094.stdout:check_obsoletes = 1 2026-02-25T09:22:29.464 DEBUG:teuthology.orchestra.run.trial094:> sudo yum clean all 2026-02-25T09:22:29.494 INFO:teuthology.orchestra.run.trial151.stdout:check_obsoletes = 1 2026-02-25T09:22:29.504 DEBUG:teuthology.orchestra.run.trial151:> sudo yum clean all 2026-02-25T09:22:29.626 INFO:teuthology.orchestra.run.trial094.stdout:56 files removed 2026-02-25T09:22:29.641 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-radosgw 2026-02-25T09:22:29.665 INFO:teuthology.orchestra.run.trial151.stdout:56 files removed 2026-02-25T09:22:29.680 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-radosgw 2026-02-25T09:22:30.640 INFO:teuthology.orchestra.run.trial094.stdout:ceph packages for x86_64 108 kB/s | 92 kB 00:00 2026-02-25T09:22:30.682 INFO:teuthology.orchestra.run.trial151.stdout:ceph packages for x86_64 108 kB/s | 92 kB 00:00 2026-02-25T09:22:31.305 INFO:teuthology.orchestra.run.trial094.stdout:ceph noarch packages 37 kB/s | 24 kB 00:00 2026-02-25T09:22:31.342 INFO:teuthology.orchestra.run.trial151.stdout:ceph noarch packages 37 kB/s | 24 kB 00:00 2026-02-25T09:22:31.872 INFO:teuthology.orchestra.run.trial094.stdout:ceph source packages 3.8 kB/s | 2.1 kB 00:00 2026-02-25T09:22:31.909 INFO:teuthology.orchestra.run.trial151.stdout:ceph source packages 3.8 kB/s | 2.1 kB 00:00 2026-02-25T09:22:32.289 INFO:teuthology.orchestra.run.trial151.stdout:CentOS Stream 9 - BaseOS 25 MB/s | 8.9 MB 00:00 2026-02-25T09:22:32.685 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - BaseOS 11 MB/s | 8.9 MB 00:00 2026-02-25T09:22:34.768 INFO:teuthology.orchestra.run.trial151.stdout:CentOS Stream 9 - AppStream 14 MB/s | 27 MB 00:01 2026-02-25T09:22:38.251 INFO:teuthology.orchestra.run.trial151.stdout:CentOS Stream 9 - CRB 7.5 MB/s | 8.0 MB 00:01 2026-02-25T09:22:38.537 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - AppStream 5.1 MB/s | 27 MB 00:05 2026-02-25T09:22:39.456 INFO:teuthology.orchestra.run.trial151.stdout:CentOS Stream 9 - Extras packages 45 kB/s | 20 kB 00:00 2026-02-25T09:22:40.099 INFO:teuthology.orchestra.run.trial151.stdout:Extra Packages for Enterprise Linux 35 MB/s | 20 MB 00:00 2026-02-25T09:22:43.379 INFO:teuthology.orchestra.run.trial151.stdout:lab-extras 1.0 MB/s | 50 kB 00:00 2026-02-25T09:22:44.556 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-radosgw-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:44.575 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:44.576 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:44.576 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:44.600 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-test 2026-02-25T09:22:44.916 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:01 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:44.983 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-test-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:45.002 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:45.003 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:45.003 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:45.024 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph 2026-02-25T09:22:45.341 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:45.406 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:45.425 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:45.426 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:45.426 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:45.448 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-base 2026-02-25T09:22:45.764 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:45.831 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-base-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:45.850 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:45.850 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:45.850 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:45.872 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install cephadm 2026-02-25T09:22:46.189 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:46.255 INFO:teuthology.orchestra.run.trial151.stdout:Package cephadm-2:20.3.0-5536.ge6eec768.el9.noarch is already installed. 2026-02-25T09:22:46.274 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:46.275 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:46.275 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:46.297 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T09:22:46.618 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:46.683 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-immutable-object-cache-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:46.701 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:46.702 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:46.702 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:46.724 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-mgr 2026-02-25T09:22:47.033 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:04 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:47.098 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-mgr-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:47.117 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:47.118 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:47.118 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:47.139 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T09:22:47.456 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:04 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:47.521 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-mgr-dashboard-2:20.3.0-5536.ge6eec768.el9.noarch is already installed. 2026-02-25T09:22:47.540 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:47.540 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:47.541 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:47.562 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T09:22:47.876 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:04 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:47.942 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5536.ge6eec768.el9.noarch is already installed. 2026-02-25T09:22:47.961 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:47.961 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:47.962 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:47.983 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-mgr-rook 2026-02-25T09:22:48.297 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:48.362 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-mgr-rook-2:20.3.0-5536.ge6eec768.el9.noarch is already installed. 2026-02-25T09:22:48.380 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:48.381 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:48.381 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:48.405 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T09:22:48.727 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:48.792 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-mgr-cephadm-2:20.3.0-5536.ge6eec768.el9.noarch is already installed. 2026-02-25T09:22:48.810 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:48.811 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:48.811 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:48.833 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-osd 2026-02-25T09:22:49.154 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:06 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:49.220 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-osd-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:49.239 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:49.239 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:49.239 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:49.262 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-osd-classic 2026-02-25T09:22:49.582 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:06 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:49.646 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-osd-classic-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:49.665 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:49.666 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:49.666 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:49.687 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-fuse 2026-02-25T09:22:50.006 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:07 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:50.072 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:50.091 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:50.091 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:50.092 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:50.113 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install ceph-volume 2026-02-25T09:22:50.438 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:07 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:50.503 INFO:teuthology.orchestra.run.trial151.stdout:Package ceph-volume-2:20.3.0-5536.ge6eec768.el9.noarch is already installed. 2026-02-25T09:22:50.522 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:50.522 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:50.523 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:50.544 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install librados-devel 2026-02-25T09:22:50.863 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:07 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:50.928 INFO:teuthology.orchestra.run.trial151.stdout:Package librados-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:50.946 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:50.947 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:50.947 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:50.968 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install libcephfs2 2026-02-25T09:22:51.284 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:51.351 INFO:teuthology.orchestra.run.trial151.stdout:Package libcephfs2-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:51.370 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:51.370 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:51.371 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:51.392 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install libcephfs-devel 2026-02-25T09:22:51.713 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:51.778 INFO:teuthology.orchestra.run.trial151.stdout:Package libcephfs-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:51.797 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:51.798 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:51.798 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:51.819 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install librados2 2026-02-25T09:22:52.134 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:52.202 INFO:teuthology.orchestra.run.trial151.stdout:Package librados2-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:52.220 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:52.221 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:52.221 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:52.243 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install librbd1 2026-02-25T09:22:52.564 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:52.630 INFO:teuthology.orchestra.run.trial151.stdout:Package librbd1-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:52.649 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:52.650 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:52.650 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:52.672 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install python3-rados 2026-02-25T09:22:52.991 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:53.056 INFO:teuthology.orchestra.run.trial151.stdout:Package python3-rados-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:53.075 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:53.076 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:53.076 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:53.098 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install python3-rgw 2026-02-25T09:22:53.420 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:10 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:53.486 INFO:teuthology.orchestra.run.trial151.stdout:Package python3-rgw-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:53.504 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:53.505 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:53.505 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:53.528 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install python3-cephfs 2026-02-25T09:22:53.854 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:10 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:53.920 INFO:teuthology.orchestra.run.trial151.stdout:Package python3-cephfs-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:53.939 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:53.939 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:53.940 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:53.962 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install python3-rbd 2026-02-25T09:22:54.278 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:54.343 INFO:teuthology.orchestra.run.trial151.stdout:Package python3-rbd-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:54.362 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:54.362 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:54.363 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:54.384 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install rbd-fuse 2026-02-25T09:22:54.703 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:54.768 INFO:teuthology.orchestra.run.trial151.stdout:Package rbd-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:54.787 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:54.788 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:54.788 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:54.809 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install rbd-mirror 2026-02-25T09:22:55.126 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:12 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:55.191 INFO:teuthology.orchestra.run.trial151.stdout:Package rbd-mirror-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:55.210 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:55.211 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:55.211 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:55.233 DEBUG:teuthology.orchestra.run.trial151:> sudo yum -y install rbd-nbd 2026-02-25T09:22:55.555 INFO:teuthology.orchestra.run.trial151.stdout:Last metadata expiration check: 0:00:12 ago on Wed 25 Feb 2026 09:22:43 AM UTC. 2026-02-25T09:22:55.621 INFO:teuthology.orchestra.run.trial151.stdout:Package rbd-nbd-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:22:55.639 INFO:teuthology.orchestra.run.trial151.stdout:Dependencies resolved. 2026-02-25T09:22:55.640 INFO:teuthology.orchestra.run.trial151.stdout:Nothing to do. 2026-02-25T09:22:55.640 INFO:teuthology.orchestra.run.trial151.stdout:Complete! 2026-02-25T09:22:55.664 DEBUG:teuthology.parallel:result is None 2026-02-25T09:24:51.249 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - CRB 63 kB/s | 8.0 MB 02:10 2026-02-25T09:24:52.169 INFO:teuthology.orchestra.run.trial094.stdout:CentOS Stream 9 - Extras packages 114 kB/s | 20 kB 00:00 2026-02-25T09:24:52.915 INFO:teuthology.orchestra.run.trial094.stdout:Extra Packages for Enterprise Linux 30 MB/s | 20 MB 00:00 2026-02-25T09:24:56.116 INFO:teuthology.orchestra.run.trial094.stdout:lab-extras 2.7 MB/s | 50 kB 00:00 2026-02-25T09:24:57.309 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-radosgw-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:24:57.328 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:24:57.329 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:24:57.329 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:24:57.352 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-test 2026-02-25T09:24:57.676 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:01 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:24:57.743 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-test-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:24:57.761 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:24:57.762 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:24:57.762 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:24:57.784 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph 2026-02-25T09:24:58.101 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:24:58.167 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:24:58.186 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:24:58.187 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:24:58.187 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:24:58.209 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-base 2026-02-25T09:24:58.526 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:24:58.592 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-base-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:24:58.611 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:24:58.611 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:24:58.612 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:24:58.633 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install cephadm 2026-02-25T09:24:58.952 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:02 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:24:59.018 INFO:teuthology.orchestra.run.trial094.stdout:Package cephadm-2:20.3.0-5536.ge6eec768.el9.noarch is already installed. 2026-02-25T09:24:59.037 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:24:59.038 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:24:59.038 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:24:59.059 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-immutable-object-cache 2026-02-25T09:24:59.377 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:24:59.443 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-immutable-object-cache-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:24:59.462 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:24:59.463 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:24:59.463 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:24:59.484 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mgr 2026-02-25T09:24:59.796 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:03 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:24:59.861 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-mgr-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:24:59.880 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:24:59.881 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:24:59.881 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:24:59.902 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mgr-dashboard 2026-02-25T09:25:00.218 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:04 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:00.285 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-mgr-dashboard-2:20.3.0-5536.ge6eec768.el9.noarch is already installed. 2026-02-25T09:25:00.304 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:00.305 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:00.305 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:00.327 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mgr-diskprediction-local 2026-02-25T09:25:00.643 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:04 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:00.709 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-mgr-diskprediction-local-2:20.3.0-5536.ge6eec768.el9.noarch is already installed. 2026-02-25T09:25:00.728 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:00.728 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:00.728 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:00.750 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mgr-rook 2026-02-25T09:25:01.063 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:01.128 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-mgr-rook-2:20.3.0-5536.ge6eec768.el9.noarch is already installed. 2026-02-25T09:25:01.147 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:01.148 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:01.148 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:01.169 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-mgr-cephadm 2026-02-25T09:25:01.489 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:01.555 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-mgr-cephadm-2:20.3.0-5536.ge6eec768.el9.noarch is already installed. 2026-02-25T09:25:01.574 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:01.574 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:01.574 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:01.597 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-osd 2026-02-25T09:25:01.914 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:05 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:01.978 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-osd-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:25:01.997 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:01.998 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:01.998 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:02.019 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-osd-classic 2026-02-25T09:25:02.334 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:06 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:02.399 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-osd-classic-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:25:02.418 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:02.418 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:02.418 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:02.440 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-fuse 2026-02-25T09:25:02.756 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:06 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:02.822 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:25:02.841 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:02.842 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:02.842 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:02.864 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install ceph-volume 2026-02-25T09:25:03.186 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:07 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:03.252 INFO:teuthology.orchestra.run.trial094.stdout:Package ceph-volume-2:20.3.0-5536.ge6eec768.el9.noarch is already installed. 2026-02-25T09:25:03.271 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:03.271 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:03.271 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:03.293 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install librados-devel 2026-02-25T09:25:03.616 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:07 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:03.683 INFO:teuthology.orchestra.run.trial094.stdout:Package librados-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:25:03.702 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:03.702 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:03.702 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:03.723 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install libcephfs2 2026-02-25T09:25:04.041 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:04.107 INFO:teuthology.orchestra.run.trial094.stdout:Package libcephfs2-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:25:04.127 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:04.127 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:04.128 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:04.149 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install libcephfs-devel 2026-02-25T09:25:04.471 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:04.536 INFO:teuthology.orchestra.run.trial094.stdout:Package libcephfs-devel-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:25:04.555 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:04.556 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:04.556 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:04.577 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install librados2 2026-02-25T09:25:04.894 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:08 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:04.960 INFO:teuthology.orchestra.run.trial094.stdout:Package librados2-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:25:04.979 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:04.979 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:04.979 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:05.001 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install librbd1 2026-02-25T09:25:05.319 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:05.385 INFO:teuthology.orchestra.run.trial094.stdout:Package librbd1-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:25:05.403 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:05.404 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:05.404 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:05.425 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install python3-rados 2026-02-25T09:25:05.748 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:09 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:05.813 INFO:teuthology.orchestra.run.trial094.stdout:Package python3-rados-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:25:05.832 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:05.833 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:05.833 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:05.854 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install python3-rgw 2026-02-25T09:25:06.179 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:10 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:06.245 INFO:teuthology.orchestra.run.trial094.stdout:Package python3-rgw-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:25:06.264 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:06.265 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:06.265 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:06.287 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install python3-cephfs 2026-02-25T09:25:06.603 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:10 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:06.668 INFO:teuthology.orchestra.run.trial094.stdout:Package python3-cephfs-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:25:06.688 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:06.689 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:06.689 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:06.710 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install python3-rbd 2026-02-25T09:25:07.030 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:07.097 INFO:teuthology.orchestra.run.trial094.stdout:Package python3-rbd-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:25:07.115 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:07.116 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:07.116 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:07.138 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install rbd-fuse 2026-02-25T09:25:07.457 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:07.523 INFO:teuthology.orchestra.run.trial094.stdout:Package rbd-fuse-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:25:07.542 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:07.542 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:07.542 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:07.563 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install rbd-mirror 2026-02-25T09:25:07.886 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:11 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:07.953 INFO:teuthology.orchestra.run.trial094.stdout:Package rbd-mirror-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:25:07.972 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:07.972 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:07.972 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:07.994 DEBUG:teuthology.orchestra.run.trial094:> sudo yum -y install rbd-nbd 2026-02-25T09:25:08.311 INFO:teuthology.orchestra.run.trial094.stdout:Last metadata expiration check: 0:00:12 ago on Wed 25 Feb 2026 09:24:56 AM UTC. 2026-02-25T09:25:08.377 INFO:teuthology.orchestra.run.trial094.stdout:Package rbd-nbd-2:20.3.0-5536.ge6eec768.el9.x86_64 is already installed. 2026-02-25T09:25:08.396 INFO:teuthology.orchestra.run.trial094.stdout:Dependencies resolved. 2026-02-25T09:25:08.397 INFO:teuthology.orchestra.run.trial094.stdout:Nothing to do. 2026-02-25T09:25:08.397 INFO:teuthology.orchestra.run.trial094.stdout:Complete! 2026-02-25T09:25:08.423 DEBUG:teuthology.parallel:result is None 2026-02-25T09:25:08.424 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=e6eec7683022e6824acc16f4c26421a3804e27e6 2026-02-25T09:25:08.594 DEBUG:teuthology.orchestra.run.trial094:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T09:25:08.615 INFO:teuthology.orchestra.run.trial094.stdout:20.3.0-5536.ge6eec768.el9 2026-02-25T09:25:08.615 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5536.ge6eec768.el9 2026-02-25T09:25:08.615 INFO:teuthology.task.install:The correct ceph version 20.3.0-5536.ge6eec768 is installed. 2026-02-25T09:25:08.618 DEBUG:teuthology.packaging:Querying https://shaman.ceph.com/api/search?status=ready&project=ceph&flavor=debug&distros=centos%2F9%2Fx86_64&sha1=e6eec7683022e6824acc16f4c26421a3804e27e6 2026-02-25T09:25:08.764 DEBUG:teuthology.orchestra.run.trial151:> rpm -q ceph --qf '%{VERSION}-%{RELEASE}' 2026-02-25T09:25:08.796 INFO:teuthology.orchestra.run.trial151.stdout:20.3.0-5536.ge6eec768.el9 2026-02-25T09:25:08.796 INFO:teuthology.packaging:The installed version of ceph is 20.3.0-5536.ge6eec768.el9 2026-02-25T09:25:08.796 INFO:teuthology.task.install:The correct ceph version 20.3.0-5536.ge6eec768 is installed. 2026-02-25T09:25:08.799 INFO:teuthology.task.install.util:Utilities already shipped, skip it... 2026-02-25T09:25:08.799 INFO:teuthology.run_tasks:Running task ceph... 2026-02-25T09:25:08.900 INFO:tasks.ceph:Making ceph log dir writeable by non-root... 2026-02-25T09:25:08.900 DEBUG:teuthology.orchestra.run.trial094:> sudo chmod 777 /var/log/ceph 2026-02-25T09:25:08.902 DEBUG:teuthology.orchestra.run.trial151:> sudo chmod 777 /var/log/ceph 2026-02-25T09:25:08.927 INFO:tasks.ceph:Disabling ceph logrotate... 2026-02-25T09:25:08.927 DEBUG:teuthology.orchestra.run.trial094:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-25T09:25:08.967 DEBUG:teuthology.orchestra.run.trial151:> sudo rm -f -- /etc/logrotate.d/ceph 2026-02-25T09:25:08.989 INFO:tasks.ceph:Creating extra log directories... 2026-02-25T09:25:08.990 DEBUG:teuthology.orchestra.run.trial094:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-25T09:25:09.030 DEBUG:teuthology.orchestra.run.trial151:> sudo install -d -m0777 -- /var/log/ceph/valgrind /var/log/ceph/profiling-logger 2026-02-25T09:25:09.069 INFO:tasks.ceph:Creating ceph cluster ceph... 2026-02-25T09:25:09.070 INFO:tasks.ceph:config {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug objclass': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'fs': 'xfs', 'mkfs_options': None, 'mount_options': None, 'skip_mgr_daemons': False, 'log_ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'cpu_profile': set(), 'cluster': 'ceph', 'mon_bind_msgr2': True, 'mon_bind_addrvec': True} 2026-02-25T09:25:09.070 INFO:tasks.ceph:ctx.config {'archive_path': '/home/teuthworker/mnt/teuthology/matan-2026-02-25_07:58:25-crimson-rados-main-distro-debug-trial/69215', 'branch': 'main', 'description': 'crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/cls}', 'email': None, 'first_in_suite': False, 'flavor': 'debug', 'job_id': '69215', 'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}, 'last_in_suite': False, 'machine_type': 'trial', 'name': 'matan-2026-02-25_07:58:25-crimson-rados-main-distro-debug-trial', 'no_nested_subset': False, 'os_type': 'centos', 'os_version': '9.stream', 'overrides': {'admin_socket': {'branch': 'main'}, 'ceph': {'conf': {'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}, 'mon': {'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug objclass': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'e6eec7683022e6824acc16f4c26421a3804e27e6'}, 'ceph-deploy': {'conf': {'client': {'log file': '/var/log/ceph/ceph-$name.$pid.log'}, 'mon': {}}}, 'install': {'ceph': {'flavor': 'debug', 'sha1': 'e6eec7683022e6824acc16f4c26421a3804e27e6'}}, 'workunit': {'branch': 'main', 'env': {'CRIMSON_COMPAT': '1'}, 'sha1': 'c32469671c6b0222f786dd36c34eb702f780a11d'}}, 'owner': 'scheduled_matan@soko04.front.sepia.ceph.com', 'priority': 50, 'repo': 'https://git.ceph.com/ceph-ci.git', 'roles': [['mon.a', 'osd.0', 'osd.1', 'client.0', 'node-exporter.a'], ['mgr.x', 'osd.2', 'osd.3', 'client.1', 'prometheus.a', 'node-exporter.b']], 'seed': 6481, 'sha1': 'e6eec7683022e6824acc16f4c26421a3804e27e6', 'sleep_before_teardown': 0, 'suite': 'crimson-rados', 'suite_branch': 'main', 'suite_path': '/home/teuthworker/src/github.com_ceph_ceph_c32469671c6b0222f786dd36c34eb702f780a11d/qa', 'suite_relpath': 'qa', 'suite_repo': 'https://github.com/ceph/ceph.git', 'suite_sha1': 'c32469671c6b0222f786dd36c34eb702f780a11d', 'targets': {'trial094.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBNItKcM4F54n75w9bYtOy+mNooHfRZ0e9nBYaGEz+Fwne9dqiisMQDaPCAKfnA3EU8Wwe796Ug5dwYqxYwGlPOs=', 'trial151.front.sepia.ceph.com': 'ecdsa-sha2-nistp256 AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBEWZ+lNTu2ibpKKgOhd/cEcvkOJ9586tIGMTc7D50i+WUZai6cS4c4HmCNcwWvmtIz8PpHRs1az9tjS00k1s+iw='}, 'tasks': [{'internal.check_packages': None}, {'internal.buildpackages_prep': None}, {'internal.save_config': None}, {'internal.check_lock': None}, {'internal.add_remotes': None}, {'console_log': None}, {'internal.connect': None}, {'internal.push_inventory': None}, {'internal.serialize_remote_roles': None}, {'internal.check_conflict': None}, {'internal.check_ceph_data': None}, {'internal.vm_setup': None}, {'kernel': {'branch': 'distro', 'kdb': 1, 'sha1': 'distro'}}, {'internal.base': None}, {'internal.archive_upload': None}, {'internal.archive': None}, {'internal.coredump': None}, {'internal.sudo': None}, {'internal.syslog': None}, {'internal.timer': None}, {'pcp': None}, {'selinux': None}, {'ansible.cephlab': None}, {'clock': None}, {'install': {'extra_packages': {'deb': ['ceph-osd-crimson', 'ceph-osd-crimson-dbg'], 'rpm': ['ceph-osd-crimson', 'ceph-osd-crimson-debuginfo']}, 'flavor': 'debug', 'sha1': 'e6eec7683022e6824acc16f4c26421a3804e27e6'}}, {'pexec': {'all': ['sudo update-alternatives --set ceph-osd /usr/bin/ceph-osd-crimson']}}, {'install': None}, {'ceph': {'conf': {'mon': {'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'debug mon': 20, 'debug ms': 1, 'debug paxos': 20, 'osd pool default crimson': True}, 'osd': {'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug objclass': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'global': {'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'mgr': {'debug mgr': 20, 'debug ms': 1}}, 'crimson_compat': True, 'flavor': 'debug', 'log-ignorelist': ['\\(MDS_ALL_DOWN\\)', '\\(MDS_UP_LESS_THAN_MAX\\)', '\\(POOL_APP_NOT_ENABLED\\)'], 'sha1': 'e6eec7683022e6824acc16f4c26421a3804e27e6', 'cluster': 'ceph'}}, {'workunit': {'clients': {'client.0': ['cls/test_cls_cmpomap.sh']}}}], 'teuthology': {'fragments_dropped': [], 'meta': {}, 'postmerge': []}, 'teuthology_branch': 'main', 'teuthology_sha1': 'fa17720d0088c3ac28e473468bfc79eeaff5cd38', 'timestamp': '2026-02-25_07:58:25', 'tube': 'trial', 'user': 'matan', 'verbose': False, 'worker_log': '/home/teuthworker/mnt/teuthology/worker_logs/dispatcher.trial.904699'} 2026-02-25T09:25:09.071 DEBUG:teuthology.orchestra.run.trial094:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-25T09:25:09.095 DEBUG:teuthology.orchestra.run.trial151:> install -d -m0755 -- /home/ubuntu/cephtest/ceph.data 2026-02-25T09:25:09.113 DEBUG:teuthology.orchestra.run.trial094:> sudo install -d -m0777 -- /var/run/ceph 2026-02-25T09:25:09.150 DEBUG:teuthology.orchestra.run.trial151:> sudo install -d -m0777 -- /var/run/ceph 2026-02-25T09:25:09.177 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:25:09.177 DEBUG:teuthology.orchestra.run.trial094:> dd if=/scratch_devs of=/dev/stdout 2026-02-25T09:25:09.229 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-25T09:25:09.229 DEBUG:teuthology.orchestra.run.trial094:> stat /dev/vg_nvme/lv_1 2026-02-25T09:25:09.285 INFO:teuthology.orchestra.run.trial094.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-25T09:25:09.285 INFO:teuthology.orchestra.run.trial094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T09:25:09.285 INFO:teuthology.orchestra.run.trial094.stdout:Device: 6h/6d Inode: 915 Links: 1 2026-02-25T09:25:09.285 INFO:teuthology.orchestra.run.trial094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T09:25:09.285 INFO:teuthology.orchestra.run.trial094.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T09:25:09.285 INFO:teuthology.orchestra.run.trial094.stdout:Access: 2026-02-25 09:18:08.247605798 +0000 2026-02-25T09:25:09.285 INFO:teuthology.orchestra.run.trial094.stdout:Modify: 2026-02-25 09:11:37.905706007 +0000 2026-02-25T09:25:09.285 INFO:teuthology.orchestra.run.trial094.stdout:Change: 2026-02-25 09:11:37.905706007 +0000 2026-02-25T09:25:09.286 INFO:teuthology.orchestra.run.trial094.stdout: Birth: 2026-02-25 09:11:37.905706007 +0000 2026-02-25T09:25:09.286 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-25T09:25:09.348 INFO:teuthology.orchestra.run.trial094.stderr:1+0 records in 2026-02-25T09:25:09.348 INFO:teuthology.orchestra.run.trial094.stderr:1+0 records out 2026-02-25T09:25:09.348 INFO:teuthology.orchestra.run.trial094.stderr:512 bytes copied, 0.000155067 s, 3.3 MB/s 2026-02-25T09:25:09.349 DEBUG:teuthology.orchestra.run.trial094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-25T09:25:09.406 DEBUG:teuthology.orchestra.run.trial094:> stat /dev/vg_nvme/lv_2 2026-02-25T09:25:09.462 INFO:teuthology.orchestra.run.trial094.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-25T09:25:09.463 INFO:teuthology.orchestra.run.trial094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T09:25:09.463 INFO:teuthology.orchestra.run.trial094.stdout:Device: 6h/6d Inode: 943 Links: 1 2026-02-25T09:25:09.463 INFO:teuthology.orchestra.run.trial094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T09:25:09.463 INFO:teuthology.orchestra.run.trial094.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T09:25:09.463 INFO:teuthology.orchestra.run.trial094.stdout:Access: 2026-02-25 09:18:08.247605798 +0000 2026-02-25T09:25:09.463 INFO:teuthology.orchestra.run.trial094.stdout:Modify: 2026-02-25 09:11:38.137697195 +0000 2026-02-25T09:25:09.463 INFO:teuthology.orchestra.run.trial094.stdout:Change: 2026-02-25 09:11:38.137697195 +0000 2026-02-25T09:25:09.463 INFO:teuthology.orchestra.run.trial094.stdout: Birth: 2026-02-25 09:11:38.137697195 +0000 2026-02-25T09:25:09.464 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-25T09:25:09.525 INFO:teuthology.orchestra.run.trial094.stderr:1+0 records in 2026-02-25T09:25:09.525 INFO:teuthology.orchestra.run.trial094.stderr:1+0 records out 2026-02-25T09:25:09.525 INFO:teuthology.orchestra.run.trial094.stderr:512 bytes copied, 0.00014551 s, 3.5 MB/s 2026-02-25T09:25:09.526 DEBUG:teuthology.orchestra.run.trial094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-25T09:25:09.583 DEBUG:teuthology.orchestra.run.trial094:> stat /dev/vg_nvme/lv_3 2026-02-25T09:25:09.639 INFO:teuthology.orchestra.run.trial094.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-25T09:25:09.639 INFO:teuthology.orchestra.run.trial094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T09:25:09.639 INFO:teuthology.orchestra.run.trial094.stdout:Device: 6h/6d Inode: 973 Links: 1 2026-02-25T09:25:09.640 INFO:teuthology.orchestra.run.trial094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T09:25:09.640 INFO:teuthology.orchestra.run.trial094.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T09:25:09.640 INFO:teuthology.orchestra.run.trial094.stdout:Access: 2026-02-25 09:18:08.247605798 +0000 2026-02-25T09:25:09.640 INFO:teuthology.orchestra.run.trial094.stdout:Modify: 2026-02-25 09:11:38.392687511 +0000 2026-02-25T09:25:09.640 INFO:teuthology.orchestra.run.trial094.stdout:Change: 2026-02-25 09:11:38.392687511 +0000 2026-02-25T09:25:09.640 INFO:teuthology.orchestra.run.trial094.stdout: Birth: 2026-02-25 09:11:38.392687511 +0000 2026-02-25T09:25:09.640 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-25T09:25:09.703 INFO:teuthology.orchestra.run.trial094.stderr:1+0 records in 2026-02-25T09:25:09.703 INFO:teuthology.orchestra.run.trial094.stderr:1+0 records out 2026-02-25T09:25:09.703 INFO:teuthology.orchestra.run.trial094.stderr:512 bytes copied, 0.00014571 s, 3.5 MB/s 2026-02-25T09:25:09.705 DEBUG:teuthology.orchestra.run.trial094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-25T09:25:09.761 DEBUG:teuthology.orchestra.run.trial094:> stat /dev/vg_nvme/lv_4 2026-02-25T09:25:09.818 INFO:teuthology.orchestra.run.trial094.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-25T09:25:09.818 INFO:teuthology.orchestra.run.trial094.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T09:25:09.818 INFO:teuthology.orchestra.run.trial094.stdout:Device: 6h/6d Inode: 1000 Links: 1 2026-02-25T09:25:09.818 INFO:teuthology.orchestra.run.trial094.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T09:25:09.819 INFO:teuthology.orchestra.run.trial094.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T09:25:09.819 INFO:teuthology.orchestra.run.trial094.stdout:Access: 2026-02-25 09:18:08.248605756 +0000 2026-02-25T09:25:09.819 INFO:teuthology.orchestra.run.trial094.stdout:Modify: 2026-02-25 09:11:38.628678548 +0000 2026-02-25T09:25:09.819 INFO:teuthology.orchestra.run.trial094.stdout:Change: 2026-02-25 09:11:38.628678548 +0000 2026-02-25T09:25:09.819 INFO:teuthology.orchestra.run.trial094.stdout: Birth: 2026-02-25 09:11:38.628678548 +0000 2026-02-25T09:25:09.819 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-25T09:25:09.881 INFO:teuthology.orchestra.run.trial094.stderr:1+0 records in 2026-02-25T09:25:09.881 INFO:teuthology.orchestra.run.trial094.stderr:1+0 records out 2026-02-25T09:25:09.881 INFO:teuthology.orchestra.run.trial094.stderr:512 bytes copied, 0.00014064 s, 3.6 MB/s 2026-02-25T09:25:09.882 DEBUG:teuthology.orchestra.run.trial094:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-25T09:25:09.939 INFO:tasks.ceph:osd dev map: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-25T09:25:09.939 DEBUG:teuthology.orchestra.run.trial151:> set -ex 2026-02-25T09:25:09.939 DEBUG:teuthology.orchestra.run.trial151:> dd if=/scratch_devs of=/dev/stdout 2026-02-25T09:25:09.957 DEBUG:teuthology.misc:devs=['/dev/vg_nvme/lv_1', '/dev/vg_nvme/lv_2', '/dev/vg_nvme/lv_3', '/dev/vg_nvme/lv_4'] 2026-02-25T09:25:09.958 DEBUG:teuthology.orchestra.run.trial151:> stat /dev/vg_nvme/lv_1 2026-02-25T09:25:10.014 INFO:teuthology.orchestra.run.trial151.stdout: File: /dev/vg_nvme/lv_1 -> ../dm-0 2026-02-25T09:25:10.014 INFO:teuthology.orchestra.run.trial151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T09:25:10.014 INFO:teuthology.orchestra.run.trial151.stdout:Device: 6h/6d Inode: 902 Links: 1 2026-02-25T09:25:10.014 INFO:teuthology.orchestra.run.trial151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T09:25:10.014 INFO:teuthology.orchestra.run.trial151.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T09:25:10.015 INFO:teuthology.orchestra.run.trial151.stdout:Access: 2026-02-25 09:17:38.410885907 +0000 2026-02-25T09:25:10.015 INFO:teuthology.orchestra.run.trial151.stdout:Modify: 2026-02-25 09:11:40.499193268 +0000 2026-02-25T09:25:10.015 INFO:teuthology.orchestra.run.trial151.stdout:Change: 2026-02-25 09:11:40.499193268 +0000 2026-02-25T09:25:10.015 INFO:teuthology.orchestra.run.trial151.stdout: Birth: 2026-02-25 09:11:40.499193268 +0000 2026-02-25T09:25:10.015 DEBUG:teuthology.orchestra.run.trial151:> sudo dd if=/dev/vg_nvme/lv_1 of=/dev/null count=1 2026-02-25T09:25:10.076 INFO:teuthology.orchestra.run.trial151.stderr:1+0 records in 2026-02-25T09:25:10.076 INFO:teuthology.orchestra.run.trial151.stderr:1+0 records out 2026-02-25T09:25:10.076 INFO:teuthology.orchestra.run.trial151.stderr:512 bytes copied, 0.000184555 s, 2.8 MB/s 2026-02-25T09:25:10.077 DEBUG:teuthology.orchestra.run.trial151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_1 2026-02-25T09:25:10.136 DEBUG:teuthology.orchestra.run.trial151:> stat /dev/vg_nvme/lv_2 2026-02-25T09:25:10.192 INFO:teuthology.orchestra.run.trial151.stdout: File: /dev/vg_nvme/lv_2 -> ../dm-1 2026-02-25T09:25:10.192 INFO:teuthology.orchestra.run.trial151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T09:25:10.192 INFO:teuthology.orchestra.run.trial151.stdout:Device: 6h/6d Inode: 930 Links: 1 2026-02-25T09:25:10.192 INFO:teuthology.orchestra.run.trial151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T09:25:10.192 INFO:teuthology.orchestra.run.trial151.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T09:25:10.193 INFO:teuthology.orchestra.run.trial151.stdout:Access: 2026-02-25 09:17:38.410885907 +0000 2026-02-25T09:25:10.193 INFO:teuthology.orchestra.run.trial151.stdout:Modify: 2026-02-25 09:11:40.742189827 +0000 2026-02-25T09:25:10.193 INFO:teuthology.orchestra.run.trial151.stdout:Change: 2026-02-25 09:11:40.742189827 +0000 2026-02-25T09:25:10.193 INFO:teuthology.orchestra.run.trial151.stdout: Birth: 2026-02-25 09:11:40.742189827 +0000 2026-02-25T09:25:10.193 DEBUG:teuthology.orchestra.run.trial151:> sudo dd if=/dev/vg_nvme/lv_2 of=/dev/null count=1 2026-02-25T09:25:10.255 INFO:teuthology.orchestra.run.trial151.stderr:1+0 records in 2026-02-25T09:25:10.255 INFO:teuthology.orchestra.run.trial151.stderr:1+0 records out 2026-02-25T09:25:10.256 INFO:teuthology.orchestra.run.trial151.stderr:512 bytes copied, 0.000155752 s, 3.3 MB/s 2026-02-25T09:25:10.257 DEBUG:teuthology.orchestra.run.trial151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_2 2026-02-25T09:25:10.314 DEBUG:teuthology.orchestra.run.trial151:> stat /dev/vg_nvme/lv_3 2026-02-25T09:25:10.370 INFO:teuthology.orchestra.run.trial151.stdout: File: /dev/vg_nvme/lv_3 -> ../dm-2 2026-02-25T09:25:10.371 INFO:teuthology.orchestra.run.trial151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T09:25:10.371 INFO:teuthology.orchestra.run.trial151.stdout:Device: 6h/6d Inode: 957 Links: 1 2026-02-25T09:25:10.371 INFO:teuthology.orchestra.run.trial151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T09:25:10.371 INFO:teuthology.orchestra.run.trial151.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T09:25:10.371 INFO:teuthology.orchestra.run.trial151.stdout:Access: 2026-02-25 09:17:38.411885896 +0000 2026-02-25T09:25:10.371 INFO:teuthology.orchestra.run.trial151.stdout:Modify: 2026-02-25 09:11:41.001186161 +0000 2026-02-25T09:25:10.371 INFO:teuthology.orchestra.run.trial151.stdout:Change: 2026-02-25 09:11:41.001186161 +0000 2026-02-25T09:25:10.371 INFO:teuthology.orchestra.run.trial151.stdout: Birth: 2026-02-25 09:11:41.001186161 +0000 2026-02-25T09:25:10.372 DEBUG:teuthology.orchestra.run.trial151:> sudo dd if=/dev/vg_nvme/lv_3 of=/dev/null count=1 2026-02-25T09:25:10.433 INFO:teuthology.orchestra.run.trial151.stderr:1+0 records in 2026-02-25T09:25:10.434 INFO:teuthology.orchestra.run.trial151.stderr:1+0 records out 2026-02-25T09:25:10.434 INFO:teuthology.orchestra.run.trial151.stderr:512 bytes copied, 0.000144581 s, 3.5 MB/s 2026-02-25T09:25:10.435 DEBUG:teuthology.orchestra.run.trial151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_3 2026-02-25T09:25:10.492 DEBUG:teuthology.orchestra.run.trial151:> stat /dev/vg_nvme/lv_4 2026-02-25T09:25:10.550 INFO:teuthology.orchestra.run.trial151.stdout: File: /dev/vg_nvme/lv_4 -> ../dm-3 2026-02-25T09:25:10.550 INFO:teuthology.orchestra.run.trial151.stdout: Size: 7 Blocks: 0 IO Block: 4096 symbolic link 2026-02-25T09:25:10.550 INFO:teuthology.orchestra.run.trial151.stdout:Device: 6h/6d Inode: 985 Links: 1 2026-02-25T09:25:10.550 INFO:teuthology.orchestra.run.trial151.stdout:Access: (0777/lrwxrwxrwx) Uid: ( 0/ root) Gid: ( 0/ root) 2026-02-25T09:25:10.550 INFO:teuthology.orchestra.run.trial151.stdout:Context: system_u:object_r:device_t:s0 2026-02-25T09:25:10.550 INFO:teuthology.orchestra.run.trial151.stdout:Access: 2026-02-25 09:17:38.411885896 +0000 2026-02-25T09:25:10.550 INFO:teuthology.orchestra.run.trial151.stdout:Modify: 2026-02-25 09:11:41.261182483 +0000 2026-02-25T09:25:10.550 INFO:teuthology.orchestra.run.trial151.stdout:Change: 2026-02-25 09:11:41.261182483 +0000 2026-02-25T09:25:10.550 INFO:teuthology.orchestra.run.trial151.stdout: Birth: 2026-02-25 09:11:41.261182483 +0000 2026-02-25T09:25:10.551 DEBUG:teuthology.orchestra.run.trial151:> sudo dd if=/dev/vg_nvme/lv_4 of=/dev/null count=1 2026-02-25T09:25:10.613 INFO:teuthology.orchestra.run.trial151.stderr:1+0 records in 2026-02-25T09:25:10.613 INFO:teuthology.orchestra.run.trial151.stderr:1+0 records out 2026-02-25T09:25:10.613 INFO:teuthology.orchestra.run.trial151.stderr:512 bytes copied, 0.000142356 s, 3.6 MB/s 2026-02-25T09:25:10.614 DEBUG:teuthology.orchestra.run.trial151:> ! mount | grep -v devtmpfs | grep -q /dev/vg_nvme/lv_4 2026-02-25T09:25:10.672 INFO:tasks.ceph:osd dev map: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-25T09:25:10.672 INFO:tasks.ceph:remote_to_roles_to_devs: {Remote(name='ubuntu@trial094.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@trial151.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2026-02-25T09:25:10.672 INFO:tasks.ceph:Generating config... 2026-02-25T09:25:10.674 INFO:tasks.ceph:[mon] mon min osdmap epochs = 50 2026-02-25T09:25:10.674 INFO:tasks.ceph:[mon] mon osdmap full prune interval = 2 2026-02-25T09:25:10.675 INFO:tasks.ceph:[mon] mon osdmap full prune min = 15 2026-02-25T09:25:10.675 INFO:tasks.ceph:[mon] mon osdmap full prune txsize = 2 2026-02-25T09:25:10.675 INFO:tasks.ceph:[mon] paxos service trim min = 10 2026-02-25T09:25:10.675 INFO:tasks.ceph:[mon] debug mon = 20 2026-02-25T09:25:10.675 INFO:tasks.ceph:[mon] debug ms = 1 2026-02-25T09:25:10.675 INFO:tasks.ceph:[mon] debug paxos = 20 2026-02-25T09:25:10.675 INFO:tasks.ceph:[mon] osd pool default crimson = True 2026-02-25T09:25:10.675 INFO:tasks.ceph:[osd] debug monc = 20 2026-02-25T09:25:10.675 INFO:tasks.ceph:[osd] crimson cpu num = 4 2026-02-25T09:25:10.675 INFO:tasks.ceph:[osd] crimson osd obc lru size = 10 2026-02-25T09:25:10.675 INFO:tasks.ceph:[osd] debug ms = 20 2026-02-25T09:25:10.676 INFO:tasks.ceph:[osd] debug objclass = 20 2026-02-25T09:25:10.676 INFO:tasks.ceph:[osd] debug osd = 20 2026-02-25T09:25:10.676 INFO:tasks.ceph:[osd] debug seastore = 20 2026-02-25T09:25:10.676 INFO:tasks.ceph:[osd] debug seastore backref = 20 2026-02-25T09:25:10.676 INFO:tasks.ceph:[osd] debug seastore cache = 20 2026-02-25T09:25:10.676 INFO:tasks.ceph:[osd] debug seastore cleaner = 20 2026-02-25T09:25:10.676 INFO:tasks.ceph:[osd] debug seastore device = 20 2026-02-25T09:25:10.676 INFO:tasks.ceph:[osd] debug seastore epm = 20 2026-02-25T09:25:10.676 INFO:tasks.ceph:[osd] debug seastore fixedkv tree = 20 2026-02-25T09:25:10.676 INFO:tasks.ceph:[osd] debug seastore journal = 20 2026-02-25T09:25:10.676 INFO:tasks.ceph:[osd] debug seastore lba = 20 2026-02-25T09:25:10.676 INFO:tasks.ceph:[osd] debug seastore odata = 20 2026-02-25T09:25:10.676 INFO:tasks.ceph:[osd] debug seastore omap = 20 2026-02-25T09:25:10.677 INFO:tasks.ceph:[osd] debug seastore onode = 20 2026-02-25T09:25:10.677 INFO:tasks.ceph:[osd] debug seastore t = 20 2026-02-25T09:25:10.677 INFO:tasks.ceph:[osd] debug seastore tm = 20 2026-02-25T09:25:10.677 INFO:tasks.ceph:[osd] osd objectstore = seastore 2026-02-25T09:25:10.677 INFO:tasks.ceph:[osd] osd shutdown pgref assert = True 2026-02-25T09:25:10.677 INFO:tasks.ceph:[osd] seastore cachepin size pershard = 64M 2026-02-25T09:25:10.677 INFO:tasks.ceph:[osd] seastore cachepin type = 2Q 2026-02-25T09:25:10.677 INFO:tasks.ceph:[osd] seastore max concurrent transactions = 8 2026-02-25T09:25:10.677 INFO:tasks.ceph:[global] enable experimental unrecoverable data corrupting features = crimson 2026-02-25T09:25:10.677 INFO:tasks.ceph:[global] ms client mode = crc 2026-02-25T09:25:10.677 INFO:tasks.ceph:[global] ms cluster mode = crc 2026-02-25T09:25:10.677 INFO:tasks.ceph:[global] ms mon client mode = crc 2026-02-25T09:25:10.678 INFO:tasks.ceph:[global] ms mon cluster mode = crc 2026-02-25T09:25:10.678 INFO:tasks.ceph:[global] ms mon service mode = crc 2026-02-25T09:25:10.678 INFO:tasks.ceph:[global] ms service mode = crc 2026-02-25T09:25:10.678 INFO:tasks.ceph:[mgr] debug mgr = 20 2026-02-25T09:25:10.678 INFO:tasks.ceph:[mgr] debug ms = 1 2026-02-25T09:25:10.678 INFO:tasks.ceph:Setting up mon.a... 2026-02-25T09:25:10.678 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring /etc/ceph/ceph.keyring 2026-02-25T09:25:10.721 INFO:teuthology.orchestra.run.trial094.stdout:creating /etc/ceph/ceph.keyring 2026-02-25T09:25:10.724 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=mon. /etc/ceph/ceph.keyring 2026-02-25T09:25:10.808 DEBUG:teuthology.orchestra.run.trial094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-25T09:25:10.872 DEBUG:tasks.ceph:Ceph mon addresses: [('mon.a', '10.20.193.94')] 2026-02-25T09:25:10.873 DEBUG:tasks.ceph:writing out conf {'global': {'chdir': '', 'pid file': '/var/run/ceph/$cluster-$name.pid', 'auth supported': 'cephx', 'filestore xattr use omap': 'true', 'mon clock drift allowed': '1.000', 'osd crush chooseleaf type': '0', 'auth debug': 'true', 'ms die on old message': 'true', 'ms die on bug': 'true', 'mon max pg per osd': '10000', 'mon pg warn max object skew': '0', 'osd_pool_default_pg_autoscale_mode': 'off', 'osd pool default size': '2', 'mon osd allow primary affinity': 'true', 'mon osd allow pg remap': 'true', 'mon warn on legacy crush tunables': 'false', 'mon warn on crush straw calc version zero': 'false', 'mon warn on no sortbitwise': 'false', 'mon warn on osd down out interval zero': 'false', 'mon warn on too few osds': 'false', 'mon_warn_on_pool_pg_num_not_power_of_two': 'false', 'mon_warn_on_pool_no_redundancy': 'false', 'mon_allow_pool_size_one': 'true', 'osd pool default erasure code profile': 'plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd', 'osd default data pool replay window': '5', 'mon allow pool delete': 'true', 'mon cluster log file level': 'debug', 'debug asserts on shutdown': 'true', 'mon health detail to clog': 'false', 'mon host': '10.20.193.94', 'enable experimental unrecoverable data corrupting features': 'crimson', 'ms client mode': 'crc', 'ms cluster mode': 'crc', 'ms mon client mode': 'crc', 'ms mon cluster mode': 'crc', 'ms mon service mode': 'crc', 'ms service mode': 'crc'}, 'osd': {'osd journal size': '100', 'osd scrub load threshold': '5.0', 'osd scrub max interval': '600', 'osd mclock profile': 'high_recovery_ops', 'osd mclock skip benchmark': 'true', 'osd recover clone overlap': 'true', 'osd recovery max chunk': '1048576', 'osd debug shutdown': 'true', 'osd debug op order': 'true', 'osd debug verify stray on activate': 'true', 'osd debug trim objects': 'true', 'osd open classes on start': 'true', 'osd debug pg log writeout': 'true', 'osd deep scrub update digest min age': '30', 'osd map max advance': '10', 'journal zero on create': 'true', 'filestore ondisk finisher threads': '3', 'filestore apply finisher threads': '3', 'bdev debug aio': 'true', 'osd debug misdirected ops': 'true', 'debug monc': 20, 'crimson cpu num': 4, 'crimson osd obc lru size': 10, 'debug ms': 20, 'debug objclass': 20, 'debug osd': 20, 'debug seastore': 20, 'debug seastore backref': 20, 'debug seastore cache': 20, 'debug seastore cleaner': 20, 'debug seastore device': 20, 'debug seastore epm': 20, 'debug seastore fixedkv tree': 20, 'debug seastore journal': 20, 'debug seastore lba': 20, 'debug seastore odata': 20, 'debug seastore omap': 20, 'debug seastore onode': 20, 'debug seastore t': 20, 'debug seastore tm': 20, 'osd objectstore': 'seastore', 'osd shutdown pgref assert': True, 'seastore cachepin size pershard': '64M', 'seastore cachepin type': '2Q', 'seastore max concurrent transactions': 8}, 'mgr': {'debug ms': 1, 'debug mgr': 20, 'debug mon': '20', 'debug auth': '20', 'mon reweight min pgs per osd': '4', 'mon reweight min bytes per osd': '10', 'mgr/telemetry/nag': 'false'}, 'mon': {'debug ms': 1, 'debug mon': 20, 'debug paxos': 20, 'debug auth': '20', 'mon data avail warn': '5', 'mon mgr mkfs grace': '240', 'mon reweight min pgs per osd': '4', 'mon osd reporter subtree level': 'osd', 'mon osd prime pg temp': 'true', 'mon reweight min bytes per osd': '10', 'auth mon ticket ttl': '660', 'auth service ticket ttl': '240', 'mon_warn_on_insecure_global_id_reclaim': 'false', 'mon_warn_on_insecure_global_id_reclaim_allowed': 'false', 'mon_down_mkfs_grace': '2m', 'mon_warn_on_filestore_osds': 'false', 'mon min osdmap epochs': 50, 'mon osdmap full prune interval': 2, 'mon osdmap full prune min': 15, 'mon osdmap full prune txsize': 2, 'paxos service trim min': 10, 'osd pool default crimson': True}, 'client': {'rgw cache enabled': 'true', 'rgw enable ops log': 'true', 'rgw enable usage log': 'true', 'log file': '/var/log/ceph/$cluster-$name.$pid.log', 'admin socket': '/var/run/ceph/$cluster-$name.$pid.asok'}, 'mon.a': {}} 2026-02-25T09:25:10.875 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:25:10.875 DEBUG:teuthology.orchestra.run.trial094:> dd of=/home/ubuntu/cephtest/ceph.tmp.conf 2026-02-25T09:25:10.929 DEBUG:teuthology.orchestra.run.trial094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage monmaptool -c /home/ubuntu/cephtest/ceph.tmp.conf --create --clobber --enable-all-features --add a 10.20.193.94 --print /home/ubuntu/cephtest/ceph.monmap 2026-02-25T09:25:11.007 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:11.006+0000 7f72ec6fdf40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:11.008 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:11.006+0000 7f72ec6fdf40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:11.008 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:11.006+0000 7f72ec6fdf40 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:11.009 INFO:teuthology.orchestra.run.trial094.stdout:monmaptool: monmap file /home/ubuntu/cephtest/ceph.monmap 2026-02-25T09:25:11.009 INFO:teuthology.orchestra.run.trial094.stdout:monmaptool: generated fsid c0375da7-c136-43b7-8240-9e8e7e46c4e5 2026-02-25T09:25:11.009 INFO:teuthology.orchestra.run.trial094.stdout:setting min_mon_release = tentacle 2026-02-25T09:25:11.009 INFO:teuthology.orchestra.run.trial094.stdout:epoch 0 2026-02-25T09:25:11.010 INFO:teuthology.orchestra.run.trial094.stdout:fsid c0375da7-c136-43b7-8240-9e8e7e46c4e5 2026-02-25T09:25:11.010 INFO:teuthology.orchestra.run.trial094.stdout:last_changed 2026-02-25T09:25:11.007470+0000 2026-02-25T09:25:11.010 INFO:teuthology.orchestra.run.trial094.stdout:created 2026-02-25T09:25:11.007470+0000 2026-02-25T09:25:11.010 INFO:teuthology.orchestra.run.trial094.stdout:min_mon_release 20 (tentacle) 2026-02-25T09:25:11.010 INFO:teuthology.orchestra.run.trial094.stdout:election_strategy: 1 2026-02-25T09:25:11.010 INFO:teuthology.orchestra.run.trial094.stdout:0: [v2:10.20.193.94:3300/0,v1:10.20.193.94:6789/0] mon.a 2026-02-25T09:25:11.010 INFO:teuthology.orchestra.run.trial094.stdout:monmaptool: writing epoch 0 to /home/ubuntu/cephtest/ceph.monmap (1 monitors) 2026-02-25T09:25:11.011 DEBUG:teuthology.orchestra.run.trial094:> rm -- /home/ubuntu/cephtest/ceph.tmp.conf 2026-02-25T09:25:11.065 INFO:tasks.ceph:Writing /etc/ceph/ceph.conf for FSID c0375da7-c136-43b7-8240-9e8e7e46c4e5... 2026-02-25T09:25:11.067 DEBUG:teuthology.orchestra.run.trial094:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-25T09:25:11.109 DEBUG:teuthology.orchestra.run.trial151:> sudo mkdir -p /etc/ceph && sudo chmod 0755 /etc/ceph && sudo tee /etc/ceph/ceph.conf && sudo chmod 0644 /etc/ceph/ceph.conf > /dev/null 2026-02-25T09:25:11.145 INFO:teuthology.orchestra.run.trial094.stdout:[global] 2026-02-25T09:25:11.145 INFO:teuthology.orchestra.run.trial094.stdout: chdir = "" 2026-02-25T09:25:11.145 INFO:teuthology.orchestra.run.trial094.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-25T09:25:11.146 INFO:teuthology.orchestra.run.trial094.stdout: auth supported = cephx 2026-02-25T09:25:11.146 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.146 INFO:teuthology.orchestra.run.trial094.stdout: filestore xattr use omap = true 2026-02-25T09:25:11.146 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.146 INFO:teuthology.orchestra.run.trial094.stdout: mon clock drift allowed = 1.000 2026-02-25T09:25:11.146 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.146 INFO:teuthology.orchestra.run.trial094.stdout: osd crush chooseleaf type = 0 2026-02-25T09:25:11.146 INFO:teuthology.orchestra.run.trial094.stdout: auth debug = true 2026-02-25T09:25:11.146 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.146 INFO:teuthology.orchestra.run.trial094.stdout: ms die on old message = true 2026-02-25T09:25:11.146 INFO:teuthology.orchestra.run.trial094.stdout: ms die on bug = true 2026-02-25T09:25:11.146 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.147 INFO:teuthology.orchestra.run.trial094.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-25T09:25:11.147 INFO:teuthology.orchestra.run.trial094.stdout: mon pg warn max object skew = 0 2026-02-25T09:25:11.147 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.147 INFO:teuthology.orchestra.run.trial094.stdout: # disable pg_autoscaler by default for new pools 2026-02-25T09:25:11.147 INFO:teuthology.orchestra.run.trial094.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-25T09:25:11.147 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.147 INFO:teuthology.orchestra.run.trial094.stdout: osd pool default size = 2 2026-02-25T09:25:11.147 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.147 INFO:teuthology.orchestra.run.trial094.stdout: mon osd allow primary affinity = true 2026-02-25T09:25:11.147 INFO:teuthology.orchestra.run.trial094.stdout: mon osd allow pg remap = true 2026-02-25T09:25:11.147 INFO:teuthology.orchestra.run.trial094.stdout: mon warn on legacy crush tunables = false 2026-02-25T09:25:11.148 INFO:teuthology.orchestra.run.trial094.stdout: mon warn on crush straw calc version zero = false 2026-02-25T09:25:11.148 INFO:teuthology.orchestra.run.trial094.stdout: mon warn on no sortbitwise = false 2026-02-25T09:25:11.148 INFO:teuthology.orchestra.run.trial094.stdout: mon warn on osd down out interval zero = false 2026-02-25T09:25:11.148 INFO:teuthology.orchestra.run.trial094.stdout: mon warn on too few osds = false 2026-02-25T09:25:11.148 INFO:teuthology.orchestra.run.trial094.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-25T09:25:11.148 INFO:teuthology.orchestra.run.trial094.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-25T09:25:11.148 INFO:teuthology.orchestra.run.trial094.stdout: mon_allow_pool_size_one = true 2026-02-25T09:25:11.148 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.148 INFO:teuthology.orchestra.run.trial094.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-25T09:25:11.148 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.148 INFO:teuthology.orchestra.run.trial094.stdout: osd default data pool replay window = 5 2026-02-25T09:25:11.148 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.149 INFO:teuthology.orchestra.run.trial094.stdout: mon allow pool delete = true 2026-02-25T09:25:11.149 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.149 INFO:teuthology.orchestra.run.trial094.stdout: mon cluster log file level = debug 2026-02-25T09:25:11.149 INFO:teuthology.orchestra.run.trial094.stdout: debug asserts on shutdown = true 2026-02-25T09:25:11.149 INFO:teuthology.orchestra.run.trial094.stdout: mon health detail to clog = false 2026-02-25T09:25:11.149 INFO:teuthology.orchestra.run.trial094.stdout: mon host = 10.20.193.94 2026-02-25T09:25:11.149 INFO:teuthology.orchestra.run.trial094.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-25T09:25:11.149 INFO:teuthology.orchestra.run.trial094.stdout: ms client mode = crc 2026-02-25T09:25:11.149 INFO:teuthology.orchestra.run.trial094.stdout: ms cluster mode = crc 2026-02-25T09:25:11.149 INFO:teuthology.orchestra.run.trial094.stdout: ms mon client mode = crc 2026-02-25T09:25:11.149 INFO:teuthology.orchestra.run.trial094.stdout: ms mon cluster mode = crc 2026-02-25T09:25:11.149 INFO:teuthology.orchestra.run.trial094.stdout: ms mon service mode = crc 2026-02-25T09:25:11.149 INFO:teuthology.orchestra.run.trial094.stdout: ms service mode = crc 2026-02-25T09:25:11.150 INFO:teuthology.orchestra.run.trial094.stdout: fsid = c0375da7-c136-43b7-8240-9e8e7e46c4e5 2026-02-25T09:25:11.150 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.150 INFO:teuthology.orchestra.run.trial094.stdout:[osd] 2026-02-25T09:25:11.150 INFO:teuthology.orchestra.run.trial094.stdout: osd journal size = 100 2026-02-25T09:25:11.150 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.150 INFO:teuthology.orchestra.run.trial094.stdout: osd scrub load threshold = 5.0 2026-02-25T09:25:11.150 INFO:teuthology.orchestra.run.trial094.stdout: osd scrub max interval = 600 2026-02-25T09:25:11.150 INFO:teuthology.orchestra.run.trial094.stdout: osd mclock profile = high_recovery_ops 2026-02-25T09:25:11.150 INFO:teuthology.orchestra.run.trial094.stdout: osd mclock skip benchmark = true 2026-02-25T09:25:11.150 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.150 INFO:teuthology.orchestra.run.trial094.stdout: osd recover clone overlap = true 2026-02-25T09:25:11.150 INFO:teuthology.orchestra.run.trial094.stdout: osd recovery max chunk = 1048576 2026-02-25T09:25:11.151 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.151 INFO:teuthology.orchestra.run.trial094.stdout: osd debug shutdown = true 2026-02-25T09:25:11.151 INFO:teuthology.orchestra.run.trial094.stdout: osd debug op order = true 2026-02-25T09:25:11.151 INFO:teuthology.orchestra.run.trial094.stdout: osd debug verify stray on activate = true 2026-02-25T09:25:11.151 INFO:teuthology.orchestra.run.trial094.stdout: osd debug trim objects = true 2026-02-25T09:25:11.151 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.151 INFO:teuthology.orchestra.run.trial094.stdout: osd open classes on start = true 2026-02-25T09:25:11.151 INFO:teuthology.orchestra.run.trial094.stdout: osd debug pg log writeout = true 2026-02-25T09:25:11.151 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.151 INFO:teuthology.orchestra.run.trial094.stdout: osd deep scrub update digest min age = 30 2026-02-25T09:25:11.151 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.151 INFO:teuthology.orchestra.run.trial094.stdout: osd map max advance = 10 2026-02-25T09:25:11.151 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.152 INFO:teuthology.orchestra.run.trial094.stdout: journal zero on create = true 2026-02-25T09:25:11.152 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.152 INFO:teuthology.orchestra.run.trial094.stdout: filestore ondisk finisher threads = 3 2026-02-25T09:25:11.152 INFO:teuthology.orchestra.run.trial094.stdout: filestore apply finisher threads = 3 2026-02-25T09:25:11.152 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.152 INFO:teuthology.orchestra.run.trial094.stdout: bdev debug aio = true 2026-02-25T09:25:11.152 INFO:teuthology.orchestra.run.trial094.stdout: osd debug misdirected ops = true 2026-02-25T09:25:11.152 INFO:teuthology.orchestra.run.trial094.stdout: debug monc = 20 2026-02-25T09:25:11.152 INFO:teuthology.orchestra.run.trial094.stdout: crimson cpu num = 4 2026-02-25T09:25:11.152 INFO:teuthology.orchestra.run.trial094.stdout: crimson osd obc lru size = 10 2026-02-25T09:25:11.152 INFO:teuthology.orchestra.run.trial094.stdout: debug ms = 20 2026-02-25T09:25:11.152 INFO:teuthology.orchestra.run.trial094.stdout: debug objclass = 20 2026-02-25T09:25:11.152 INFO:teuthology.orchestra.run.trial094.stdout: debug osd = 20 2026-02-25T09:25:11.153 INFO:teuthology.orchestra.run.trial094.stdout: debug seastore = 20 2026-02-25T09:25:11.153 INFO:teuthology.orchestra.run.trial094.stdout: debug seastore backref = 20 2026-02-25T09:25:11.153 INFO:teuthology.orchestra.run.trial094.stdout: debug seastore cache = 20 2026-02-25T09:25:11.153 INFO:teuthology.orchestra.run.trial094.stdout: debug seastore cleaner = 20 2026-02-25T09:25:11.153 INFO:teuthology.orchestra.run.trial094.stdout: debug seastore device = 20 2026-02-25T09:25:11.153 INFO:teuthology.orchestra.run.trial094.stdout: debug seastore epm = 20 2026-02-25T09:25:11.153 INFO:teuthology.orchestra.run.trial094.stdout: debug seastore fixedkv tree = 20 2026-02-25T09:25:11.153 INFO:teuthology.orchestra.run.trial094.stdout: debug seastore journal = 20 2026-02-25T09:25:11.153 INFO:teuthology.orchestra.run.trial094.stdout: debug seastore lba = 20 2026-02-25T09:25:11.153 INFO:teuthology.orchestra.run.trial094.stdout: debug seastore odata = 20 2026-02-25T09:25:11.153 INFO:teuthology.orchestra.run.trial094.stdout: debug seastore omap = 20 2026-02-25T09:25:11.153 INFO:teuthology.orchestra.run.trial094.stdout: debug seastore onode = 20 2026-02-25T09:25:11.153 INFO:teuthology.orchestra.run.trial094.stdout: debug seastore t = 20 2026-02-25T09:25:11.153 INFO:teuthology.orchestra.run.trial094.stdout: debug seastore tm = 20 2026-02-25T09:25:11.154 INFO:teuthology.orchestra.run.trial094.stdout: osd objectstore = seastore 2026-02-25T09:25:11.154 INFO:teuthology.orchestra.run.trial094.stdout: osd shutdown pgref assert = True 2026-02-25T09:25:11.154 INFO:teuthology.orchestra.run.trial094.stdout: seastore cachepin size pershard = 64M 2026-02-25T09:25:11.154 INFO:teuthology.orchestra.run.trial094.stdout: seastore cachepin type = 2Q 2026-02-25T09:25:11.154 INFO:teuthology.orchestra.run.trial094.stdout: seastore max concurrent transactions = 8 2026-02-25T09:25:11.154 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.154 INFO:teuthology.orchestra.run.trial094.stdout:[mgr] 2026-02-25T09:25:11.154 INFO:teuthology.orchestra.run.trial094.stdout: debug ms = 1 2026-02-25T09:25:11.154 INFO:teuthology.orchestra.run.trial094.stdout: debug mgr = 20 2026-02-25T09:25:11.154 INFO:teuthology.orchestra.run.trial094.stdout: debug mon = 20 2026-02-25T09:25:11.154 INFO:teuthology.orchestra.run.trial094.stdout: debug auth = 20 2026-02-25T09:25:11.154 INFO:teuthology.orchestra.run.trial094.stdout: mon reweight min pgs per osd = 4 2026-02-25T09:25:11.154 INFO:teuthology.orchestra.run.trial094.stdout: mon reweight min bytes per osd = 10 2026-02-25T09:25:11.155 INFO:teuthology.orchestra.run.trial094.stdout: mgr/telemetry/nag = false 2026-02-25T09:25:11.155 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.155 INFO:teuthology.orchestra.run.trial094.stdout:[mon] 2026-02-25T09:25:11.155 INFO:teuthology.orchestra.run.trial094.stdout: debug ms = 1 2026-02-25T09:25:11.155 INFO:teuthology.orchestra.run.trial094.stdout: debug mon = 20 2026-02-25T09:25:11.155 INFO:teuthology.orchestra.run.trial094.stdout: debug paxos = 20 2026-02-25T09:25:11.155 INFO:teuthology.orchestra.run.trial094.stdout: debug auth = 20 2026-02-25T09:25:11.155 INFO:teuthology.orchestra.run.trial094.stdout: mon data avail warn = 5 2026-02-25T09:25:11.155 INFO:teuthology.orchestra.run.trial094.stdout: mon mgr mkfs grace = 240 2026-02-25T09:25:11.155 INFO:teuthology.orchestra.run.trial094.stdout: mon reweight min pgs per osd = 4 2026-02-25T09:25:11.155 INFO:teuthology.orchestra.run.trial094.stdout: mon osd reporter subtree level = osd 2026-02-25T09:25:11.155 INFO:teuthology.orchestra.run.trial094.stdout: mon osd prime pg temp = true 2026-02-25T09:25:11.156 INFO:teuthology.orchestra.run.trial094.stdout: mon reweight min bytes per osd = 10 2026-02-25T09:25:11.156 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.156 INFO:teuthology.orchestra.run.trial094.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-25T09:25:11.156 INFO:teuthology.orchestra.run.trial094.stdout: auth mon ticket ttl = 660 # 11m 2026-02-25T09:25:11.156 INFO:teuthology.orchestra.run.trial094.stdout: auth service ticket ttl = 240 # 4m 2026-02-25T09:25:11.156 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.156 INFO:teuthology.orchestra.run.trial094.stdout: # don't complain about insecure global_id in the test suite 2026-02-25T09:25:11.156 INFO:teuthology.orchestra.run.trial094.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-25T09:25:11.156 INFO:teuthology.orchestra.run.trial094.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-25T09:25:11.156 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.156 INFO:teuthology.orchestra.run.trial094.stdout: # 1m isn't quite enough 2026-02-25T09:25:11.156 INFO:teuthology.orchestra.run.trial094.stdout: mon_down_mkfs_grace = 2m 2026-02-25T09:25:11.156 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.157 INFO:teuthology.orchestra.run.trial094.stdout: mon_warn_on_filestore_osds = false 2026-02-25T09:25:11.157 INFO:teuthology.orchestra.run.trial094.stdout: mon min osdmap epochs = 50 2026-02-25T09:25:11.157 INFO:teuthology.orchestra.run.trial094.stdout: mon osdmap full prune interval = 2 2026-02-25T09:25:11.157 INFO:teuthology.orchestra.run.trial094.stdout: mon osdmap full prune min = 15 2026-02-25T09:25:11.157 INFO:teuthology.orchestra.run.trial094.stdout: mon osdmap full prune txsize = 2 2026-02-25T09:25:11.157 INFO:teuthology.orchestra.run.trial094.stdout: paxos service trim min = 10 2026-02-25T09:25:11.157 INFO:teuthology.orchestra.run.trial094.stdout: osd pool default crimson = True 2026-02-25T09:25:11.157 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:11.157 INFO:teuthology.orchestra.run.trial094.stdout:[client] 2026-02-25T09:25:11.157 INFO:teuthology.orchestra.run.trial094.stdout: rgw cache enabled = true 2026-02-25T09:25:11.157 INFO:teuthology.orchestra.run.trial094.stdout: rgw enable ops log = true 2026-02-25T09:25:11.157 INFO:teuthology.orchestra.run.trial094.stdout: rgw enable usage log = true 2026-02-25T09:25:11.157 INFO:teuthology.orchestra.run.trial094.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-25T09:25:11.158 INFO:teuthology.orchestra.run.trial094.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-25T09:25:11.158 INFO:teuthology.orchestra.run.trial094.stdout:[mon.a] 2026-02-25T09:25:11.158 INFO:teuthology.orchestra.run.trial151.stdout:[global] 2026-02-25T09:25:11.158 INFO:teuthology.orchestra.run.trial151.stdout: chdir = "" 2026-02-25T09:25:11.159 INFO:teuthology.orchestra.run.trial151.stdout: pid file = /var/run/ceph/$cluster-$name.pid 2026-02-25T09:25:11.159 INFO:teuthology.orchestra.run.trial151.stdout: auth supported = cephx 2026-02-25T09:25:11.159 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.159 INFO:teuthology.orchestra.run.trial151.stdout: filestore xattr use omap = true 2026-02-25T09:25:11.159 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.159 INFO:teuthology.orchestra.run.trial151.stdout: mon clock drift allowed = 1.000 2026-02-25T09:25:11.159 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.159 INFO:teuthology.orchestra.run.trial151.stdout: osd crush chooseleaf type = 0 2026-02-25T09:25:11.159 INFO:teuthology.orchestra.run.trial151.stdout: auth debug = true 2026-02-25T09:25:11.159 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.159 INFO:teuthology.orchestra.run.trial151.stdout: ms die on old message = true 2026-02-25T09:25:11.160 INFO:teuthology.orchestra.run.trial151.stdout: ms die on bug = true 2026-02-25T09:25:11.160 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.160 INFO:teuthology.orchestra.run.trial151.stdout: mon max pg per osd = 10000 # >= luminous 2026-02-25T09:25:11.160 INFO:teuthology.orchestra.run.trial151.stdout: mon pg warn max object skew = 0 2026-02-25T09:25:11.160 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.160 INFO:teuthology.orchestra.run.trial151.stdout: # disable pg_autoscaler by default for new pools 2026-02-25T09:25:11.160 INFO:teuthology.orchestra.run.trial151.stdout: osd_pool_default_pg_autoscale_mode = off 2026-02-25T09:25:11.160 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.160 INFO:teuthology.orchestra.run.trial151.stdout: osd pool default size = 2 2026-02-25T09:25:11.160 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.160 INFO:teuthology.orchestra.run.trial151.stdout: mon osd allow primary affinity = true 2026-02-25T09:25:11.160 INFO:teuthology.orchestra.run.trial151.stdout: mon osd allow pg remap = true 2026-02-25T09:25:11.161 INFO:teuthology.orchestra.run.trial151.stdout: mon warn on legacy crush tunables = false 2026-02-25T09:25:11.161 INFO:teuthology.orchestra.run.trial151.stdout: mon warn on crush straw calc version zero = false 2026-02-25T09:25:11.161 INFO:teuthology.orchestra.run.trial151.stdout: mon warn on no sortbitwise = false 2026-02-25T09:25:11.161 INFO:teuthology.orchestra.run.trial151.stdout: mon warn on osd down out interval zero = false 2026-02-25T09:25:11.161 INFO:teuthology.orchestra.run.trial151.stdout: mon warn on too few osds = false 2026-02-25T09:25:11.161 INFO:teuthology.orchestra.run.trial151.stdout: mon_warn_on_pool_pg_num_not_power_of_two = false 2026-02-25T09:25:11.161 INFO:teuthology.orchestra.run.trial151.stdout: mon_warn_on_pool_no_redundancy = false 2026-02-25T09:25:11.161 INFO:teuthology.orchestra.run.trial151.stdout: mon_allow_pool_size_one = true 2026-02-25T09:25:11.161 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.161 INFO:teuthology.orchestra.run.trial151.stdout: osd pool default erasure code profile = plugin=isa technique=reed_sol_van k=2 m=1 crush-failure-domain=osd 2026-02-25T09:25:11.161 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.161 INFO:teuthology.orchestra.run.trial151.stdout: osd default data pool replay window = 5 2026-02-25T09:25:11.162 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.162 INFO:teuthology.orchestra.run.trial151.stdout: mon allow pool delete = true 2026-02-25T09:25:11.162 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.162 INFO:teuthology.orchestra.run.trial151.stdout: mon cluster log file level = debug 2026-02-25T09:25:11.162 INFO:teuthology.orchestra.run.trial151.stdout: debug asserts on shutdown = true 2026-02-25T09:25:11.162 INFO:teuthology.orchestra.run.trial151.stdout: mon health detail to clog = false 2026-02-25T09:25:11.162 INFO:teuthology.orchestra.run.trial151.stdout: mon host = 10.20.193.94 2026-02-25T09:25:11.162 INFO:teuthology.orchestra.run.trial151.stdout: enable experimental unrecoverable data corrupting features = crimson 2026-02-25T09:25:11.162 INFO:teuthology.orchestra.run.trial151.stdout: ms client mode = crc 2026-02-25T09:25:11.162 INFO:teuthology.orchestra.run.trial151.stdout: ms cluster mode = crc 2026-02-25T09:25:11.162 INFO:teuthology.orchestra.run.trial151.stdout: ms mon client mode = crc 2026-02-25T09:25:11.162 INFO:teuthology.orchestra.run.trial151.stdout: ms mon cluster mode = crc 2026-02-25T09:25:11.162 INFO:teuthology.orchestra.run.trial151.stdout: ms mon service mode = crc 2026-02-25T09:25:11.163 INFO:teuthology.orchestra.run.trial151.stdout: ms service mode = crc 2026-02-25T09:25:11.163 INFO:teuthology.orchestra.run.trial151.stdout: fsid = c0375da7-c136-43b7-8240-9e8e7e46c4e5 2026-02-25T09:25:11.163 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.163 INFO:teuthology.orchestra.run.trial151.stdout:[osd] 2026-02-25T09:25:11.163 INFO:teuthology.orchestra.run.trial151.stdout: osd journal size = 100 2026-02-25T09:25:11.163 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.163 INFO:teuthology.orchestra.run.trial151.stdout: osd scrub load threshold = 5.0 2026-02-25T09:25:11.163 INFO:teuthology.orchestra.run.trial151.stdout: osd scrub max interval = 600 2026-02-25T09:25:11.163 INFO:teuthology.orchestra.run.trial151.stdout: osd mclock profile = high_recovery_ops 2026-02-25T09:25:11.163 INFO:teuthology.orchestra.run.trial151.stdout: osd mclock skip benchmark = true 2026-02-25T09:25:11.163 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.163 INFO:teuthology.orchestra.run.trial151.stdout: osd recover clone overlap = true 2026-02-25T09:25:11.164 INFO:teuthology.orchestra.run.trial151.stdout: osd recovery max chunk = 1048576 2026-02-25T09:25:11.164 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.164 INFO:teuthology.orchestra.run.trial151.stdout: osd debug shutdown = true 2026-02-25T09:25:11.164 INFO:teuthology.orchestra.run.trial151.stdout: osd debug op order = true 2026-02-25T09:25:11.164 INFO:teuthology.orchestra.run.trial151.stdout: osd debug verify stray on activate = true 2026-02-25T09:25:11.164 INFO:teuthology.orchestra.run.trial151.stdout: osd debug trim objects = true 2026-02-25T09:25:11.164 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.164 INFO:teuthology.orchestra.run.trial151.stdout: osd open classes on start = true 2026-02-25T09:25:11.164 INFO:teuthology.orchestra.run.trial151.stdout: osd debug pg log writeout = true 2026-02-25T09:25:11.164 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.164 INFO:teuthology.orchestra.run.trial151.stdout: osd deep scrub update digest min age = 30 2026-02-25T09:25:11.164 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.164 INFO:teuthology.orchestra.run.trial151.stdout: osd map max advance = 10 2026-02-25T09:25:11.165 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.165 INFO:teuthology.orchestra.run.trial151.stdout: journal zero on create = true 2026-02-25T09:25:11.165 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.165 INFO:teuthology.orchestra.run.trial151.stdout: filestore ondisk finisher threads = 3 2026-02-25T09:25:11.165 INFO:teuthology.orchestra.run.trial151.stdout: filestore apply finisher threads = 3 2026-02-25T09:25:11.165 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.165 INFO:teuthology.orchestra.run.trial151.stdout: bdev debug aio = true 2026-02-25T09:25:11.165 INFO:teuthology.orchestra.run.trial151.stdout: osd debug misdirected ops = true 2026-02-25T09:25:11.165 INFO:teuthology.orchestra.run.trial151.stdout: debug monc = 20 2026-02-25T09:25:11.165 INFO:teuthology.orchestra.run.trial151.stdout: crimson cpu num = 4 2026-02-25T09:25:11.165 INFO:teuthology.orchestra.run.trial151.stdout: crimson osd obc lru size = 10 2026-02-25T09:25:11.165 INFO:teuthology.orchestra.run.trial151.stdout: debug ms = 20 2026-02-25T09:25:11.165 INFO:teuthology.orchestra.run.trial151.stdout: debug objclass = 20 2026-02-25T09:25:11.166 INFO:teuthology.orchestra.run.trial151.stdout: debug osd = 20 2026-02-25T09:25:11.166 INFO:teuthology.orchestra.run.trial151.stdout: debug seastore = 20 2026-02-25T09:25:11.166 INFO:teuthology.orchestra.run.trial151.stdout: debug seastore backref = 20 2026-02-25T09:25:11.166 INFO:teuthology.orchestra.run.trial151.stdout: debug seastore cache = 20 2026-02-25T09:25:11.166 INFO:teuthology.orchestra.run.trial151.stdout: debug seastore cleaner = 20 2026-02-25T09:25:11.166 INFO:teuthology.orchestra.run.trial151.stdout: debug seastore device = 20 2026-02-25T09:25:11.166 INFO:teuthology.orchestra.run.trial151.stdout: debug seastore epm = 20 2026-02-25T09:25:11.166 INFO:teuthology.orchestra.run.trial151.stdout: debug seastore fixedkv tree = 20 2026-02-25T09:25:11.166 INFO:teuthology.orchestra.run.trial151.stdout: debug seastore journal = 20 2026-02-25T09:25:11.166 INFO:teuthology.orchestra.run.trial151.stdout: debug seastore lba = 20 2026-02-25T09:25:11.166 INFO:teuthology.orchestra.run.trial151.stdout: debug seastore odata = 20 2026-02-25T09:25:11.166 INFO:teuthology.orchestra.run.trial151.stdout: debug seastore omap = 20 2026-02-25T09:25:11.166 INFO:teuthology.orchestra.run.trial151.stdout: debug seastore onode = 20 2026-02-25T09:25:11.167 INFO:teuthology.orchestra.run.trial151.stdout: debug seastore t = 20 2026-02-25T09:25:11.167 INFO:teuthology.orchestra.run.trial151.stdout: debug seastore tm = 20 2026-02-25T09:25:11.167 INFO:teuthology.orchestra.run.trial151.stdout: osd objectstore = seastore 2026-02-25T09:25:11.167 INFO:teuthology.orchestra.run.trial151.stdout: osd shutdown pgref assert = True 2026-02-25T09:25:11.167 INFO:teuthology.orchestra.run.trial151.stdout: seastore cachepin size pershard = 64M 2026-02-25T09:25:11.167 INFO:teuthology.orchestra.run.trial151.stdout: seastore cachepin type = 2Q 2026-02-25T09:25:11.167 INFO:teuthology.orchestra.run.trial151.stdout: seastore max concurrent transactions = 8 2026-02-25T09:25:11.167 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.167 INFO:teuthology.orchestra.run.trial151.stdout:[mgr] 2026-02-25T09:25:11.167 INFO:teuthology.orchestra.run.trial151.stdout: debug ms = 1 2026-02-25T09:25:11.167 INFO:teuthology.orchestra.run.trial151.stdout: debug mgr = 20 2026-02-25T09:25:11.167 INFO:teuthology.orchestra.run.trial151.stdout: debug mon = 20 2026-02-25T09:25:11.168 INFO:teuthology.orchestra.run.trial151.stdout: debug auth = 20 2026-02-25T09:25:11.168 INFO:teuthology.orchestra.run.trial151.stdout: mon reweight min pgs per osd = 4 2026-02-25T09:25:11.168 INFO:teuthology.orchestra.run.trial151.stdout: mon reweight min bytes per osd = 10 2026-02-25T09:25:11.168 INFO:teuthology.orchestra.run.trial151.stdout: mgr/telemetry/nag = false 2026-02-25T09:25:11.168 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.168 INFO:teuthology.orchestra.run.trial151.stdout:[mon] 2026-02-25T09:25:11.168 INFO:teuthology.orchestra.run.trial151.stdout: debug ms = 1 2026-02-25T09:25:11.168 INFO:teuthology.orchestra.run.trial151.stdout: debug mon = 20 2026-02-25T09:25:11.168 INFO:teuthology.orchestra.run.trial151.stdout: debug paxos = 20 2026-02-25T09:25:11.168 INFO:teuthology.orchestra.run.trial151.stdout: debug auth = 20 2026-02-25T09:25:11.169 INFO:teuthology.orchestra.run.trial151.stdout: mon data avail warn = 5 2026-02-25T09:25:11.169 INFO:teuthology.orchestra.run.trial151.stdout: mon mgr mkfs grace = 240 2026-02-25T09:25:11.169 INFO:teuthology.orchestra.run.trial151.stdout: mon reweight min pgs per osd = 4 2026-02-25T09:25:11.169 INFO:teuthology.orchestra.run.trial151.stdout: mon osd reporter subtree level = osd 2026-02-25T09:25:11.169 INFO:teuthology.orchestra.run.trial151.stdout: mon osd prime pg temp = true 2026-02-25T09:25:11.169 INFO:teuthology.orchestra.run.trial151.stdout: mon reweight min bytes per osd = 10 2026-02-25T09:25:11.169 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.169 INFO:teuthology.orchestra.run.trial151.stdout: # rotate auth tickets quickly to exercise renewal paths 2026-02-25T09:25:11.169 INFO:teuthology.orchestra.run.trial151.stdout: auth mon ticket ttl = 660 # 11m 2026-02-25T09:25:11.170 INFO:teuthology.orchestra.run.trial151.stdout: auth service ticket ttl = 240 # 4m 2026-02-25T09:25:11.170 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.170 INFO:teuthology.orchestra.run.trial151.stdout: # don't complain about insecure global_id in the test suite 2026-02-25T09:25:11.170 INFO:teuthology.orchestra.run.trial151.stdout: mon_warn_on_insecure_global_id_reclaim = false 2026-02-25T09:25:11.170 INFO:teuthology.orchestra.run.trial151.stdout: mon_warn_on_insecure_global_id_reclaim_allowed = false 2026-02-25T09:25:11.170 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.170 INFO:teuthology.orchestra.run.trial151.stdout: # 1m isn't quite enough 2026-02-25T09:25:11.170 INFO:teuthology.orchestra.run.trial151.stdout: mon_down_mkfs_grace = 2m 2026-02-25T09:25:11.170 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.170 INFO:teuthology.orchestra.run.trial151.stdout: mon_warn_on_filestore_osds = false 2026-02-25T09:25:11.170 INFO:teuthology.orchestra.run.trial151.stdout: mon min osdmap epochs = 50 2026-02-25T09:25:11.170 INFO:teuthology.orchestra.run.trial151.stdout: mon osdmap full prune interval = 2 2026-02-25T09:25:11.171 INFO:teuthology.orchestra.run.trial151.stdout: mon osdmap full prune min = 15 2026-02-25T09:25:11.171 INFO:teuthology.orchestra.run.trial151.stdout: mon osdmap full prune txsize = 2 2026-02-25T09:25:11.171 INFO:teuthology.orchestra.run.trial151.stdout: paxos service trim min = 10 2026-02-25T09:25:11.171 INFO:teuthology.orchestra.run.trial151.stdout: osd pool default crimson = True 2026-02-25T09:25:11.171 INFO:teuthology.orchestra.run.trial151.stdout: 2026-02-25T09:25:11.171 INFO:teuthology.orchestra.run.trial151.stdout:[client] 2026-02-25T09:25:11.171 INFO:teuthology.orchestra.run.trial151.stdout: rgw cache enabled = true 2026-02-25T09:25:11.171 INFO:teuthology.orchestra.run.trial151.stdout: rgw enable ops log = true 2026-02-25T09:25:11.171 INFO:teuthology.orchestra.run.trial151.stdout: rgw enable usage log = true 2026-02-25T09:25:11.171 INFO:teuthology.orchestra.run.trial151.stdout: log file = /var/log/ceph/$cluster-$name.$pid.log 2026-02-25T09:25:11.171 INFO:teuthology.orchestra.run.trial151.stdout: admin socket = /var/run/ceph/$cluster-$name.$pid.asok 2026-02-25T09:25:11.171 INFO:teuthology.orchestra.run.trial151.stdout:[mon.a] 2026-02-25T09:25:11.173 INFO:tasks.ceph:Creating admin key on mon.a... 2026-02-25T09:25:11.174 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --gen-key --name=client.admin --cap mon 'allow *' --cap osd 'allow *' --cap mds 'allow *' --cap mgr 'allow *' /etc/ceph/ceph.keyring 2026-02-25T09:25:11.262 INFO:tasks.ceph:Copying monmap to all nodes... 2026-02-25T09:25:11.263 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:25:11.263 DEBUG:teuthology.orchestra.run.trial094:> dd if=/etc/ceph/ceph.keyring of=/dev/stdout 2026-02-25T09:25:11.279 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:25:11.279 DEBUG:teuthology.orchestra.run.trial094:> dd if=/home/ubuntu/cephtest/ceph.monmap of=/dev/stdout 2026-02-25T09:25:11.336 INFO:tasks.ceph:Sending monmap to node ubuntu@trial094.front.sepia.ceph.com 2026-02-25T09:25:11.336 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:25:11.336 DEBUG:teuthology.orchestra.run.trial094:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-25T09:25:11.336 DEBUG:teuthology.orchestra.run.trial094:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-25T09:25:11.408 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:25:11.408 DEBUG:teuthology.orchestra.run.trial094:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-25T09:25:11.464 INFO:tasks.ceph:Sending monmap to node ubuntu@trial151.front.sepia.ceph.com 2026-02-25T09:25:11.465 DEBUG:teuthology.orchestra.run.trial151:> set -ex 2026-02-25T09:25:11.465 DEBUG:teuthology.orchestra.run.trial151:> sudo dd of=/etc/ceph/ceph.keyring 2026-02-25T09:25:11.465 DEBUG:teuthology.orchestra.run.trial151:> sudo chmod 0644 /etc/ceph/ceph.keyring 2026-02-25T09:25:11.498 DEBUG:teuthology.orchestra.run.trial151:> set -ex 2026-02-25T09:25:11.498 DEBUG:teuthology.orchestra.run.trial151:> dd of=/home/ubuntu/cephtest/ceph.monmap 2026-02-25T09:25:11.554 INFO:tasks.ceph:Setting up mon nodes... 2026-02-25T09:25:11.554 INFO:tasks.ceph:Setting up mgr nodes... 2026-02-25T09:25:11.554 DEBUG:teuthology.orchestra.run.trial151:> sudo mkdir -p /var/lib/ceph/mgr/ceph-x && sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=mgr.x /var/lib/ceph/mgr/ceph-x/keyring 2026-02-25T09:25:11.644 INFO:teuthology.orchestra.run.trial151.stdout:creating /var/lib/ceph/mgr/ceph-x/keyring 2026-02-25T09:25:11.647 INFO:tasks.ceph:Setting up mds nodes... 2026-02-25T09:25:11.648 INFO:tasks.ceph_client:Setting up client nodes... 2026-02-25T09:25:11.648 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.0 /etc/ceph/ceph.client.0.keyring && sudo chmod 0644 /etc/ceph/ceph.client.0.keyring 2026-02-25T09:25:11.690 INFO:teuthology.orchestra.run.trial094.stdout:creating /etc/ceph/ceph.client.0.keyring 2026-02-25T09:25:11.702 DEBUG:teuthology.orchestra.run.trial151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool --create-keyring --gen-key --name=client.1 /etc/ceph/ceph.client.1.keyring && sudo chmod 0644 /etc/ceph/ceph.client.1.keyring 2026-02-25T09:25:11.744 INFO:teuthology.orchestra.run.trial151.stdout:creating /etc/ceph/ceph.client.1.keyring 2026-02-25T09:25:11.756 INFO:tasks.ceph:Running mkfs on osd nodes... 2026-02-25T09:25:11.756 INFO:tasks.ceph:ctx.disk_config.remote_to_roles_to_dev: {Remote(name='ubuntu@trial094.front.sepia.ceph.com'): {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'}, Remote(name='ubuntu@trial151.front.sepia.ceph.com'): {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'}} 2026-02-25T09:25:11.756 DEBUG:teuthology.orchestra.run.trial094:> sudo mkdir -p /var/lib/ceph/osd/ceph-0 2026-02-25T09:25:11.781 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-25T09:25:11.781 INFO:tasks.ceph:role: osd.0 2026-02-25T09:25:11.782 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial094.front.sepia.ceph.com 2026-02-25T09:25:11.782 DEBUG:teuthology.orchestra.run.trial094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-25T09:25:11.847 INFO:teuthology.orchestra.run.trial094.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T09:25:11.847 INFO:teuthology.orchestra.run.trial094.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T09:25:11.847 INFO:teuthology.orchestra.run.trial094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T09:25:11.847 INFO:teuthology.orchestra.run.trial094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T09:25:11.847 INFO:teuthology.orchestra.run.trial094.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T09:25:11.847 INFO:teuthology.orchestra.run.trial094.stdout: = sunit=0 swidth=0 blks 2026-02-25T09:25:11.847 INFO:teuthology.orchestra.run.trial094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T09:25:11.847 INFO:teuthology.orchestra.run.trial094.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T09:25:11.847 INFO:teuthology.orchestra.run.trial094.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T09:25:11.848 INFO:teuthology.orchestra.run.trial094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T09:25:12.425 INFO:teuthology.orchestra.run.trial094.stdout:Discarding blocks...Done. 2026-02-25T09:25:12.427 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial094.front.sepia.ceph.com -o noatime 2026-02-25T09:25:12.427 DEBUG:teuthology.orchestra.run.trial094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-0 2026-02-25T09:25:12.495 DEBUG:teuthology.orchestra.run.trial094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-0 2026-02-25T09:25:12.563 DEBUG:teuthology.orchestra.run.trial094:> sudo mkdir -p /var/lib/ceph/osd/ceph-1 2026-02-25T09:25:12.626 INFO:tasks.ceph:roles_to_devs: {'osd.0': '/dev/vg_nvme/lv_1', 'osd.1': '/dev/vg_nvme/lv_2'} 2026-02-25T09:25:12.627 INFO:tasks.ceph:role: osd.1 2026-02-25T09:25:12.627 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial094.front.sepia.ceph.com 2026-02-25T09:25:12.627 DEBUG:teuthology.orchestra.run.trial094:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-25T09:25:12.691 INFO:teuthology.orchestra.run.trial094.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T09:25:12.691 INFO:teuthology.orchestra.run.trial094.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T09:25:12.691 INFO:teuthology.orchestra.run.trial094.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T09:25:12.691 INFO:teuthology.orchestra.run.trial094.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T09:25:12.691 INFO:teuthology.orchestra.run.trial094.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T09:25:12.691 INFO:teuthology.orchestra.run.trial094.stdout: = sunit=0 swidth=0 blks 2026-02-25T09:25:12.692 INFO:teuthology.orchestra.run.trial094.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T09:25:12.692 INFO:teuthology.orchestra.run.trial094.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T09:25:12.692 INFO:teuthology.orchestra.run.trial094.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T09:25:12.692 INFO:teuthology.orchestra.run.trial094.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T09:25:13.251 INFO:teuthology.orchestra.run.trial094.stdout:Discarding blocks...Done. 2026-02-25T09:25:13.253 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial094.front.sepia.ceph.com -o noatime 2026-02-25T09:25:13.253 DEBUG:teuthology.orchestra.run.trial094:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-1 2026-02-25T09:25:13.322 DEBUG:teuthology.orchestra.run.trial094:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-1 2026-02-25T09:25:13.389 DEBUG:teuthology.orchestra.run.trial094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 0 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T09:25:13.541 INFO:teuthology.orchestra.run.trial094.stderr:INFO  osd - OSD::main: parsing early config 2026-02-25T09:25:13.595 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:13,594 seastar - Reactor backend: linux-aio 2026-02-25T09:25:13.608 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:13,607 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T09:25:13.608 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:13,607 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T09:25:13.610 INFO:teuthology.orchestra.run.trial094.stderr:==38855==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T09:25:13.666 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:13,665 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-25T09:25:13.666 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:13,665 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-25T09:25:13.666 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:13,665 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-25T09:25:13.667 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:13,665 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-25T09:25:13.667 INFO:teuthology.orchestra.run.trial094.stderr:DEBUG 2026-02-25 09:25:13,666 [shard 0:main] seastar - reactor::drain 2026-02-25T09:25:15.740 INFO:teuthology.orchestra.run.trial094.stderr:----------------------------------------------------- 2026-02-25T09:25:15.740 INFO:teuthology.orchestra.run.trial094.stderr:Suppressions used: 2026-02-25T09:25:15.740 INFO:teuthology.orchestra.run.trial094.stderr: count bytes template 2026-02-25T09:25:15.741 INFO:teuthology.orchestra.run.trial094.stderr: 1 8 MallocExtension::Register 2026-02-25T09:25:15.741 INFO:teuthology.orchestra.run.trial094.stderr:----------------------------------------------------- 2026-02-25T09:25:15.741 INFO:teuthology.orchestra.run.trial094.stderr: 2026-02-25T09:25:15.760 INFO:teuthology.orchestra.run.trial094.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-25T09:25:15.760 INFO:teuthology.orchestra.run.trial094.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-25T09:25:15.808 INFO:teuthology.orchestra.run.trial094.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-25T09:25:15.808 INFO:teuthology.orchestra.run.trial094.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-25T09:25:15.815 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:15,814 seastar - Reactor backend: linux-aio 2026-02-25T09:25:15.829 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:15,829 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T09:25:15.829 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:15,829 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T09:25:15.833 INFO:teuthology.orchestra.run.trial094.stderr:==38853==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T09:25:15.833 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:15,833 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-25T09:25:15.890 INFO:teuthology.orchestra.run.trial094.stderr:DEBUG 2026-02-25 09:25:15,890 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-25T09:25:15.971 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x316cbcf 0x9b5c24c 0x9baf7e1 0x9cffea3 0x9b66177 0x9b667a7 0x9b66911 0x9c74e1d 0x9c86534 0x9bd29d3 0x9bd19d3 0x7bea805 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6faf42c 0x6fb322d 0x6fb8dd7 0x748b7b0 0x72b8d6e 0x72b8e70 0x72b8f68 0x738168e 0x7381747 0x73818fa 0x7381a32 0x7381aad 0x7381b32 0x7381c8d 0x7436297 0x748b839 0x72a0671 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:15.971 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x316b7b9 0x9b5bfcc 0x9b5c25e 0x9baf7e1 0x9cffea3 0x9b66177 0x9b667a7 0x9b66911 0x9c74e1d 0x9c86534 0x9bd29d3 0x9bd19d3 0x7bea805 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6faf42c 0x6fb322d 0x6fb8dd7 0x748b7b0 0x72b8d6e 0x72b8e70 0x72b8f68 0x738168e 0x7381747 0x73818fa 0x7381a32 0x7381aad 0x7381b32 0x7381c8d 0x7436297 0x748b839 0x72a0671 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:15.971 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65c2 /lib64/libasan.so.8+0x15197 0x9b5bb8a 0x9b5bfeb 0x9b5c25e 0x9baf7e1 0x9cffea3 0x9b66177 0x9b667a7 0x9b66911 0x9c74e1d 0x9c86534 0x9bd29d3 0x9bd19d3 0x7bea805 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6faf42c 0x6fb322d 0x6fb8dd7 0x748b7b0 0x72b8d6e 0x72b8e70 0x72b8f68 0x738168e 0x7381747 0x73818fa 0x7381a32 0x7381aad 0x7381b32 0x7381c8d 0x7436297 0x748b839 0x72a0671 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:16.003 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119045 /lib64/libasan.so.8+0xc5f4 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9c67de2 0x9c67eae 0x9c82e11 0x9c87fbb 0x9bd29d3 0x9bd19d3 0x7e7101b 0x7ea85c2 0x8edf659 0x8ee1fc7 0x7889765 0x78af280 0x6fb05f7 0x756b036 0x756b18a 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9e13a8c 0x9c2a677 0x9c2aeb2 0xecf56f /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47084 2026-02-25T09:25:16.069 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a5317b 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9e13a8c 0x9c2a677 0x9c2aeb2 0xecf56f /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47084 2026-02-25T09:25:16.109 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc65bc /lib64/libasan.so.8+0x15197 0x2a5317b 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:16.109 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297db /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x2a5317b 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:16.109 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x2a53048 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:16.232 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xea88 /lib64/libasan.so.8+0xec776 0xcb0635 0x6f7f348 0x7bea882 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6f47f22 0x6f4a28a 0x748861c 0x72b6bab 0x72b6cad 0x72b6da5 0x7380162 0x738021b 0x73803ce 0x7380506 0x7380581 0x7380606 0x7380761 0x7435f79 0x74886a5 0x729f1a8 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:16.232 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x119122 /lib64/libasan.so.8+0xebbf /lib64/libasan.so.8+0xef905 0x9bca93d 0x9bcab48 0x9bcae1a 0x7bea843 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6f47f22 0x6f4a28a 0x748861c 0x72b6bab 0x72b6cad 0x72b6da5 0x7380162 0x738021b 0x73803ce 0x7380506 0x7380581 0x7380606 0x7380761 0x7435f79 0x74886a5 0x729f1a8 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:16.232 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x3170038 0x317013a 0x9c86551 0x9bd29d3 0x9bd19d3 0x7bea805 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6f47f22 0x6f4a28a 0x748861c 0x72b6bab 0x72b6cad 0x72b6da5 0x7380162 0x738021b 0x73803ce 0x7380506 0x7380581 0x7380606 0x7380761 0x7435f79 0x74886a5 0x729f1a8 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:16.296 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x2a5316a 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:16.314 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2a5317b 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9e13a8c 0x9c2a677 0x9c2aeb2 0xecf56f /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47084 2026-02-25T09:25:16.329 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x2a52aba 0x7da438e 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:16.329 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1518b 0x2a5317b 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:18.586 INFO:teuthology.orchestra.run.trial094.stderr:----------------------------------------------------- 2026-02-25T09:25:18.586 INFO:teuthology.orchestra.run.trial094.stderr:Suppressions used: 2026-02-25T09:25:18.586 INFO:teuthology.orchestra.run.trial094.stderr: count bytes template 2026-02-25T09:25:18.586 INFO:teuthology.orchestra.run.trial094.stderr: 1 8 MallocExtension::Register 2026-02-25T09:25:18.586 INFO:teuthology.orchestra.run.trial094.stderr:----------------------------------------------------- 2026-02-25T09:25:18.586 INFO:teuthology.orchestra.run.trial094.stderr: 2026-02-25T09:25:18.620 DEBUG:teuthology.orchestra.run.trial094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-0 2026-02-25T09:25:18.645 DEBUG:teuthology.orchestra.run.trial094:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 1 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T09:25:18.796 INFO:teuthology.orchestra.run.trial094.stderr:INFO  osd - OSD::main: parsing early config 2026-02-25T09:25:18.856 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:18,855 seastar - Reactor backend: linux-aio 2026-02-25T09:25:18.874 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:18,873 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T09:25:18.874 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:18,873 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T09:25:18.876 INFO:teuthology.orchestra.run.trial094.stderr:==38922==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T09:25:18.931 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:18,931 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-25T09:25:18.932 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:18,931 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-25T09:25:18.932 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:18,931 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-25T09:25:18.932 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:18,931 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-25T09:25:18.932 INFO:teuthology.orchestra.run.trial094.stderr:DEBUG 2026-02-25 09:25:18,931 [shard 0:main] seastar - reactor::drain 2026-02-25T09:25:20.993 INFO:teuthology.orchestra.run.trial094.stderr:----------------------------------------------------- 2026-02-25T09:25:20.993 INFO:teuthology.orchestra.run.trial094.stderr:Suppressions used: 2026-02-25T09:25:20.993 INFO:teuthology.orchestra.run.trial094.stderr: count bytes template 2026-02-25T09:25:20.993 INFO:teuthology.orchestra.run.trial094.stderr: 1 8 MallocExtension::Register 2026-02-25T09:25:20.993 INFO:teuthology.orchestra.run.trial094.stderr:----------------------------------------------------- 2026-02-25T09:25:20.993 INFO:teuthology.orchestra.run.trial094.stderr: 2026-02-25T09:25:21.013 INFO:teuthology.orchestra.run.trial094.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-25T09:25:21.013 INFO:teuthology.orchestra.run.trial094.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-25T09:25:21.059 INFO:teuthology.orchestra.run.trial094.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-25T09:25:21.059 INFO:teuthology.orchestra.run.trial094.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-25T09:25:21.066 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:21,065 seastar - Reactor backend: linux-aio 2026-02-25T09:25:21.075 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:21,074 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T09:25:21.075 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:21,074 [shard 1:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T09:25:21.079 INFO:teuthology.orchestra.run.trial094.stderr:==38920==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T09:25:21.079 INFO:teuthology.orchestra.run.trial094.stderr:INFO 2026-02-25 09:25:21,079 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-25T09:25:21.138 INFO:teuthology.orchestra.run.trial094.stderr:DEBUG 2026-02-25 09:25:21,138 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-25T09:25:21.221 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x316b83d 0x9b5bfcc 0x9b5c25e 0x9bb0988 0x9bb0ed6 0x9bb10d5 0x9c79155 0x9c798b0 0x9c83b42 0x9c87fbb 0x9bd29d3 0x9bd19d3 0x7bea805 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6faf42c 0x6fb322d 0x6fb8dd7 0x748b7b0 0x72b8d6e 0x72b8e70 0x72b8f68 0x738168e 0x7381747 0x73818fa 0x7381a32 0x7381aad 0x7381b32 0x7381c8d 0x7436297 0x748b839 0x72a0671 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:21.221 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x9b25a2a 0x9c8a340 0x77dcc3b 0x7bea137 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6faf42c 0x6fb322d 0x6fb8dd7 0x748b7b0 0x72b8d6e 0x72b8e70 0x72b8f68 0x738168e 0x7381747 0x73818fa 0x7381a32 0x7381aad 0x7381b32 0x7381c8d 0x7436297 0x748b839 0x72a0671 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:21.223 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 35 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x191c62 /lib64/libasan.so.8+0xc748 /lib64/libasan.so.8+0x8eea /lib64/libasan.so.8+0xedc45 0xe1d6df 0xe1da36 0x9c8acd3 0x77dcc3b 0x7bea73c 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6faf42c 0x6fb322d 0x6fb8dd7 0x748b7b0 0x72b8d6e 0x72b8e70 0x72b8f68 0x738168e 0x7381747 0x73818fa 0x7381a32 0x7381aad 0x7381b32 0x7381c8d 0x7436297 0x748b839 0x72a0671 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:21.223 INFO:teuthology.orchestra.run.trial094.stderr:kernel callstack: 0xffffffffffffff80 0xffffffffb69fa8b4 0xffffffffb69fc7f9 0xffffffffb69fdd21 0xffffffffb6a25a53 0xffffffffb6a25c15 0xffffffffb69cc8ce 0xffffffffb69d4bab 0xffffffffb69d500b 0xffffffffb66910d2 0xffffffffb7350451 0xffffffffb7400ba2 2026-02-25T09:25:21.317 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x2a52fc4 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9e13a8c 0x9c2a677 0x9c2aeb2 0xecf56f /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47084 2026-02-25T09:25:21.358 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x2a524da 0x7da4381 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:21.358 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x2a52afd 0x7da438e 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:21.361 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x2a52b1b 0x7da438e 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:21.486 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0xb7bd1d 0xbb761e 0x7bdb034 0x7bea3e1 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6f47f22 0x6f4a28a 0x748861c 0x72b6bab 0x72b6cad 0x72b6da5 0x7380162 0x738021b 0x73803ce 0x7380506 0x7380581 0x7380606 0x7380761 0x7435f79 0x74886a5 0x729f1a8 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:21.486 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x3606d5d 0x9d010a1 0x9d01163 0x9d54a30 0x9c6d8c5 0x9c6dc57 0x9c6de62 0x9c6e19e 0x9c6e5be 0x9c832d3 0x9c87fbb 0x9bd29d3 0x9bd19d3 0x7bea805 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6f47f22 0x6f4a28a 0x748861c 0x72b6bab 0x72b6cad 0x72b6da5 0x7380162 0x738021b 0x73803ce 0x7380506 0x7380581 0x7380606 0x7380761 0x7435f79 0x74886a5 0x729f1a8 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:21.486 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x9bd2bb5 0x9bd2c74 0x9c67554 0x9c67bf1 0x9c67e02 0x9c67eae 0x9c82e11 0x9c87fbb 0x9bd29d3 0x9bd19d3 0x7bea805 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6f47f22 0x6f4a28a 0x748861c 0x72b6bab 0x72b6cad 0x72b6da5 0x7380162 0x738021b 0x73803ce 0x7380506 0x7380581 0x7380606 0x7380761 0x7435f79 0x74886a5 0x729f1a8 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:21.569 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cbf3 /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2a5317b 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9e13a8c 0x9c2a677 0x9c2aeb2 0xecf56f /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47084 2026-02-25T09:25:21.582 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x2a52ae0 0x7da438e 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:21.583 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151b9 0x2a5317b 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:21.583 INFO:teuthology.orchestra.run.trial094.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x2a52128 0x7da436c 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:23.765 INFO:teuthology.orchestra.run.trial094.stderr:----------------------------------------------------- 2026-02-25T09:25:23.765 INFO:teuthology.orchestra.run.trial094.stderr:Suppressions used: 2026-02-25T09:25:23.765 INFO:teuthology.orchestra.run.trial094.stderr: count bytes template 2026-02-25T09:25:23.765 INFO:teuthology.orchestra.run.trial094.stderr: 1 8 MallocExtension::Register 2026-02-25T09:25:23.765 INFO:teuthology.orchestra.run.trial094.stderr:----------------------------------------------------- 2026-02-25T09:25:23.765 INFO:teuthology.orchestra.run.trial094.stderr: 2026-02-25T09:25:23.799 DEBUG:teuthology.orchestra.run.trial094:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-1 2026-02-25T09:25:23.825 DEBUG:teuthology.orchestra.run.trial151:> sudo mkdir -p /var/lib/ceph/osd/ceph-2 2026-02-25T09:25:23.851 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-25T09:25:23.851 INFO:tasks.ceph:role: osd.2 2026-02-25T09:25:23.851 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_1 on ubuntu@trial151.front.sepia.ceph.com 2026-02-25T09:25:23.851 DEBUG:teuthology.orchestra.run.trial151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_1 2026-02-25T09:25:23.916 INFO:teuthology.orchestra.run.trial151.stdout:meta-data=/dev/vg_nvme/lv_1 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T09:25:23.916 INFO:teuthology.orchestra.run.trial151.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T09:25:23.916 INFO:teuthology.orchestra.run.trial151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T09:25:23.917 INFO:teuthology.orchestra.run.trial151.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T09:25:23.917 INFO:teuthology.orchestra.run.trial151.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T09:25:23.917 INFO:teuthology.orchestra.run.trial151.stdout: = sunit=0 swidth=0 blks 2026-02-25T09:25:23.917 INFO:teuthology.orchestra.run.trial151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T09:25:23.917 INFO:teuthology.orchestra.run.trial151.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T09:25:23.917 INFO:teuthology.orchestra.run.trial151.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T09:25:23.917 INFO:teuthology.orchestra.run.trial151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T09:25:24.491 INFO:teuthology.orchestra.run.trial151.stdout:Discarding blocks...Done. 2026-02-25T09:25:24.493 INFO:tasks.ceph:mount /dev/vg_nvme/lv_1 on ubuntu@trial151.front.sepia.ceph.com -o noatime 2026-02-25T09:25:24.494 DEBUG:teuthology.orchestra.run.trial151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_1 /var/lib/ceph/osd/ceph-2 2026-02-25T09:25:24.562 DEBUG:teuthology.orchestra.run.trial151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-2 2026-02-25T09:25:24.630 DEBUG:teuthology.orchestra.run.trial151:> sudo mkdir -p /var/lib/ceph/osd/ceph-3 2026-02-25T09:25:24.694 INFO:tasks.ceph:roles_to_devs: {'osd.2': '/dev/vg_nvme/lv_1', 'osd.3': '/dev/vg_nvme/lv_2'} 2026-02-25T09:25:24.694 INFO:tasks.ceph:role: osd.3 2026-02-25T09:25:24.694 INFO:tasks.ceph:['mkfs.xfs', '-f', '-i', 'size=2048'] on /dev/vg_nvme/lv_2 on ubuntu@trial151.front.sepia.ceph.com 2026-02-25T09:25:24.695 DEBUG:teuthology.orchestra.run.trial151:> yes | sudo mkfs.xfs -f -i size=2048 /dev/vg_nvme/lv_2 2026-02-25T09:25:24.759 INFO:teuthology.orchestra.run.trial151.stdout:meta-data=/dev/vg_nvme/lv_2 isize=2048 agcount=4, agsize=45875200 blks 2026-02-25T09:25:24.759 INFO:teuthology.orchestra.run.trial151.stdout: = sectsz=4096 attr=2, projid32bit=1 2026-02-25T09:25:24.759 INFO:teuthology.orchestra.run.trial151.stdout: = crc=1 finobt=1, sparse=1, rmapbt=0 2026-02-25T09:25:24.759 INFO:teuthology.orchestra.run.trial151.stdout: = reflink=1 bigtime=1 inobtcount=1 nrext64=0 2026-02-25T09:25:24.759 INFO:teuthology.orchestra.run.trial151.stdout:data = bsize=4096 blocks=183500800, imaxpct=25 2026-02-25T09:25:24.760 INFO:teuthology.orchestra.run.trial151.stdout: = sunit=0 swidth=0 blks 2026-02-25T09:25:24.760 INFO:teuthology.orchestra.run.trial151.stdout:naming =version 2 bsize=4096 ascii-ci=0, ftype=1 2026-02-25T09:25:24.760 INFO:teuthology.orchestra.run.trial151.stdout:log =internal log bsize=4096 blocks=89600, version=2 2026-02-25T09:25:24.760 INFO:teuthology.orchestra.run.trial151.stdout: = sectsz=4096 sunit=1 blks, lazy-count=1 2026-02-25T09:25:24.760 INFO:teuthology.orchestra.run.trial151.stdout:realtime =none extsz=4096 blocks=0, rtextents=0 2026-02-25T09:25:25.321 INFO:teuthology.orchestra.run.trial151.stdout:Discarding blocks...Done. 2026-02-25T09:25:25.323 INFO:tasks.ceph:mount /dev/vg_nvme/lv_2 on ubuntu@trial151.front.sepia.ceph.com -o noatime 2026-02-25T09:25:25.324 DEBUG:teuthology.orchestra.run.trial151:> sudo mount -t xfs -o noatime /dev/vg_nvme/lv_2 /var/lib/ceph/osd/ceph-3 2026-02-25T09:25:25.392 DEBUG:teuthology.orchestra.run.trial151:> sudo /sbin/restorecon /var/lib/ceph/osd/ceph-3 2026-02-25T09:25:25.459 DEBUG:teuthology.orchestra.run.trial151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 2 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T09:25:25.610 INFO:teuthology.orchestra.run.trial151.stderr:INFO  osd - OSD::main: parsing early config 2026-02-25T09:25:25.664 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:25,663 seastar - Reactor backend: linux-aio 2026-02-25T09:25:25.672 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:25,671 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T09:25:25.672 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:25,672 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T09:25:25.674 INFO:teuthology.orchestra.run.trial151.stderr:==38262==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T09:25:25.729 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:25,728 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-25T09:25:25.729 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:25,728 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-25T09:25:25.729 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:25,728 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-25T09:25:25.729 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:25,728 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-25T09:25:25.729 INFO:teuthology.orchestra.run.trial151.stderr:DEBUG 2026-02-25 09:25:25,728 [shard 0:main] seastar - reactor::drain 2026-02-25T09:25:27.725 INFO:teuthology.orchestra.run.trial151.stderr:----------------------------------------------------- 2026-02-25T09:25:27.725 INFO:teuthology.orchestra.run.trial151.stderr:Suppressions used: 2026-02-25T09:25:27.725 INFO:teuthology.orchestra.run.trial151.stderr: count bytes template 2026-02-25T09:25:27.725 INFO:teuthology.orchestra.run.trial151.stderr: 1 8 MallocExtension::Register 2026-02-25T09:25:27.725 INFO:teuthology.orchestra.run.trial151.stderr:----------------------------------------------------- 2026-02-25T09:25:27.725 INFO:teuthology.orchestra.run.trial151.stderr: 2026-02-25T09:25:27.745 INFO:teuthology.orchestra.run.trial151.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-25T09:25:27.745 INFO:teuthology.orchestra.run.trial151.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-25T09:25:27.787 INFO:teuthology.orchestra.run.trial151.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-25T09:25:27.787 INFO:teuthology.orchestra.run.trial151.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-25T09:25:27.794 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:27,793 seastar - Reactor backend: linux-aio 2026-02-25T09:25:27.805 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:27,805 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T09:25:27.805 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:27,805 [shard 3:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T09:25:27.809 INFO:teuthology.orchestra.run.trial151.stderr:==38260==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T09:25:27.810 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:27,809 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-25T09:25:27.867 INFO:teuthology.orchestra.run.trial151.stderr:DEBUG 2026-02-25 09:25:27,866 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-25T09:25:27.947 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0xe1d536 0x9bd2b4c 0x9c8ac10 0x77dcc3b 0x7bea137 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6faf42c 0x6fb322d 0x6fb8dd7 0x748b7b0 0x72b8d6e 0x72b8e70 0x72b8f68 0x738168e 0x7381747 0x73818fa 0x7381a32 0x7381aad 0x7381b32 0x7381c8d 0x7436297 0x748b839 0x72a0671 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:27.947 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1521e 0x1125f6c 0xf2e4f3 0x9b5bb21 0x9b5bfeb 0x9b5c25e 0x9baf7e1 0x9cffea3 0x9b66177 0x9b667a7 0x9b66911 0x9c74e1d 0x9c86534 0x9bd29d3 0x9bd19d3 0x7bea805 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6faf42c 0x6fb322d 0x6fb8dd7 0x748b7b0 0x72b8d6e 0x72b8e70 0x72b8f68 0x738168e 0x7381747 0x73818fa 0x7381a32 0x7381aad 0x7381b32 0x7381c8d 0x7436297 0x748b839 0x72a0671 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:27.947 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 33 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc597 /lib64/libasan.so.8+0x91c1 /lib64/libasan.so.8+0xee7dc 0x9bd5608 0x9bd5719 0x7bea779 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6faf42c 0x6fb322d 0x6fb8dd7 0x748b7b0 0x72b8d6e 0x72b8e70 0x72b8f68 0x738168e 0x7381747 0x73818fa 0x7381a32 0x7381aad 0x7381b32 0x7381c8d 0x7436297 0x748b839 0x72a0671 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:27.947 INFO:teuthology.orchestra.run.trial151.stderr:kernel callstack: 0xffffffffffffff80 0xffffffff8f81ffa4 0xffffffff8f8203b0 0xffffffff8f69525e 0xffffffff8f9e7269 0xffffffff8f9ca7d5 0xffffffff8f9d4bab 0xffffffff8f9d500b 0xffffffff8f6910d2 0xffffffff90350451 0xffffffff90400ba2 2026-02-25T09:25:28.020 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x9bbd8ff 0x9bd3abb 0x9bd3c83 0x9c6e0ca 0x9c6e5be 0x9c832d3 0x9c87fbb 0x9bd29d3 0x9bd19d3 0x7e7101b 0x7ea85c2 0x8edf659 0x8ee1fc7 0x7889765 0x78af280 0x6fb05f7 0x756b036 0x756b18a 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:28.044 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x7da4347 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9e13a8c 0x9c2a677 0x9c2aeb2 0xecf56f /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47084 2026-02-25T09:25:28.084 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x2a5317b 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:28.084 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xf4e8f /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2a5317b 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:28.085 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x2a5317b 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:28.213 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x9c79d0c 0x9c7f3d4 0x9c7fa12 0x9c8abf5 0x77dcc3b 0x7bea137 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6f47f22 0x6f4a28a 0x748861c 0x72b6bab 0x72b6cad 0x72b6da5 0x7380162 0x738021b 0x73803ce 0x7380506 0x7380581 0x7380606 0x7380761 0x7435f79 0x74886a5 0x729f1a8 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:28.213 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xb0497 /lib64/libasan.so.8+0xb1437 0xb69859 0x10913d0 0xf2e4fe 0x9b5d5a0 0x9b5dedf 0x9c825bd 0x9c834dd 0x9c87fbb 0x9bd29d3 0x9bd19d3 0x7bea805 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6f47f22 0x6f4a28a 0x748861c 0x72b6bab 0x72b6cad 0x72b6da5 0x7380162 0x738021b 0x73803ce 0x7380506 0x7380581 0x7380606 0x7380761 0x7435f79 0x74886a5 0x729f1a8 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:28.213 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x4d4b1b4 0x9bbd91f 0x9bd3abb 0x9bd3c83 0x9c6e094 0x9c6e5be 0x9c832d3 0x9c87fbb 0x9bd29d3 0x9bd19d3 0x7bea805 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6f47f22 0x6f4a28a 0x748861c 0x72b6bab 0x72b6cad 0x72b6da5 0x7380162 0x738021b 0x73803ce 0x7380506 0x7380581 0x7380606 0x7380761 0x7435f79 0x74886a5 0x729f1a8 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:28.263 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 32 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0xc6592 /lib64/libasan.so.8+0x15197 0x2a5317b 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9e13a8c 0x9c2a677 0x9c2aeb2 0xecf56f /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47084 2026-02-25T09:25:28.309 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x2a52ab1 0x7da438e 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:28.310 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x2a52ad5 0x7da438e 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:28.310 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x29fe326 0x2a5311c 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:30.427 INFO:teuthology.orchestra.run.trial151.stderr:----------------------------------------------------- 2026-02-25T09:25:30.427 INFO:teuthology.orchestra.run.trial151.stderr:Suppressions used: 2026-02-25T09:25:30.427 INFO:teuthology.orchestra.run.trial151.stderr: count bytes template 2026-02-25T09:25:30.427 INFO:teuthology.orchestra.run.trial151.stderr: 1 8 MallocExtension::Register 2026-02-25T09:25:30.427 INFO:teuthology.orchestra.run.trial151.stderr:----------------------------------------------------- 2026-02-25T09:25:30.427 INFO:teuthology.orchestra.run.trial151.stderr: 2026-02-25T09:25:30.462 DEBUG:teuthology.orchestra.run.trial151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-2 2026-02-25T09:25:30.489 DEBUG:teuthology.orchestra.run.trial151:> sudo MALLOC_CHECK_=3 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-osd --no-mon-config --cluster ceph --mkfs --mkkey -i 3 --monmap /home/ubuntu/cephtest/ceph.monmap 2026-02-25T09:25:30.641 INFO:teuthology.orchestra.run.trial151.stderr:INFO  osd - OSD::main: parsing early config 2026-02-25T09:25:30.696 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:30,696 seastar - Reactor backend: linux-aio 2026-02-25T09:25:30.708 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:30,707 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T09:25:30.708 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:30,707 [shard 0:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T09:25:30.710 INFO:teuthology.orchestra.run.trial151.stderr:==38331==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T09:25:30.765 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:30,763 [shard 0:main] osd - bypassing the config fetch due to --no-mon-config 2026-02-25T09:25:30.765 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:30,763 [shard 0:main] osd - Configure option_name crimson_reactor_task_quota_ms with value : 0.500000 2026-02-25T09:25:30.765 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:30,763 [shard 0:main] osd - Configure option_name crimson_reactor_idle_poll_time_us with value : 200 2026-02-25T09:25:30.765 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:30,763 [shard 0:main] osd - get_early_config: set --thread-affinity 0 --smp 4 2026-02-25T09:25:30.765 INFO:teuthology.orchestra.run.trial151.stderr:DEBUG 2026-02-25 09:25:30,764 [shard 0:main] seastar - reactor::drain 2026-02-25T09:25:32.800 INFO:teuthology.orchestra.run.trial151.stderr:----------------------------------------------------- 2026-02-25T09:25:32.800 INFO:teuthology.orchestra.run.trial151.stderr:Suppressions used: 2026-02-25T09:25:32.801 INFO:teuthology.orchestra.run.trial151.stderr: count bytes template 2026-02-25T09:25:32.801 INFO:teuthology.orchestra.run.trial151.stderr: 1 8 MallocExtension::Register 2026-02-25T09:25:32.801 INFO:teuthology.orchestra.run.trial151.stderr:----------------------------------------------------- 2026-02-25T09:25:32.801 INFO:teuthology.orchestra.run.trial151.stderr: 2026-02-25T09:25:32.820 INFO:teuthology.orchestra.run.trial151.stderr:INFO  osd - OSD::main: early config parsed successfully 2026-02-25T09:25:32.820 INFO:teuthology.orchestra.run.trial151.stderr:INFO  osd - OSD::main: initializing seastar app_template 2026-02-25T09:25:32.867 INFO:teuthology.orchestra.run.trial151.stderr:INFO  osd - OSD::main: registering CLI options 2026-02-25T09:25:32.867 INFO:teuthology.orchestra.run.trial151.stderr:INFO  osd - OSD::main: entering seastar runtime 2026-02-25T09:25:32.875 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:32,874 seastar - Reactor backend: linux-aio 2026-02-25T09:25:32.890 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:32,889 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T09:25:32.891 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:32,890 [shard 2:main] io - IO queue was unable to find a suitable maximum request length, the search was cut-off early at: 16MB 2026-02-25T09:25:32.895 INFO:teuthology.orchestra.run.trial151.stderr:==38329==WARNING: ASan doesn't fully support makecontext/swapcontext functions and may produce false positives in some cases! 2026-02-25T09:25:32.895 INFO:teuthology.orchestra.run.trial151.stderr:INFO 2026-02-25 09:25:32,894 [shard 0:main] osd - OSD::main: seastar runtime started 2026-02-25T09:25:32.953 INFO:teuthology.orchestra.run.trial151.stderr:DEBUG 2026-02-25 09:25:32,952 [shard 0:main] osd - OSD::main: initializing logger output 2026-02-25T09:25:33.034 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1294d2 /lib64/libasan.so.8+0x1297ad /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x16507 0x77dcacb 0x7bea137 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6faf42c 0x6fb322d 0x6fb8dd7 0x748b7b0 0x72b8d6e 0x72b8e70 0x72b8f68 0x738168e 0x7381747 0x73818fa 0x7381a32 0x7381aad 0x7381b32 0x7381c8d 0x7436297 0x748b839 0x72a0671 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:33.034 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1297e4 /lib64/libasan.so.8+0xc65d1 /lib64/libasan.so.8+0x15197 0x1125f6c 0xf2e4f3 0x9b5bb21 0x9b5bfeb 0x9b5c25e 0x9baf7e1 0x9bafda2 0x9baffdb 0x9bb06bb 0x9c83910 0x9c87fbb 0x9bd29d3 0x9bd19d3 0x7bea805 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6faf42c 0x6fb322d 0x6fb8dd7 0x748b7b0 0x72b8d6e 0x72b8e70 0x72b8f68 0x738168e 0x7381747 0x73818fa 0x7381a32 0x7381aad 0x7381b32 0x7381c8d 0x7436297 0x748b839 0x72a0671 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:33.034 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0xb6e6da 0x109134d 0xf2e4fe 0x9b5bcee 0x9b5bfeb 0x9b5c25e 0x9baf7e1 0x9bafda2 0x9baffdb 0x9bb06bb 0x9c83910 0x9c87fbb 0x9bd29d3 0x9bd19d3 0x7bea805 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6faf42c 0x6fb322d 0x6fb8dd7 0x748b7b0 0x72b8d6e 0x72b8e70 0x72b8f68 0x738168e 0x7381747 0x73818fa 0x7381a32 0x7381aad 0x7381b32 0x7381c8d 0x7436297 0x748b839 0x72a0671 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:33.131 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x151f5 0x2a5317b 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9e13a8c 0x9c2a677 0x9c2aeb2 0xecf56f /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47084 2026-02-25T09:25:33.172 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x2a5303d 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:33.172 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libc.so.6+0x8cc0c /lib64/libasan.so.8+0x10069c /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2a5317b 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:33.172 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x2a524cd 0x7da4381 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:33.299 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 32 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x77dcaa1 0x7bea137 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6f47f22 0x6f4a28a 0x748861c 0x72b6bab 0x72b6cad 0x72b6da5 0x7380162 0x738021b 0x73803ce 0x7380506 0x7380581 0x7380606 0x7380761 0x7435f79 0x74886a5 0x729f1a8 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:33.299 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 32 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x10914eb 0xf2e4fe 0x9b5bb21 0x9b5bfeb 0x9b5c25e 0x9bb0988 0x9bb0ed6 0x9bb10d5 0x9c79155 0x9c798b0 0x9c83b42 0x9c87fbb 0x9bd29d3 0x9bd19d3 0x7bea805 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6f47f22 0x6f4a28a 0x748861c 0x72b6bab 0x72b6cad 0x72b6da5 0x7380162 0x738021b 0x73803ce 0x7380506 0x7380581 0x7380606 0x7380761 0x7435f79 0x74886a5 0x729f1a8 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:33.299 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 32 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1589b 0x35eae1b 0x35eaf9b 0x3606e18 0x9d010a1 0x9d011a9 0x9d54a30 0x9c6d8c5 0x9c6dc57 0x9c6de62 0x9c6e19e 0x9c6e5be 0x9c832d3 0x9c87fbb 0x9bd29d3 0x9bd19d3 0x7bea805 0x7bf4374 0x7a8710d 0x7969441 0x712fea1 0x6f47f22 0x6f4a28a 0x748861c 0x72b6bab 0x72b6cad 0x72b6da5 0x7380162 0x738021b 0x73803ce 0x7380506 0x7380581 0x7380606 0x7380761 0x7435f79 0x74886a5 0x729f1a8 0x72b6f22 0x73cec1c 0x7443dd0 0x7443e4b 0x7443ec6 0x74f762e 0x74f7796 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:33.382 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 65 ms on shard 0, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1521e 0x2a5317b 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9e13a8c 0x9c2a677 0x9c2aeb2 0xecf56f /lib64/libc.so.6+0x2a60f /lib64/libc.so.6+0x2a6bf 0xb47084 2026-02-25T09:25:33.397 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 65 ms on shard 1, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x1006a8 /lib64/libasan.so.8+0xc65b9 /lib64/libasan.so.8+0x15197 0x2a5317b 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:33.397 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 65 ms on shard 2, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f 0x2a5254b 0x7da4381 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:33.397 INFO:teuthology.orchestra.run.trial151.stderr:Reactor stalled for 65 ms on shard 3, in scheduling group main. Backtrace: /lib64/libasan.so.8+0x54e93 0x9ed3ee6 0x9ced91b 0x9d38485 0x9d3c7fc 0x9d3ca6a 0x9d3cbb2 0x9d3d071 /lib64/libc.so.6+0x3fc2f /lib64/libasan.so.8+0x15187 0x2a5317b 0x7da4353 0x7e54afc 0x7e555f3 0x7de7c85 0x78845d9 0x78852ce 0x79d54ee 0x783d4f8 0x78ad89c 0x78f90df 0x78f9358 0x79cb418 0x79cb577 0x79d5651 0x786fca4 0x79b023e 0x79b02ce 0x79b0372 0x73fe504 0x741c921 0x7420b5f 0x7552814 0x9e0b8cd 0x9cf6cfe 0x9dce597 0x9e1226f 0x9f5f5ca 0x9f5fc46 0x26ed657 0xa5ac0e4 /lib64/libasan.so.8+0x2a549 /lib64/libc.so.6+0x8b2f9 /lib64/libc.so.6+0x1103ff 2026-02-25T09:25:35.552 INFO:teuthology.orchestra.run.trial151.stderr:----------------------------------------------------- 2026-02-25T09:25:35.552 INFO:teuthology.orchestra.run.trial151.stderr:Suppressions used: 2026-02-25T09:25:35.553 INFO:teuthology.orchestra.run.trial151.stderr: count bytes template 2026-02-25T09:25:35.553 INFO:teuthology.orchestra.run.trial151.stderr: 1 8 MallocExtension::Register 2026-02-25T09:25:35.553 INFO:teuthology.orchestra.run.trial151.stderr:----------------------------------------------------- 2026-02-25T09:25:35.553 INFO:teuthology.orchestra.run.trial151.stderr: 2026-02-25T09:25:35.588 DEBUG:teuthology.orchestra.run.trial151:> sudo chown -R ceph:ceph /var/lib/ceph/osd/ceph-3 2026-02-25T09:25:35.613 INFO:tasks.ceph:Reading keys from all nodes... 2026-02-25T09:25:35.614 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:25:35.614 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/var/lib/ceph/osd/ceph-0/keyring of=/dev/stdout 2026-02-25T09:25:35.640 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:25:35.640 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/var/lib/ceph/osd/ceph-1/keyring of=/dev/stdout 2026-02-25T09:25:35.705 DEBUG:teuthology.orchestra.run.trial151:> set -ex 2026-02-25T09:25:35.705 DEBUG:teuthology.orchestra.run.trial151:> sudo dd if=/var/lib/ceph/mgr/ceph-x/keyring of=/dev/stdout 2026-02-25T09:25:35.732 DEBUG:teuthology.orchestra.run.trial151:> set -ex 2026-02-25T09:25:35.732 DEBUG:teuthology.orchestra.run.trial151:> sudo dd if=/var/lib/ceph/osd/ceph-2/keyring of=/dev/stdout 2026-02-25T09:25:35.796 DEBUG:teuthology.orchestra.run.trial151:> set -ex 2026-02-25T09:25:35.796 DEBUG:teuthology.orchestra.run.trial151:> sudo dd if=/var/lib/ceph/osd/ceph-3/keyring of=/dev/stdout 2026-02-25T09:25:35.861 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:25:35.861 DEBUG:teuthology.orchestra.run.trial094:> dd if=/etc/ceph/ceph.client.0.keyring of=/dev/stdout 2026-02-25T09:25:35.879 DEBUG:teuthology.orchestra.run.trial151:> set -ex 2026-02-25T09:25:35.879 DEBUG:teuthology.orchestra.run.trial151:> dd if=/etc/ceph/ceph.client.1.keyring of=/dev/stdout 2026-02-25T09:25:35.917 INFO:tasks.ceph:Adding keys to all mons... 2026-02-25T09:25:35.917 DEBUG:teuthology.orchestra.run.trial094:> sudo tee -a /etc/ceph/ceph.keyring 2026-02-25T09:25:35.941 INFO:teuthology.orchestra.run.trial094.stdout:[osd.0] 2026-02-25T09:25:35.941 INFO:teuthology.orchestra.run.trial094.stdout: key = AQD7v55pv7JQNRAA5lgzdE2GZ2PBjb/eQfvzLA== 2026-02-25T09:25:35.942 INFO:teuthology.orchestra.run.trial094.stdout:[osd.1] 2026-02-25T09:25:35.942 INFO:teuthology.orchestra.run.trial094.stdout: key = AQABwJ5p4Rl6CBAATtewr63/CqMH6uEKexWKlA== 2026-02-25T09:25:35.942 INFO:teuthology.orchestra.run.trial094.stdout:[mgr.x] 2026-02-25T09:25:35.942 INFO:teuthology.orchestra.run.trial094.stdout: key = AQD3v55pzLBmJhAA5H7QrX18MgtAXRXxiDJJgA== 2026-02-25T09:25:35.942 INFO:teuthology.orchestra.run.trial094.stdout:[osd.2] 2026-02-25T09:25:35.942 INFO:teuthology.orchestra.run.trial094.stdout: key = AQAHwJ5p6OrgMxAAOoMU8zuSbAsDUD6LiPbwMg== 2026-02-25T09:25:35.942 INFO:teuthology.orchestra.run.trial094.stdout:[osd.3] 2026-02-25T09:25:35.942 INFO:teuthology.orchestra.run.trial094.stdout: key = AQAMwJ5p1pkDORAA8sAsULpY7hgi8ggwKkiqGQ== 2026-02-25T09:25:35.942 INFO:teuthology.orchestra.run.trial094.stdout:[client.0] 2026-02-25T09:25:35.942 INFO:teuthology.orchestra.run.trial094.stdout: key = AQD3v55pKqQqKRAApNksg8uYNMite652JOo1Sw== 2026-02-25T09:25:35.942 INFO:teuthology.orchestra.run.trial094.stdout:[client.1] 2026-02-25T09:25:35.942 INFO:teuthology.orchestra.run.trial094.stdout: key = AQD3v55pl4BSLBAA/e2nKcIB+PDSr7pJUbzFeg== 2026-02-25T09:25:35.944 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.0 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T09:25:36.030 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.1 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T09:25:36.079 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=mgr.x --cap mon 'allow profile mgr' --cap osd 'allow *' --cap mds 'allow *' 2026-02-25T09:25:36.127 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.2 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T09:25:36.175 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=osd.3 --cap mon 'allow profile osd' --cap mgr 'allow profile osd' --cap osd 'allow *' 2026-02-25T09:25:36.224 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.0 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-25T09:25:36.275 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-authtool /etc/ceph/ceph.keyring --name=client.1 --cap mon 'allow rw' --cap mgr 'allow r' --cap osd 'allow rwx' --cap mds allow 2026-02-25T09:25:36.324 INFO:tasks.ceph:Running mkfs on mon nodes... 2026-02-25T09:25:36.324 DEBUG:teuthology.orchestra.run.trial094:> sudo mkdir -p /var/lib/ceph/mon/ceph-a 2026-02-25T09:25:36.349 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph-mon --cluster ceph --mkfs -i a --monmap /home/ubuntu/cephtest/ceph.monmap --keyring /etc/ceph/ceph.keyring 2026-02-25T09:25:36.433 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:36.431+0000 7fd4c2687c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:36.433 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:36.431+0000 7fd4c2687c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:36.445 DEBUG:teuthology.orchestra.run.trial094:> sudo chown -R ceph:ceph /var/lib/ceph/mon/ceph-a 2026-02-25T09:25:36.470 DEBUG:teuthology.orchestra.run.trial094:> rm -- /home/ubuntu/cephtest/ceph.monmap 2026-02-25T09:25:36.526 INFO:tasks.ceph:Starting mon daemons in cluster ceph... 2026-02-25T09:25:36.527 INFO:tasks.ceph.mon.a:Restarting daemon 2026-02-25T09:25:36.527 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mon -f --cluster ceph -i a 2026-02-25T09:25:36.568 INFO:tasks.ceph.mon.a:Started 2026-02-25T09:25:36.569 INFO:tasks.ceph:Starting mgr daemons in cluster ceph... 2026-02-25T09:25:36.569 INFO:tasks.ceph.mgr.x:Restarting daemon 2026-02-25T09:25:36.569 DEBUG:teuthology.orchestra.run.trial151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-mgr -f --cluster ceph -i x 2026-02-25T09:25:36.573 INFO:tasks.ceph.mgr.x:Started 2026-02-25T09:25:36.573 DEBUG:tasks.ceph:set 0 configs 2026-02-25T09:25:36.573 DEBUG:teuthology.orchestra.run.trial094:> sudo ceph --cluster ceph config dump 2026-02-25T09:25:36.631 INFO:tasks.ceph.mon.a.trial094.stderr:2026-02-25T09:25:36.629+0000 7fd652032c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:36.631 INFO:tasks.ceph.mon.a.trial094.stderr:2026-02-25T09:25:36.629+0000 7fd652032c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:36.631 INFO:tasks.ceph.mon.a.trial094.stderr:2026-02-25T09:25:36.629+0000 7fd652032c00 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:36.633 INFO:tasks.ceph.mgr.x.trial151.stderr:2026-02-25T09:25:36.632+0000 7fb2a6043fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:36.647 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:36.643+0000 7fc4c93d3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:36.650 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:36.648+0000 7fc4c93d3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:36.657 INFO:tasks.ceph.mgr.x.trial151.stderr:2026-02-25T09:25:36.657+0000 7fb2a6043fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:36.658 INFO:tasks.ceph.mgr.x.trial151.stderr:2026-02-25T09:25:36.657+0000 7fb2a6043fc0 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:36.682 INFO:teuthology.orchestra.run.trial094.stdout:WHO MASK LEVEL OPTION VALUE RO 2026-02-25T09:25:36.691 INFO:tasks.ceph:Setting crush tunables to default 2026-02-25T09:25:36.691 DEBUG:teuthology.orchestra.run.trial094:> sudo ceph --cluster ceph osd crush tunables default 2026-02-25T09:25:36.766 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:36.761+0000 7fe52650b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:36.769 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:36.767+0000 7fe52650b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:36.804 INFO:teuthology.orchestra.run.trial094.stderr:adjusted tunables profile to default 2026-02-25T09:25:36.812 INFO:tasks.ceph:check_enable_crimson: True 2026-02-25T09:25:36.812 INFO:tasks.ceph:check_enable_crimson: setting set-allow-crimson 2026-02-25T09:25:36.812 DEBUG:teuthology.orchestra.run.trial094:> sudo ceph --cluster ceph osd set-allow-crimson --yes-i-really-mean-it 2026-02-25T09:25:36.886 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:36.882+0000 7fd614ba4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:36.889 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:36.888+0000 7fd614ba4640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:36.928 INFO:tasks.ceph.mon.a.trial094.stderr:2026-02-25T09:25:36.927+0000 7fd6493f1640 -1 WARNING: experimental feature 'crimson' is enabled 2026-02-25T09:25:36.929 INFO:tasks.ceph.mon.a.trial094.stderr:Please be aware that this feature is experimental, untested, 2026-02-25T09:25:36.929 INFO:tasks.ceph.mon.a.trial094.stderr:unsupported, and may result in data corruption, data loss, 2026-02-25T09:25:36.929 INFO:tasks.ceph.mon.a.trial094.stderr:and/or irreparable damage to your cluster. Do not use 2026-02-25T09:25:36.929 INFO:tasks.ceph.mon.a.trial094.stderr:feature with important data. 2026-02-25T09:25:36.929 INFO:tasks.ceph.mon.a.trial094.stderr: 2026-02-25T09:25:37.488 INFO:tasks.ceph.mgr.x.trial151.stderr:/usr/lib64/python3.9/site-packages/scipy/__init__.py:73: UserWarning: NumPy was imported from a Python sub-interpreter but NumPy does not properly support sub-interpreters. This will likely work for most users but might cause hard to track down issues or subtle bugs. A common user of the rare sub-interpreter feature is wsgi which also allows single-interpreter mode. 2026-02-25T09:25:37.488 INFO:tasks.ceph.mgr.x.trial151.stderr:Improvements in the case of bugs are welcome, but is not on the NumPy roadmap, and full support may require significant effort to achieve. 2026-02-25T09:25:37.488 INFO:tasks.ceph.mgr.x.trial151.stderr: from numpy import show_config as show_numpy_config 2026-02-25T09:25:37.815 INFO:tasks.ceph:Starting osd daemons in cluster ceph... 2026-02-25T09:25:37.816 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:25:37.816 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/var/lib/ceph/osd/ceph-0/fsid of=/dev/stdout 2026-02-25T09:25:37.842 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:25:37.842 DEBUG:teuthology.orchestra.run.trial094:> sudo dd if=/var/lib/ceph/osd/ceph-1/fsid of=/dev/stdout 2026-02-25T09:25:37.906 DEBUG:teuthology.orchestra.run.trial151:> set -ex 2026-02-25T09:25:37.906 DEBUG:teuthology.orchestra.run.trial151:> sudo dd if=/var/lib/ceph/osd/ceph-2/fsid of=/dev/stdout 2026-02-25T09:25:37.932 DEBUG:teuthology.orchestra.run.trial151:> set -ex 2026-02-25T09:25:37.932 DEBUG:teuthology.orchestra.run.trial151:> sudo dd if=/var/lib/ceph/osd/ceph-3/fsid of=/dev/stdout 2026-02-25T09:25:37.996 DEBUG:teuthology.orchestra.run.trial151:> sudo ceph --cluster ceph osd new 84b58e5b-579c-49b7-9672-6379199b2e0c 0 2026-02-25T09:25:38.108 INFO:teuthology.orchestra.run.trial151.stderr:2026-02-25T09:25:38.105+0000 7f522c051640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:38.111 INFO:teuthology.orchestra.run.trial151.stderr:2026-02-25T09:25:38.110+0000 7f522c051640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:38.148 INFO:teuthology.orchestra.run.trial151.stdout:0 2026-02-25T09:25:38.157 DEBUG:teuthology.orchestra.run.trial151:> sudo ceph --cluster ceph osd new 47831adf-49cb-4d5d-af8a-f010094da3b4 1 2026-02-25T09:25:38.230 INFO:teuthology.orchestra.run.trial151.stderr:2026-02-25T09:25:38.226+0000 7fc9439b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:38.231 INFO:teuthology.orchestra.run.trial151.stderr:2026-02-25T09:25:38.231+0000 7fc9439b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:38.269 INFO:teuthology.orchestra.run.trial151.stdout:1 2026-02-25T09:25:38.276 DEBUG:teuthology.orchestra.run.trial151:> sudo ceph --cluster ceph osd new 764573ae-9128-448c-a36d-b40e3e9ca850 2 2026-02-25T09:25:38.350 INFO:teuthology.orchestra.run.trial151.stderr:2026-02-25T09:25:38.346+0000 7ffbab174640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:38.352 INFO:teuthology.orchestra.run.trial151.stderr:2026-02-25T09:25:38.351+0000 7ffbab174640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:38.389 INFO:teuthology.orchestra.run.trial151.stdout:2 2026-02-25T09:25:38.397 DEBUG:teuthology.orchestra.run.trial151:> sudo ceph --cluster ceph osd new 272a2704-6263-47ab-87f5-4c0dc76ddfac 3 2026-02-25T09:25:38.471 INFO:teuthology.orchestra.run.trial151.stderr:2026-02-25T09:25:38.468+0000 7fa136a8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:38.473 INFO:teuthology.orchestra.run.trial151.stderr:2026-02-25T09:25:38.472+0000 7fa136a8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:38.511 INFO:teuthology.orchestra.run.trial151.stdout:3 2026-02-25T09:25:38.519 DEBUG:teuthology.orchestra.run.trial094:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.0.log 2026-02-25T09:25:38.549 INFO:tasks.ceph.osd.0:Restarting daemon 2026-02-25T09:25:38.549 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 0 2>> /var/log/ceph/ceph-osd.0.log 2026-02-25T09:25:38.592 INFO:tasks.ceph.osd.0:Started 2026-02-25T09:25:38.592 DEBUG:teuthology.orchestra.run.trial094:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.1.log 2026-02-25T09:25:38.621 INFO:tasks.ceph.osd.1:Restarting daemon 2026-02-25T09:25:38.622 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 1 2>> /var/log/ceph/ceph-osd.1.log 2026-02-25T09:25:38.665 INFO:tasks.ceph.osd.1:Started 2026-02-25T09:25:38.665 DEBUG:teuthology.orchestra.run.trial151:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.2.log 2026-02-25T09:25:38.693 INFO:tasks.ceph.osd.2:Restarting daemon 2026-02-25T09:25:38.693 DEBUG:teuthology.orchestra.run.trial151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 2 2>> /var/log/ceph/ceph-osd.2.log 2026-02-25T09:25:38.735 INFO:tasks.ceph.osd.2:Started 2026-02-25T09:25:38.735 DEBUG:teuthology.orchestra.run.trial151:> sudo install -b -o ubuntu -g ubuntu /dev/null /var/log/ceph/ceph-osd.3.log 2026-02-25T09:25:38.763 INFO:tasks.ceph.osd.3:Restarting daemon 2026-02-25T09:25:38.763 DEBUG:teuthology.orchestra.run.trial151:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage daemon-helper kill ceph-osd -f --cluster ceph -i 3 2>> /var/log/ceph/ceph-osd.3.log 2026-02-25T09:25:38.805 INFO:tasks.ceph.osd.3:Started 2026-02-25T09:25:38.805 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T09:25:38.880 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:38.877+0000 7f7dd199c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:38.883 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:38.882+0000 7f7dd199c640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:38.915 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:38.916 INFO:teuthology.orchestra.run.trial094.stdout:{"epoch":7,"fsid":"c0375da7-c136-43b7-8240-9e8e7e46c4e5","created":"2026-02-25T09:25:36.641907+0000","modified":"2026-02-25T09:25:38.510236+0000","last_up_change":"0.000000","last_in_change":"2026-02-25T09:25:38.510236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"84b58e5b-579c-49b7-9672-6379199b2e0c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"47831adf-49cb-4d5d-af8a-f010094da3b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"764573ae-9128-448c-a36d-b40e3e9ca850","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"272a2704-6263-47ab-87f5-4c0dc76ddfac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T09:25:38.924 INFO:tasks.ceph.ceph_manager.ceph:[] 2026-02-25T09:25:38.924 INFO:tasks.ceph:Waiting for OSDs to come up 2026-02-25T09:25:39.327 DEBUG:teuthology.orchestra.run.trial094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-25T09:25:39.394 INFO:teuthology.misc.health.trial094.stderr:2026-02-25T09:25:39.390+0000 7f0202374640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:39.396 INFO:teuthology.misc.health.trial094.stderr:2026-02-25T09:25:39.395+0000 7f0202374640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:39.429 INFO:teuthology.misc.health.trial094.stdout: 2026-02-25T09:25:39.429 INFO:teuthology.misc.health.trial094.stdout:{"epoch":7,"fsid":"c0375da7-c136-43b7-8240-9e8e7e46c4e5","created":"2026-02-25T09:25:36.641907+0000","modified":"2026-02-25T09:25:38.510236+0000","last_up_change":"0.000000","last_in_change":"2026-02-25T09:25:38.510236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":2,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"84b58e5b-579c-49b7-9672-6379199b2e0c","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":1,"uuid":"47831adf-49cb-4d5d-af8a-f010094da3b4","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":2,"uuid":"764573ae-9128-448c-a36d-b40e3e9ca850","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]},{"osd":3,"uuid":"272a2704-6263-47ab-87f5-4c0dc76ddfac","up":0,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":0,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[]},"cluster_addrs":{"addrvec":[]},"heartbeat_back_addrs":{"addrvec":[]},"heartbeat_front_addrs":{"addrvec":[]},"public_addr":"(unrecognized address family 0)/0","cluster_addr":"(unrecognized address family 0)/0","heartbeat_back_addr":"(unrecognized address family 0)/0","heartbeat_front_addr":"(unrecognized address family 0)/0","state":["exists","new"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":0,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T09:25:39.435 DEBUG:teuthology.misc:0 of 4 OSDs are up 2026-02-25T09:25:39.675 INFO:tasks.ceph.mgr.x.trial151.stderr:2026-02-25T09:25:39.674+0000 7fb1c2be1640 -1 mgr.server handle_report got status from non-daemon mon.a 2026-02-25T09:25:45.838 DEBUG:teuthology.orchestra.run.trial094:> adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage ceph --cluster ceph osd dump --format=json 2026-02-25T09:25:45.909 INFO:teuthology.misc.health.trial094.stderr:2026-02-25T09:25:45.905+0000 7f31270cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:45.911 INFO:teuthology.misc.health.trial094.stderr:2026-02-25T09:25:45.910+0000 7f31270cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:46.027 INFO:teuthology.misc.health.trial094.stdout: 2026-02-25T09:25:46.027 INFO:teuthology.misc.health.trial094.stdout:{"epoch":11,"fsid":"c0375da7-c136-43b7-8240-9e8e7e46c4e5","created":"2026-02-25T09:25:36.641907+0000","modified":"2026-02-25T09:25:45.931020+0000","last_up_change":"2026-02-25T09:25:45.931020+0000","last_in_change":"2026-02-25T09:25:38.510236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":0,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[],"osds":[{"osd":0,"uuid":"84b58e5b-579c-49b7-9672-6379199b2e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6804","nonce":889111984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6803","nonce":889111984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6805","nonce":889111984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6807","nonce":889111984}]},"public_addr":"10.20.193.94:6804/889111984","cluster_addr":"10.20.193.94:6803/889111984","heartbeat_back_addr":"10.20.193.94:6805/889111984","heartbeat_front_addr":"10.20.193.94:6807/889111984","state":["exists","up"]},{"osd":1,"uuid":"47831adf-49cb-4d5d-af8a-f010094da3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6802","nonce":3409185002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6801","nonce":3409185002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6808","nonce":3409185002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6806","nonce":3409185002}]},"public_addr":"10.20.193.94:6802/3409185002","cluster_addr":"10.20.193.94:6801/3409185002","heartbeat_back_addr":"10.20.193.94:6808/3409185002","heartbeat_front_addr":"10.20.193.94:6806/3409185002","state":["exists","up"]},{"osd":2,"uuid":"764573ae-9128-448c-a36d-b40e3e9ca850","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6805","nonce":2663299575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6806","nonce":2663299575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6815","nonce":2663299575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6816","nonce":2663299575}]},"public_addr":"10.20.193.151:6805/2663299575","cluster_addr":"10.20.193.151:6806/2663299575","heartbeat_back_addr":"10.20.193.151:6815/2663299575","heartbeat_front_addr":"10.20.193.151:6816/2663299575","state":["exists","up"]},{"osd":3,"uuid":"272a2704-6263-47ab-87f5-4c0dc76ddfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6803","nonce":4248319012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6802","nonce":4248319012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6808","nonce":4248319012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6809","nonce":4248319012}]},"public_addr":"10.20.193.151:6803/4248319012","cluster_addr":"10.20.193.151:6802/4248319012","heartbeat_back_addr":"10.20.193.151:6808/4248319012","heartbeat_front_addr":"10.20.193.151:6809/4248319012","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T09:25:46.037 DEBUG:teuthology.misc:4 of 4 OSDs are up 2026-02-25T09:25:46.037 INFO:tasks.ceph:Creating RBD pool 2026-02-25T09:25:46.037 DEBUG:teuthology.orchestra.run.trial094:> sudo ceph --cluster ceph osd pool create rbd 8 2026-02-25T09:25:46.112 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:46.106+0000 7fc0704cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:46.115 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:46.113+0000 7fc0704cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:46.938 INFO:teuthology.orchestra.run.trial094.stderr:pool 'rbd' created 2026-02-25T09:25:46.951 DEBUG:teuthology.orchestra.run.trial094:> rbd --cluster ceph pool init rbd 2026-02-25T09:25:46.992 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:46.990+0000 7f81fdee2240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:46.992 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:46.990+0000 7f81fdee2240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:46.995 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:46.994+0000 7f81fdee2240 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:49.976 INFO:tasks.ceph:Starting mds daemons in cluster ceph... 2026-02-25T09:25:49.977 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph config log 1 --format=json 2026-02-25T09:25:49.977 INFO:tasks.daemonwatchdog.daemon_watchdog:watchdog starting 2026-02-25T09:25:50.093 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:50.088+0000 7f19e350b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:50.095 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:50.094+0000 7f19e350b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:50.206 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:50.215 INFO:teuthology.orchestra.run.trial094.stdout:[{"version":1,"timestamp":"0.000000","name":"","changes":[]}] 2026-02-25T09:25:50.216 INFO:tasks.ceph_manager:config epoch is 1 2026-02-25T09:25:50.216 INFO:tasks.ceph:Waiting until ceph daemons up and pgs clean... 2026-02-25T09:25:50.216 INFO:tasks.ceph.ceph_manager.ceph:waiting for mgr available 2026-02-25T09:25:50.216 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph mgr dump --format=json 2026-02-25T09:25:50.291 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:50.287+0000 7f11b33d3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:50.293 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:50.292+0000 7f11b33d3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:50.442 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:50.452 INFO:teuthology.orchestra.run.trial094.stdout:{"epoch":5,"flags":0,"active_gid":4102,"active_name":"x","active_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6800","nonce":4165898860},{"type":"v1","addr":"10.20.193.151:6801","nonce":4165898860}]},"active_addr":"10.20.193.151:6801/4165898860","active_change":"2026-02-25T09:25:38.667545+0000","active_mgr_features":4544132024016699391,"available":true,"standbys":[],"modules":["iostat","nfs"],"available_modules":[{"name":"alerts","can_run":true,"error_string":"","module_options":{"interval":{"name":"interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"How frequently to reexamine health status","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"smtp_destination":{"name":"smtp_destination","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Email address to send alerts to, use commas to separate multiple","long_desc":"","tags":[],"see_also":[]},"smtp_from_name":{"name":"smtp_from_name","type":"str","level":"advanced","flags":1,"default_value":"Ceph","min":"","max":"","enum_allowed":[],"desc":"Email From: name","long_desc":"","tags":[],"see_also":[]},"smtp_host":{"name":"smtp_host","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_password":{"name":"smtp_password","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Password to authenticate with","long_desc":"","tags":[],"see_also":[]},"smtp_port":{"name":"smtp_port","type":"int","level":"advanced","flags":1,"default_value":"465","min":"","max":"","enum_allowed":[],"desc":"SMTP port","long_desc":"","tags":[],"see_also":[]},"smtp_sender":{"name":"smtp_sender","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"SMTP envelope sender","long_desc":"","tags":[],"see_also":[]},"smtp_ssl":{"name":"smtp_ssl","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Use SSL to connect to SMTP server","long_desc":"","tags":[],"see_also":[]},"smtp_user":{"name":"smtp_user","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"User to authenticate as","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"balancer","can_run":true,"error_string":"","module_options":{"active":{"name":"active","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically balance PGs across cluster","long_desc":"","tags":[],"see_also":[]},"begin_time":{"name":"begin_time","type":"str","level":"advanced","flags":1,"default_value":"0000","min":"","max":"","enum_allowed":[],"desc":"beginning time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"begin_weekday":{"name":"begin_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to this day of the week or later","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"crush_compat_max_iterations":{"name":"crush_compat_max_iterations","type":"uint","level":"advanced","flags":1,"default_value":"25","min":"1","max":"250","enum_allowed":[],"desc":"maximum number of iterations to attempt optimization","long_desc":"","tags":[],"see_also":[]},"crush_compat_metrics":{"name":"crush_compat_metrics","type":"str","level":"advanced","flags":1,"default_value":"pgs,objects,bytes","min":"","max":"","enum_allowed":[],"desc":"metrics with which to calculate OSD utilization","long_desc":"Value is a list of one or more of \"pgs\", \"objects\", or \"bytes\", and indicates which metrics to use to balance utilization.","tags":[],"see_also":[]},"crush_compat_step":{"name":"crush_compat_step","type":"float","level":"advanced","flags":1,"default_value":"0.5","min":"0.001","max":"0.999","enum_allowed":[],"desc":"aggressiveness of optimization","long_desc":".99 is very aggressive, .01 is less aggressive","tags":[],"see_also":[]},"end_time":{"name":"end_time","type":"str","level":"advanced","flags":1,"default_value":"2359","min":"","max":"","enum_allowed":[],"desc":"ending time of day to automatically balance","long_desc":"This is a time of day in the format HHMM.","tags":[],"see_also":[]},"end_weekday":{"name":"end_weekday","type":"uint","level":"advanced","flags":1,"default_value":"0","min":"0","max":"6","enum_allowed":[],"desc":"Restrict automatic balancing to days of the week earlier than this","long_desc":"0 = Sunday, 1 = Monday, etc.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_score":{"name":"min_score","type":"float","level":"advanced","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"minimum score, below which no optimization is attempted","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":1,"default_value":"upmap","min":"","max":"","enum_allowed":["crush-compat","none","read","upmap","upmap-read"],"desc":"Balancer mode","long_desc":"","tags":[],"see_also":[]},"pool_ids":{"name":"pool_ids","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"pools which the automatic balancing will be limited to","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and attempt optimization","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_pg_upmap_activity":{"name":"update_pg_upmap_activity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Updates pg_upmap activity stats to be used in `balancer status detail`","long_desc":"","tags":[],"see_also":[]},"upmap_max_deviation":{"name":"upmap_max_deviation","type":"int","level":"advanced","flags":1,"default_value":"5","min":"1","max":"","enum_allowed":[],"desc":"deviation below which no optimization is attempted","long_desc":"If the number of PGs are within this count then no optimization is attempted","tags":[],"see_also":[]},"upmap_max_optimizations":{"name":"upmap_max_optimizations","type":"uint","level":"advanced","flags":1,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"maximum upmap optimizations to make per attempt","long_desc":"","tags":[],"see_also":[]}}},{"name":"cephadm","can_run":true,"error_string":"","module_options":{"agent_down_multiplier":{"name":"agent_down_multiplier","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"","max":"","enum_allowed":[],"desc":"Multiplied by agent refresh rate to calculate how long agent must not report before being marked down","long_desc":"","tags":[],"see_also":[]},"agent_refresh_rate":{"name":"agent_refresh_rate","type":"secs","level":"advanced","flags":0,"default_value":"20","min":"","max":"","enum_allowed":[],"desc":"How often the agent on each host will gather and send metadata","long_desc":"","tags":[],"see_also":[]},"agent_starting_port":{"name":"agent_starting_port","type":"int","level":"advanced","flags":0,"default_value":"4721","min":"","max":"","enum_allowed":[],"desc":"The first TCP port the agent will try to bind to. Up to 1000 subsequent ports will be attempted if this port cannot be bound","long_desc":"","tags":[],"see_also":[]},"allow_ptrace":{"name":"allow_ptrace","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow SYS_PTRACE capability on ceph containers","long_desc":"The SYS_PTRACE capability is needed to attach to a process with gdb or strace. Enabling this option can facilitate debugging daemons that encounter runtime problems.","tags":[],"see_also":[]},"autotune_interval":{"name":"autotune_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to autotune daemon memory","long_desc":"","tags":[],"see_also":[]},"autotune_memory_target_ratio":{"name":"autotune_memory_target_ratio","type":"float","level":"advanced","flags":0,"default_value":"0.7","min":"","max":"","enum_allowed":[],"desc":"Fraction of total system memory to divide among autotuned daemons. Converged systems hosting both Ceph and compute should set a lower value; systems with more than 10 GiB of memory per OSD may benefit from a slightly larger value so that the abundance of RAM can be fully utilized","long_desc":"","tags":[],"see_also":[]},"cephadm_log_destination":{"name":"cephadm_log_destination","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":["file","file,syslog","syslog"],"desc":"Destination for cephadm command persistent logging","long_desc":"","tags":[],"see_also":[]},"certificate_automated_rotation_enabled":{"name":"certificate_automated_rotation_enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"This flag controls whether cephadm automatically rotates certificates upon expiration.","long_desc":"","tags":[],"see_also":[]},"certificate_check_debug_mode":{"name":"certificate_check_debug_mode","type":"bool","level":"dev","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"FOR TESTING ONLY: This flag forces the certificate check instead of waiting for certificate_check_period.","long_desc":"","tags":[],"see_also":[]},"certificate_check_period":{"name":"certificate_check_period","type":"int","level":"advanced","flags":0,"default_value":"1","min":"0","max":"30","enum_allowed":[],"desc":"Specifies how often (in days) the certificate should be checked for validity.","long_desc":"","tags":[],"see_also":[]},"certificate_duration_days":{"name":"certificate_duration_days","type":"int","level":"advanced","flags":0,"default_value":"1095","min":"90","max":"3650","enum_allowed":[],"desc":"Specifies the duration of self-signed certificates generated and signed by the cephadm root CA","long_desc":"","tags":[],"see_also":[]},"certificate_renewal_threshold_days":{"name":"certificate_renewal_threshold_days","type":"int","level":"advanced","flags":0,"default_value":"30","min":"10","max":"90","enum_allowed":[],"desc":"Specifies the lead time in days to initiate certificate renewal before expiration.","long_desc":"","tags":[],"see_also":[]},"cgroups_split":{"name":"cgroups_split","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Pass --cgroups=split when cephadm creates containers (currently Podman only)","long_desc":"","tags":[],"see_also":[]},"config_checks_enabled":{"name":"config_checks_enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable or disable cephadm configuration analysis","long_desc":"","tags":[],"see_also":[]},"config_dashboard":{"name":"config_dashboard","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Manage configs like API endpoints in Dashboard.","long_desc":"","tags":[],"see_also":[]},"container_image_alertmanager":{"name":"container_image_alertmanager","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/alertmanager:v0.28.1","min":"","max":"","enum_allowed":[],"desc":"Alertmanager container image","long_desc":"","tags":[],"see_also":[]},"container_image_alloy":{"name":"container_image_alloy","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/alloy:latest","min":"","max":"","enum_allowed":[],"desc":"Alloy container image","long_desc":"","tags":[],"see_also":[]},"container_image_base":{"name":"container_image_base","type":"str","level":"advanced","flags":1,"default_value":"quay.io/ceph/ceph","min":"","max":"","enum_allowed":[],"desc":"Container image name, without the tag","long_desc":"","tags":[],"see_also":[]},"container_image_elasticsearch":{"name":"container_image_elasticsearch","type":"str","level":"advanced","flags":0,"default_value":"quay.io/omrizeneva/elasticsearch:6.8.23","min":"","max":"","enum_allowed":[],"desc":"Elasticsearch container image","long_desc":"","tags":[],"see_also":[]},"container_image_grafana":{"name":"container_image_grafana","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/grafana:12.3.1","min":"","max":"","enum_allowed":[],"desc":"Grafana container image","long_desc":"","tags":[],"see_also":[]},"container_image_haproxy":{"name":"container_image_haproxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/haproxy:2.3","min":"","max":"","enum_allowed":[],"desc":"Haproxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_agent":{"name":"container_image_jaeger_agent","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-agent:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger agent container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_collector":{"name":"container_image_jaeger_collector","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-collector:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger collector container image","long_desc":"","tags":[],"see_also":[]},"container_image_jaeger_query":{"name":"container_image_jaeger_query","type":"str","level":"advanced","flags":0,"default_value":"quay.io/jaegertracing/jaeger-query:1.29","min":"","max":"","enum_allowed":[],"desc":"Jaeger query container image","long_desc":"","tags":[],"see_also":[]},"container_image_keepalived":{"name":"container_image_keepalived","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/keepalived:2.2.4","min":"","max":"","enum_allowed":[],"desc":"Keepalived container image","long_desc":"","tags":[],"see_also":[]},"container_image_loki":{"name":"container_image_loki","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/loki:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Loki container image","long_desc":"","tags":[],"see_also":[]},"container_image_nginx":{"name":"container_image_nginx","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nginx:sclorg-nginx-126","min":"","max":"","enum_allowed":[],"desc":"Nginx container image","long_desc":"","tags":[],"see_also":[]},"container_image_node_exporter":{"name":"container_image_node_exporter","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/node-exporter:v1.9.1","min":"","max":"","enum_allowed":[],"desc":"Node exporter container image","long_desc":"","tags":[],"see_also":[]},"container_image_nvmeof":{"name":"container_image_nvmeof","type":"str","level":"advanced","flags":0,"default_value":"quay.io/ceph/nvmeof:1.7","min":"","max":"","enum_allowed":[],"desc":"Nvmeof container image","long_desc":"","tags":[],"see_also":[]},"container_image_oauth2_proxy":{"name":"container_image_oauth2_proxy","type":"str","level":"advanced","flags":0,"default_value":"quay.io/oauth2-proxy/oauth2-proxy:v7.6.0","min":"","max":"","enum_allowed":[],"desc":"Oauth2 proxy container image","long_desc":"","tags":[],"see_also":[]},"container_image_prometheus":{"name":"container_image_prometheus","type":"str","level":"advanced","flags":0,"default_value":"quay.io/prometheus/prometheus:v3.6.0","min":"","max":"","enum_allowed":[],"desc":"Prometheus container image","long_desc":"","tags":[],"see_also":[]},"container_image_promtail":{"name":"container_image_promtail","type":"str","level":"advanced","flags":0,"default_value":"docker.io/grafana/promtail:3.0.0","min":"","max":"","enum_allowed":[],"desc":"Promtail container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba":{"name":"container_image_samba","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-server:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba container image","long_desc":"","tags":[],"see_also":[]},"container_image_samba_metrics":{"name":"container_image_samba_metrics","type":"str","level":"advanced","flags":0,"default_value":"quay.io/samba.org/samba-metrics:devbuilds-centos-amd64","min":"","max":"","enum_allowed":[],"desc":"Samba metrics container image","long_desc":"","tags":[],"see_also":[]},"container_image_snmp_gateway":{"name":"container_image_snmp_gateway","type":"str","level":"advanced","flags":0,"default_value":"docker.io/maxwo/snmp-notifier:v1.2.1","min":"","max":"","enum_allowed":[],"desc":"Snmp gateway container image","long_desc":"","tags":[],"see_also":[]},"container_init":{"name":"container_init","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Run podman/docker with `--init`","long_desc":"","tags":[],"see_also":[]},"daemon_cache_timeout":{"name":"daemon_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache service (daemon) inventory","long_desc":"","tags":[],"see_also":[]},"default_cephadm_command_timeout":{"name":"default_cephadm_command_timeout","type":"int","level":"advanced","flags":0,"default_value":"900","min":"","max":"","enum_allowed":[],"desc":"Default timeout applied to cephadm commands run directly on the host (in seconds)","long_desc":"","tags":[],"see_also":[]},"default_registry":{"name":"default_registry","type":"str","level":"advanced","flags":0,"default_value":"quay.io","min":"","max":"","enum_allowed":[],"desc":"Search-registry to which we should normalize unqualified image names. This is not the default registry","long_desc":"","tags":[],"see_also":[]},"device_cache_timeout":{"name":"device_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache device inventory","long_desc":"","tags":[],"see_also":[]},"device_enhanced_scan":{"name":"device_enhanced_scan","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use libstoragemgmt during device scans","long_desc":"","tags":[],"see_also":[]},"facts_cache_timeout":{"name":"facts_cache_timeout","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"Seconds for which to cache host facts data","long_desc":"","tags":[],"see_also":[]},"grafana_dashboards_path":{"name":"grafana_dashboards_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/grafana/dashboards/ceph-dashboard/","min":"","max":"","enum_allowed":[],"desc":"Location of dashboards to include in Grafana","long_desc":"","tags":[],"see_also":[]},"host_check_interval":{"name":"host_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"How frequently to perform a host check","long_desc":"","tags":[],"see_also":[]},"hw_monitoring":{"name":"hw_monitoring","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether the hardware monitoring daemon be deployed on every host?","long_desc":"","tags":[],"see_also":[]},"inventory_list_all":{"name":"inventory_list_all","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Whether ceph-volume inventory should report more devices (mostly mappers (LVs / mpaths), partitions...)","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_refresh_metadata":{"name":"log_refresh_metadata","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Log all refresh metadata. Includes daemon, device, and host info collected regularly. Only has effect if logging at the debug level","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Log to the \"cephadm\" cluster log channel\"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf":{"name":"manage_etc_ceph_ceph_conf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Manage and own /etc/ceph/ceph.conf on hosts.","long_desc":"","tags":[],"see_also":[]},"manage_etc_ceph_ceph_conf_hosts":{"name":"manage_etc_ceph_ceph_conf_hosts","type":"str","level":"advanced","flags":0,"default_value":"*","min":"","max":"","enum_allowed":[],"desc":"PlacementSpec describing on which hosts to manage /etc/ceph/ceph.conf","long_desc":"","tags":[],"see_also":[]},"max_count_per_host":{"name":"max_count_per_host","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of daemons per service per host: guard against mistakes and runaways","long_desc":"","tags":[],"see_also":[]},"max_osd_draining_count":{"name":"max_osd_draining_count","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"Max number of OSDs that will be drained simultaneously when OSDs are removed","long_desc":"","tags":[],"see_also":[]},"max_parallel_osd_upgrades":{"name":"max_parallel_osd_upgrades","type":"int","level":"advanced","flags":0,"default_value":"16","min":"","max":"","enum_allowed":[],"desc":"Maximum number of OSD daemons upgraded in parallel.","long_desc":"","tags":[],"see_also":[]},"migration_current":{"name":"migration_current","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Internal - do not modify","long_desc":"","tags":[],"see_also":[]},"mode":{"name":"mode","type":"str","level":"advanced","flags":0,"default_value":"root","min":"","max":"","enum_allowed":["cephadm-package","root"],"desc":"Mode for remote execution of cephadm","long_desc":"","tags":[],"see_also":[]},"oob_default_addr":{"name":"oob_default_addr","type":"str","level":"advanced","flags":0,"default_value":"169.254.1.1","min":"","max":"","enum_allowed":[],"desc":"Default IP address for RedFish API (OOB management).","long_desc":"","tags":[],"see_also":[]},"prometheus_alerts_path":{"name":"prometheus_alerts_path","type":"str","level":"advanced","flags":0,"default_value":"/etc/prometheus/ceph/ceph_default_alerts.yml","min":"","max":"","enum_allowed":[],"desc":"Location of alerts to include in Prometheus Alertmanager","long_desc":"","tags":[],"see_also":[]},"registry_insecure":{"name":"registry_insecure","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Registry is to be considered insecure (no TLS available). Only for development purposes.","long_desc":"","tags":[],"see_also":[]},"registry_password":{"name":"registry_password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository password. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"registry_url":{"name":"registry_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Registry URL for login purposes. This is not the default registry.","long_desc":"","tags":[],"see_also":[]},"registry_username":{"name":"registry_username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Custom repository username. Only used for logging into a registry.","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"service_discovery_port":{"name":"service_discovery_port","type":"int","level":"advanced","flags":0,"default_value":"8765","min":"","max":"","enum_allowed":[],"desc":"Cephadm service discovery port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssh_config_file":{"name":"ssh_config_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"Customized SSH config file to distribute to managed hosts","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_count_max":{"name":"ssh_keepalive_count_max","type":"int","level":"advanced","flags":0,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"How many sequential SSH connection liveness check failures before the host is marked offline","long_desc":"","tags":[],"see_also":[]},"ssh_keepalive_interval":{"name":"ssh_keepalive_interval","type":"int","level":"advanced","flags":0,"default_value":"7","min":"","max":"","enum_allowed":[],"desc":"How often SSH connections are checked for liveness","long_desc":"","tags":[],"see_also":[]},"stray_daemon_check_interval":{"name":"stray_daemon_check_interval","type":"secs","level":"advanced","flags":0,"default_value":"1800","min":"","max":"","enum_allowed":[],"desc":"How frequently cephadm should check for the presence of stray daemons","long_desc":"","tags":[],"see_also":[]},"use_agent":{"name":"use_agent","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Use the cephadm agent on each host to gather and send metadata","long_desc":"","tags":[],"see_also":[]},"use_repo_digest":{"name":"use_repo_digest","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Automatically convert image tags to image digest to ensure that all daemons use the same image","long_desc":"","tags":[],"see_also":[]},"warn_on_failed_host_check":{"name":"warn_on_failed_host_check","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if the host check fails","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_daemons":{"name":"warn_on_stray_daemons","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected that are not managed by cephadm","long_desc":"","tags":[],"see_also":[]},"warn_on_stray_hosts":{"name":"warn_on_stray_hosts","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Raise a health warning if daemons are detected on a host that is not managed by cephadm","long_desc":"","tags":[],"see_also":[]}}},{"name":"crash","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"retain_interval":{"name":"retain_interval","type":"secs","level":"advanced","flags":1,"default_value":"31536000","min":"","max":"","enum_allowed":[],"desc":"how long to retain crashes before pruning them","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_recent_interval":{"name":"warn_recent_interval","type":"secs","level":"advanced","flags":1,"default_value":"1209600","min":"","max":"","enum_allowed":[],"desc":"time interval in which to warn about recent crashes","long_desc":"","tags":[],"see_also":[]}}},{"name":"dashboard","can_run":true,"error_string":"","module_options":{"ACCOUNT_LOCKOUT_ATTEMPTS":{"name":"ACCOUNT_LOCKOUT_ATTEMPTS","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_HOST":{"name":"ALERTMANAGER_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ALERTMANAGER_API_SSL_VERIFY":{"name":"ALERTMANAGER_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_ENABLED":{"name":"AUDIT_API_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"AUDIT_API_LOG_PAYLOAD":{"name":"AUDIT_API_LOG_PAYLOAD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ENABLE_BROWSABLE_API":{"name":"ENABLE_BROWSABLE_API","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_CEPHFS":{"name":"FEATURE_TOGGLE_CEPHFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_DASHBOARD":{"name":"FEATURE_TOGGLE_DASHBOARD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_ISCSI":{"name":"FEATURE_TOGGLE_ISCSI","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_MIRRORING":{"name":"FEATURE_TOGGLE_MIRRORING","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_NFS":{"name":"FEATURE_TOGGLE_NFS","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RBD":{"name":"FEATURE_TOGGLE_RBD","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"FEATURE_TOGGLE_RGW":{"name":"FEATURE_TOGGLE_RGW","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE":{"name":"GANESHA_CLUSTERS_RADOS_POOL_NAMESPACE","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_PASSWORD":{"name":"GRAFANA_API_PASSWORD","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_SSL_VERIFY":{"name":"GRAFANA_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_URL":{"name":"GRAFANA_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_API_USERNAME":{"name":"GRAFANA_API_USERNAME","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_FRONTEND_API_URL":{"name":"GRAFANA_FRONTEND_API_URL","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"GRAFANA_UPDATE_DASHBOARDS":{"name":"GRAFANA_UPDATE_DASHBOARDS","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISCSI_API_SSL_VERIFICATION":{"name":"ISCSI_API_SSL_VERIFICATION","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ISSUE_TRACKER_API_KEY":{"name":"ISSUE_TRACKER_API_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MANAGED_BY_CLUSTERS":{"name":"MANAGED_BY_CLUSTERS","type":"str","level":"advanced","flags":0,"default_value":"[]","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"MULTICLUSTER_CONFIG":{"name":"MULTICLUSTER_CONFIG","type":"str","level":"advanced","flags":0,"default_value":"{}","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_HOST":{"name":"PROMETHEUS_API_HOST","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROMETHEUS_API_SSL_VERIFY":{"name":"PROMETHEUS_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PROM_ALERT_CREDENTIAL_CACHE_TTL":{"name":"PROM_ALERT_CREDENTIAL_CACHE_TTL","type":"int","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_COMPLEXITY_ENABLED":{"name":"PWD_POLICY_CHECK_COMPLEXITY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED":{"name":"PWD_POLICY_CHECK_EXCLUSION_LIST_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_LENGTH_ENABLED":{"name":"PWD_POLICY_CHECK_LENGTH_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_OLDPWD_ENABLED":{"name":"PWD_POLICY_CHECK_OLDPWD_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_REPETITIVE_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED":{"name":"PWD_POLICY_CHECK_SEQUENTIAL_CHARS_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_CHECK_USERNAME_ENABLED":{"name":"PWD_POLICY_CHECK_USERNAME_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_ENABLED":{"name":"PWD_POLICY_ENABLED","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_EXCLUSION_LIST":{"name":"PWD_POLICY_EXCLUSION_LIST","type":"str","level":"advanced","flags":0,"default_value":"osd,host,dashboard,pool,block,nfs,ceph,monitors,gateway,logs,crush,maps","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_COMPLEXITY":{"name":"PWD_POLICY_MIN_COMPLEXITY","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"PWD_POLICY_MIN_LENGTH":{"name":"PWD_POLICY_MIN_LENGTH","type":"int","level":"advanced","flags":0,"default_value":"8","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"REST_REQUESTS_TIMEOUT":{"name":"REST_REQUESTS_TIMEOUT","type":"int","level":"advanced","flags":0,"default_value":"45","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ACCESS_KEY":{"name":"RGW_API_ACCESS_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_ADMIN_RESOURCE":{"name":"RGW_API_ADMIN_RESOURCE","type":"str","level":"advanced","flags":0,"default_value":"admin","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SECRET_KEY":{"name":"RGW_API_SECRET_KEY","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_API_SSL_VERIFY":{"name":"RGW_API_SSL_VERIFY","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"RGW_HOSTNAME_PER_DAEMON":{"name":"RGW_HOSTNAME_PER_DAEMON","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"UNSAFE_TLS_v1_2":{"name":"UNSAFE_TLS_v1_2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_SPAN":{"name":"USER_PWD_EXPIRATION_SPAN","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_1":{"name":"USER_PWD_EXPIRATION_WARNING_1","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"USER_PWD_EXPIRATION_WARNING_2":{"name":"USER_PWD_EXPIRATION_WARNING_2","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"cross_origin_url":{"name":"cross_origin_url","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crt_file":{"name":"crt_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"crypto_caller":{"name":"crypto_caller","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"debug":{"name":"debug","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable/disable debug options","long_desc":"","tags":[],"see_also":[]},"jwt_token_ttl":{"name":"jwt_token_ttl","type":"int","level":"advanced","flags":0,"default_value":"28800","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"key_file":{"name":"key_file","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"motd":{"name":"motd","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"The message of the day","long_desc":"","tags":[],"see_also":[]},"redirect_resolve_ip_addr":{"name":"redirect_resolve_ip_addr","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":0,"default_value":"8080","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl_server_port":{"name":"ssl_server_port","type":"int","level":"advanced","flags":0,"default_value":"8443","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sso_oauth2":{"name":"sso_oauth2","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":0,"default_value":"redirect","min":"","max":"","enum_allowed":["error","redirect"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":0,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url_prefix":{"name":"url_prefix","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"devicehealth","can_run":true,"error_string":"","module_options":{"enable_monitoring":{"name":"enable_monitoring","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"monitor device health metrics","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mark_out_threshold":{"name":"mark_out_threshold","type":"secs","level":"advanced","flags":1,"default_value":"2419200","min":"","max":"","enum_allowed":[],"desc":"automatically mark OSD if it may fail before this long","long_desc":"","tags":[],"see_also":[]},"pool_name":{"name":"pool_name","type":"str","level":"advanced","flags":1,"default_value":"device_health_metrics","min":"","max":"","enum_allowed":[],"desc":"name of pool in which to store device health metrics","long_desc":"","tags":[],"see_also":[]},"retention_period":{"name":"retention_period","type":"secs","level":"advanced","flags":1,"default_value":"15552000","min":"","max":"","enum_allowed":[],"desc":"how long to retain device health metrics","long_desc":"","tags":[],"see_also":[]},"scrape_frequency":{"name":"scrape_frequency","type":"secs","level":"advanced","flags":1,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"how frequently to scrape device health metrics","long_desc":"","tags":[],"see_also":[]},"self_heal":{"name":"self_heal","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"preemptively heal cluster around devices that may fail","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"how frequently to wake up and check device health","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"warn_threshold":{"name":"warn_threshold","type":"secs","level":"advanced","flags":1,"default_value":"7257600","min":"","max":"","enum_allowed":[],"desc":"raise health warning if OSD may fail before this long","long_desc":"","tags":[],"see_also":[]}}},{"name":"diskprediction_local","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predict_interval":{"name":"predict_interval","type":"str","level":"advanced","flags":0,"default_value":"86400","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"predictor_model":{"name":"predictor_model","type":"str","level":"advanced","flags":0,"default_value":"prophetstor","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"str","level":"advanced","flags":0,"default_value":"600","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"influx","can_run":false,"error_string":"influxdb python module not found","module_options":{"batch_size":{"name":"batch_size","type":"int","level":"advanced","flags":0,"default_value":"5000","min":"","max":"","enum_allowed":[],"desc":"How big batches of data points should be when sending to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"database":{"name":"database","type":"str","level":"advanced","flags":0,"default_value":"ceph","min":"","max":"","enum_allowed":[],"desc":"InfluxDB database name. You will need to create this database and grant write privileges to the configured username or the username must have admin privileges to create it.","long_desc":"","tags":[],"see_also":[]},"hostname":{"name":"hostname","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server hostname","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"30","min":"5","max":"","enum_allowed":[],"desc":"Time between reports to InfluxDB. Default 30 seconds.","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"password":{"name":"password","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"password of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"port":{"name":"port","type":"int","level":"advanced","flags":0,"default_value":"8086","min":"","max":"","enum_allowed":[],"desc":"InfluxDB server port","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"ssl":{"name":"ssl","type":"str","level":"advanced","flags":0,"default_value":"false","min":"","max":"","enum_allowed":[],"desc":"Use https connection for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]},"threads":{"name":"threads","type":"int","level":"advanced","flags":0,"default_value":"5","min":"1","max":"32","enum_allowed":[],"desc":"How many worker threads should be spawned for sending data to InfluxDB.","long_desc":"","tags":[],"see_also":[]},"username":{"name":"username","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"username of InfluxDB server user","long_desc":"","tags":[],"see_also":[]},"verify_ssl":{"name":"verify_ssl","type":"str","level":"advanced","flags":0,"default_value":"true","min":"","max":"","enum_allowed":[],"desc":"Verify https cert for InfluxDB server. Use \"true\" or \"false\".","long_desc":"","tags":[],"see_also":[]}}},{"name":"insights","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"iostat","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"localpool","can_run":true,"error_string":"","module_options":{"failure_domain":{"name":"failure_domain","type":"str","level":"advanced","flags":1,"default_value":"host","min":"","max":"","enum_allowed":[],"desc":"failure domain for any created local pool","long_desc":"what failure domain we should separate data replicas across.","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"min_size":{"name":"min_size","type":"int","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"default min_size for any created local pool","long_desc":"value to set min_size to (unchanged from Ceph's default if this option is not set)","tags":[],"see_also":[]},"num_rep":{"name":"num_rep","type":"int","level":"advanced","flags":1,"default_value":"3","min":"","max":"","enum_allowed":[],"desc":"default replica count for any created local pool","long_desc":"","tags":[],"see_also":[]},"pg_num":{"name":"pg_num","type":"int","level":"advanced","flags":1,"default_value":"128","min":"","max":"","enum_allowed":[],"desc":"default pg_num for any created local pool","long_desc":"","tags":[],"see_also":[]},"prefix":{"name":"prefix","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"name prefix for any created local pool","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"subtree":{"name":"subtree","type":"str","level":"advanced","flags":1,"default_value":"rack","min":"","max":"","enum_allowed":[],"desc":"CRUSH level for which to create a local pool","long_desc":"which CRUSH subtree type the module should create a pool for.","tags":[],"see_also":[]}}},{"name":"mds_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"mirroring","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"nfs","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"orchestrator","can_run":true,"error_string":"","module_options":{"fail_fs":{"name":"fail_fs","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Fail filesystem for rapid multi-rank mds upgrade","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"orchestrator":{"name":"orchestrator","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["cephadm","rook","test_orchestrator"],"desc":"Orchestrator backend","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_perf_query","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"osd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"pg_autoscaler","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":0,"default_value":"60","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"threshold":{"name":"threshold","type":"float","level":"advanced","flags":0,"default_value":"3.0","min":"1.0","max":"","enum_allowed":[],"desc":"scaling threshold","long_desc":"The factor by which the `NEW PG_NUM` must vary from the current`PG_NUM` before being accepted. Cannot be less than 1.0","tags":[],"see_also":[]}}},{"name":"progress","can_run":true,"error_string":"","module_options":{"allow_pg_recovery_event":{"name":"allow_pg_recovery_event","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow the module to show pg recovery progress","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_completed_events":{"name":"max_completed_events","type":"int","level":"advanced","flags":1,"default_value":"50","min":"","max":"","enum_allowed":[],"desc":"number of past completed events to remember","long_desc":"","tags":[],"see_also":[]},"sleep_interval":{"name":"sleep_interval","type":"secs","level":"advanced","flags":1,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"how long the module is going to sleep","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"prometheus","can_run":true,"error_string":"","module_options":{"cache":{"name":"cache","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"exclude_perf_counters":{"name":"exclude_perf_counters","type":"bool","level":"advanced","flags":1,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Do not include perf-counters in the metrics output","long_desc":"Gathering perf-counters from a single Prometheus exporter can degrade ceph-mgr performance, especially in large clusters. Instead, Ceph-exporter daemons are now used by default for perf-counter gathering. This should only be disabled when no ceph-exporters are deployed.","tags":[],"see_also":[]},"healthcheck_history_max_entries":{"name":"healthcheck_history_max_entries","type":"int","level":"advanced","flags":1,"default_value":"1000","min":"","max":"","enum_allowed":[],"desc":"Maximum number of health check history entries to keep","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools":{"name":"rbd_stats_pools","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rbd_stats_pools_refresh_interval":{"name":"rbd_stats_pools_refresh_interval","type":"int","level":"advanced","flags":0,"default_value":"300","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"scrape_interval":{"name":"scrape_interval","type":"float","level":"advanced","flags":0,"default_value":"15.0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"server_addr":{"name":"server_addr","type":"str","level":"advanced","flags":0,"default_value":"::","min":"","max":"","enum_allowed":[],"desc":"the IPv4 or IPv6 address on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"server_port":{"name":"server_port","type":"int","level":"advanced","flags":1,"default_value":"9283","min":"","max":"","enum_allowed":[],"desc":"the port on which the module listens for HTTP requests","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"stale_cache_strategy":{"name":"stale_cache_strategy","type":"str","level":"advanced","flags":0,"default_value":"log","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_behaviour":{"name":"standby_behaviour","type":"str","level":"advanced","flags":1,"default_value":"default","min":"","max":"","enum_allowed":["default","error"],"desc":"","long_desc":"","tags":[],"see_also":[]},"standby_error_status_code":{"name":"standby_error_status_code","type":"int","level":"advanced","flags":1,"default_value":"500","min":"400","max":"599","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rbd_support","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_snap_create":{"name":"max_concurrent_snap_create","type":"int","level":"advanced","flags":0,"default_value":"10","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"mirror_snapshot_schedule":{"name":"mirror_snapshot_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"trash_purge_schedule":{"name":"trash_purge_schedule","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rgw","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"secondary_zone_period_retry_limit":{"name":"secondary_zone_period_retry_limit","type":"int","level":"advanced","flags":0,"default_value":"5","min":"","max":"","enum_allowed":[],"desc":"RGW module period update retry limit for secondary site","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"rook","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"prometheus_tls_secret_name":{"name":"prometheus_tls_secret_name","type":"str","level":"advanced","flags":0,"default_value":"rook-ceph-prometheus-server-tls","min":"","max":"","enum_allowed":[],"desc":"name of tls secret in k8s for prometheus","long_desc":"","tags":[],"see_also":[]},"secure_monitoring_stack":{"name":"secure_monitoring_stack","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Enable TLS security for all the monitoring stack daemons","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"storage_class":{"name":"storage_class","type":"str","level":"advanced","flags":0,"default_value":"local","min":"","max":"","enum_allowed":[],"desc":"storage class name for LSO-discovered PVs","long_desc":"","tags":[],"see_also":[]}}},{"name":"selftest","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption1":{"name":"roption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"roption2":{"name":"roption2","type":"str","level":"advanced","flags":0,"default_value":"xyz","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption1":{"name":"rwoption1","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption2":{"name":"rwoption2","type":"int","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption3":{"name":"rwoption3","type":"float","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption4":{"name":"rwoption4","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption5":{"name":"rwoption5","type":"bool","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption6":{"name":"rwoption6","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"rwoption7":{"name":"rwoption7","type":"int","level":"advanced","flags":0,"default_value":"","min":"1","max":"42","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testkey":{"name":"testkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testlkey":{"name":"testlkey","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"testnewline":{"name":"testnewline","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"smb","can_run":true,"error_string":"","module_options":{"internal_store_backend":{"name":"internal_store_backend","type":"str","level":"dev","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"set internal store backend. for develoment and testing only","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"update_orchestration":{"name":"update_orchestration","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"automatically update orchestration when smb resources are changed","long_desc":"","tags":[],"see_also":[]}}},{"name":"snap_schedule","can_run":true,"error_string":"","module_options":{"allow_m_granularity":{"name":"allow_m_granularity","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"allow minute scheduled snapshots","long_desc":"","tags":[],"see_also":[]},"dump_on_update":{"name":"dump_on_update","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"dump database to debug log on update","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"stats","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"status","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telegraf","can_run":true,"error_string":"","module_options":{"address":{"name":"address","type":"str","level":"advanced","flags":0,"default_value":"unixgram:///tmp/telegraf.sock","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"secs","level":"advanced","flags":0,"default_value":"15","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"telemetry","can_run":true,"error_string":"","module_options":{"channel_basic":{"name":"channel_basic","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share basic cluster information (size, version)","long_desc":"","tags":[],"see_also":[]},"channel_crash":{"name":"channel_crash","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share metadata about Ceph daemon crashes (version, stack straces, etc)","long_desc":"","tags":[],"see_also":[]},"channel_device":{"name":"channel_device","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Share device health metrics (e.g., SMART data, minus potentially identifying info like serial numbers)","long_desc":"","tags":[],"see_also":[]},"channel_ident":{"name":"channel_ident","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share a user-provided description and/or contact email for the cluster","long_desc":"","tags":[],"see_also":[]},"channel_perf":{"name":"channel_perf","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Share various performance metrics of a cluster","long_desc":"","tags":[],"see_also":[]},"contact":{"name":"contact","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"description":{"name":"description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"device_url":{"name":"device_url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/device","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"enabled":{"name":"enabled","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"interval":{"name":"interval","type":"int","level":"advanced","flags":0,"default_value":"24","min":"8","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"last_opt_revision":{"name":"last_opt_revision","type":"int","level":"advanced","flags":0,"default_value":"1","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard":{"name":"leaderboard","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"leaderboard_description":{"name":"leaderboard_description","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"organization":{"name":"organization","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"proxy":{"name":"proxy","type":"str","level":"advanced","flags":0,"default_value":"","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"url":{"name":"url","type":"str","level":"advanced","flags":0,"default_value":"https://telemetry.ceph.com/report","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"test_orchestrator","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}},{"name":"volumes","can_run":true,"error_string":"","module_options":{"log_level":{"name":"log_level","type":"str","level":"advanced","flags":1,"default_value":"","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster":{"name":"log_to_cluster","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_cluster_level":{"name":"log_to_cluster_level","type":"str","level":"advanced","flags":1,"default_value":"info","min":"","max":"","enum_allowed":["","critical","debug","error","info","warning"],"desc":"","long_desc":"","tags":[],"see_also":[]},"log_to_file":{"name":"log_to_file","type":"bool","level":"advanced","flags":1,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]},"max_concurrent_clones":{"name":"max_concurrent_clones","type":"int","level":"advanced","flags":0,"default_value":"4","min":"","max":"","enum_allowed":[],"desc":"Number of asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_cloning":{"name":"pause_cloning","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous cloner threads","long_desc":"","tags":[],"see_also":[]},"pause_purging":{"name":"pause_purging","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Pause asynchronous subvolume purge threads","long_desc":"","tags":[],"see_also":[]},"periodic_async_work":{"name":"periodic_async_work","type":"bool","level":"advanced","flags":0,"default_value":"False","min":"","max":"","enum_allowed":[],"desc":"Periodically check for async work","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_delay":{"name":"snapshot_clone_delay","type":"int","level":"advanced","flags":0,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"Delay clone begin operation by snapshot_clone_delay seconds","long_desc":"","tags":[],"see_also":[]},"snapshot_clone_no_wait":{"name":"snapshot_clone_no_wait","type":"bool","level":"advanced","flags":0,"default_value":"True","min":"","max":"","enum_allowed":[],"desc":"Reject subvolume clone request when cloner threads are busy","long_desc":"","tags":[],"see_also":[]},"sqlite3_killpoint":{"name":"sqlite3_killpoint","type":"int","level":"dev","flags":1,"default_value":"0","min":"","max":"","enum_allowed":[],"desc":"","long_desc":"","tags":[],"see_also":[]}}}],"services":{},"always_on_modules":{"octopus":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"pacific":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"quincy":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"reef":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"squid":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"],"tentacle":["balancer","crash","devicehealth","orchestrator","pg_autoscaler","progress","rbd_support","status","telemetry","volumes"]},"force_disabled_modules":{},"last_failure_osd_epoch":0,"active_clients":[{"name":"devicehealth","addrvec":[{"type":"v2","addr":"10.20.193.151:0","nonce":3076529377}]},{"name":"libcephsqlite","addrvec":[{"type":"v2","addr":"10.20.193.151:0","nonce":2529746192}]},{"name":"rbd_support","addrvec":[{"type":"v2","addr":"10.20.193.151:0","nonce":1271612826}]},{"name":"volumes","addrvec":[{"type":"v2","addr":"10.20.193.151:0","nonce":1311098049}]}]} 2026-02-25T09:25:50.459 INFO:tasks.ceph.ceph_manager.ceph:mgr available! 2026-02-25T09:25:50.460 INFO:tasks.ceph.ceph_manager.ceph:waiting for all up 2026-02-25T09:25:50.460 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T09:25:50.535 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:50.532+0000 7f34e8ab7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:50.538 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:50.536+0000 7f34e8ab7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:50.652 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:50.653 INFO:teuthology.orchestra.run.trial094.stdout:{"epoch":15,"fsid":"c0375da7-c136-43b7-8240-9e8e7e46c4e5","created":"2026-02-25T09:25:36.641907+0000","modified":"2026-02-25T09:25:49.946101+0000","last_up_change":"2026-02-25T09:25:45.931020+0000","last_in_change":"2026-02-25T09:25:38.510236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-25T09:25:46.231663+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2026-02-25T09:25:46.689416+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84b58e5b-579c-49b7-9672-6379199b2e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6804","nonce":889111984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6803","nonce":889111984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6805","nonce":889111984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6807","nonce":889111984}]},"public_addr":"10.20.193.94:6804/889111984","cluster_addr":"10.20.193.94:6803/889111984","heartbeat_back_addr":"10.20.193.94:6805/889111984","heartbeat_front_addr":"10.20.193.94:6807/889111984","state":["exists","up"]},{"osd":1,"uuid":"47831adf-49cb-4d5d-af8a-f010094da3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6802","nonce":3409185002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6801","nonce":3409185002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6808","nonce":3409185002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6806","nonce":3409185002}]},"public_addr":"10.20.193.94:6802/3409185002","cluster_addr":"10.20.193.94:6801/3409185002","heartbeat_back_addr":"10.20.193.94:6808/3409185002","heartbeat_front_addr":"10.20.193.94:6806/3409185002","state":["exists","up"]},{"osd":2,"uuid":"764573ae-9128-448c-a36d-b40e3e9ca850","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6805","nonce":2663299575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6806","nonce":2663299575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6815","nonce":2663299575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6816","nonce":2663299575}]},"public_addr":"10.20.193.151:6805/2663299575","cluster_addr":"10.20.193.151:6806/2663299575","heartbeat_back_addr":"10.20.193.151:6815/2663299575","heartbeat_front_addr":"10.20.193.151:6816/2663299575","state":["exists","up"]},{"osd":3,"uuid":"272a2704-6263-47ab-87f5-4c0dc76ddfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6803","nonce":4248319012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6802","nonce":4248319012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6808","nonce":4248319012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6809","nonce":4248319012}]},"public_addr":"10.20.193.151:6803/4248319012","cluster_addr":"10.20.193.151:6802/4248319012","heartbeat_back_addr":"10.20.193.151:6808/4248319012","heartbeat_front_addr":"10.20.193.151:6809/4248319012","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T09:25:50.663 INFO:tasks.ceph.ceph_manager.ceph:all up! 2026-02-25T09:25:50.663 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T09:25:50.738 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:50.734+0000 7f9bb205a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:50.740 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:50.738+0000 7f9bb205a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:50.852 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:50.852 INFO:teuthology.orchestra.run.trial094.stdout:{"epoch":15,"fsid":"c0375da7-c136-43b7-8240-9e8e7e46c4e5","created":"2026-02-25T09:25:36.641907+0000","modified":"2026-02-25T09:25:49.946101+0000","last_up_change":"2026-02-25T09:25:45.931020+0000","last_in_change":"2026-02-25T09:25:38.510236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":2,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-25T09:25:46.231663+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2026-02-25T09:25:46.689416+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84b58e5b-579c-49b7-9672-6379199b2e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6804","nonce":889111984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6803","nonce":889111984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6805","nonce":889111984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6807","nonce":889111984}]},"public_addr":"10.20.193.94:6804/889111984","cluster_addr":"10.20.193.94:6803/889111984","heartbeat_back_addr":"10.20.193.94:6805/889111984","heartbeat_front_addr":"10.20.193.94:6807/889111984","state":["exists","up"]},{"osd":1,"uuid":"47831adf-49cb-4d5d-af8a-f010094da3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6802","nonce":3409185002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6801","nonce":3409185002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6808","nonce":3409185002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6806","nonce":3409185002}]},"public_addr":"10.20.193.94:6802/3409185002","cluster_addr":"10.20.193.94:6801/3409185002","heartbeat_back_addr":"10.20.193.94:6808/3409185002","heartbeat_front_addr":"10.20.193.94:6806/3409185002","state":["exists","up"]},{"osd":2,"uuid":"764573ae-9128-448c-a36d-b40e3e9ca850","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":0,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6805","nonce":2663299575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6806","nonce":2663299575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6815","nonce":2663299575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6816","nonce":2663299575}]},"public_addr":"10.20.193.151:6805/2663299575","cluster_addr":"10.20.193.151:6806/2663299575","heartbeat_back_addr":"10.20.193.151:6815/2663299575","heartbeat_front_addr":"10.20.193.151:6816/2663299575","state":["exists","up"]},{"osd":3,"uuid":"272a2704-6263-47ab-87f5-4c0dc76ddfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":12,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6803","nonce":4248319012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6802","nonce":4248319012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6808","nonce":4248319012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6809","nonce":4248319012}]},"public_addr":"10.20.193.151:6803/4248319012","cluster_addr":"10.20.193.151:6802/4248319012","heartbeat_back_addr":"10.20.193.151:6808/4248319012","heartbeat_front_addr":"10.20.193.151:6809/4248319012","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T09:25:50.861 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 flush_pg_stats 2026-02-25T09:25:50.862 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 flush_pg_stats 2026-02-25T09:25:50.863 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 flush_pg_stats 2026-02-25T09:25:50.863 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 flush_pg_stats 2026-02-25T09:25:50.945 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:50.940+0000 7fb91c57e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:50.945 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:50.940+0000 7f830423d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:50.945 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:50.940+0000 7f764e70b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:50.946 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:50.941+0000 7f6ba6c35640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:50.947 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:50.946+0000 7f830423d640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:50.948 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:50.946+0000 7fb91c57e640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:50.948 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:50.946+0000 7f764e70b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:50.948 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:50.946+0000 7f6ba6c35640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:50.966 INFO:teuthology.orchestra.run.trial094.stdout:42949672966 2026-02-25T09:25:50.967 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.3 2026-02-25T09:25:50.968 INFO:teuthology.orchestra.run.trial094.stdout:47244640261 2026-02-25T09:25:50.968 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.1 2026-02-25T09:25:50.969 INFO:teuthology.orchestra.run.trial094.stdout:47244640261 2026-02-25T09:25:50.970 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.2 2026-02-25T09:25:50.971 INFO:teuthology.orchestra.run.trial094.stdout:47244640261 2026-02-25T09:25:50.971 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd last-stat-seq osd.0 2026-02-25T09:25:51.090 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:51.086+0000 7f34ea4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:51.092 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:51.088+0000 7feba525a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:51.092 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:51.088+0000 7f703685a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:51.093 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:51.092+0000 7f34ea4cb640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:51.095 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:51.094+0000 7f703685a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:51.095 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:51.094+0000 7feba525a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:51.130 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:51.125+0000 7f177861a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:51.133 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:51.131+0000 7f177861a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:51.209 INFO:teuthology.orchestra.run.trial094.stdout:47244640261 2026-02-25T09:25:51.211 INFO:teuthology.orchestra.run.trial094.stdout:47244640261 2026-02-25T09:25:51.215 INFO:teuthology.orchestra.run.trial094.stdout:42949672966 2026-02-25T09:25:51.221 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.1 2026-02-25T09:25:51.221 DEBUG:teuthology.parallel:result is None 2026-02-25T09:25:51.222 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.2 2026-02-25T09:25:51.223 DEBUG:teuthology.parallel:result is None 2026-02-25T09:25:51.226 INFO:tasks.ceph.ceph_manager.ceph:need seq 42949672966 got 42949672966 for osd.3 2026-02-25T09:25:51.226 DEBUG:teuthology.parallel:result is None 2026-02-25T09:25:51.247 INFO:teuthology.orchestra.run.trial094.stdout:47244640261 2026-02-25T09:25:51.257 INFO:tasks.ceph.ceph_manager.ceph:need seq 47244640261 got 47244640261 for osd.0 2026-02-25T09:25:51.258 DEBUG:teuthology.parallel:result is None 2026-02-25T09:25:51.258 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-25T09:25:51.258 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T09:25:51.372 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:51.369+0000 7efd93b74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:51.375 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:51.373+0000 7efd93b74640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:51.485 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:51.485 INFO:teuthology.orchestra.run.trial094.stderr:dumped all 2026-02-25T09:25:51.494 INFO:teuthology.orchestra.run.trial094.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2026-02-25T09:25:50.959510+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":2720,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208663904,"statfs":{"total":213674622976,"available":213671837696,"internally_reserved":0,"allocated":2785280,"data_stored":2785280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957514+0000","last_change":"2026-02-25T09:25:49.962053+0000","last_active":"2026-02-25T09:25:49.957514+0000","last_peered":"2026-02-25T09:25:49.957514+0000","last_clean":"2026-02-25T09:25:49.957514+0000","last_became_active":"2026-02-25T09:25:47.983198+0000","last_became_peered":"2026-02-25T09:25:47.983198+0000","last_unstale":"2026-02-25T09:25:49.957514+0000","last_undegraded":"2026-02-25T09:25:49.957514+0000","last_fullsized":"2026-02-25T09:25:49.957514+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.960028+0000","last_change":"2026-02-25T09:25:49.962207+0000","last_active":"2026-02-25T09:25:49.960028+0000","last_peered":"2026-02-25T09:25:49.960028+0000","last_clean":"2026-02-25T09:25:49.960028+0000","last_became_active":"2026-02-25T09:25:47.976048+0000","last_became_peered":"2026-02-25T09:25:47.976048+0000","last_unstale":"2026-02-25T09:25:49.960028+0000","last_undegraded":"2026-02-25T09:25:49.960028+0000","last_fullsized":"2026-02-25T09:25:49.960028+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.959605+0000","last_change":"2026-02-25T09:25:49.962163+0000","last_active":"2026-02-25T09:25:49.959605+0000","last_peered":"2026-02-25T09:25:49.959605+0000","last_clean":"2026-02-25T09:25:49.959605+0000","last_became_active":"2026-02-25T09:25:47.988722+0000","last_became_peered":"2026-02-25T09:25:47.988722+0000","last_unstale":"2026-02-25T09:25:49.959605+0000","last_undegraded":"2026-02-25T09:25:49.959605+0000","last_fullsized":"2026-02-25T09:25:49.959605+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.958958+0000","last_change":"2026-02-25T09:25:49.961011+0000","last_active":"2026-02-25T09:25:49.958958+0000","last_peered":"2026-02-25T09:25:49.958958+0000","last_clean":"2026-02-25T09:25:49.958958+0000","last_became_active":"2026-02-25T09:25:47.979450+0000","last_became_peered":"2026-02-25T09:25:47.979450+0000","last_unstale":"2026-02-25T09:25:49.958958+0000","last_undegraded":"2026-02-25T09:25:49.958958+0000","last_fullsized":"2026-02-25T09:25:49.958958+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":81,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.958604+0000","last_change":"2026-02-25T09:25:47.990942+0000","last_active":"2026-02-25T09:25:49.958604+0000","last_peered":"2026-02-25T09:25:49.958604+0000","last_clean":"2026-02-25T09:25:49.958604+0000","last_became_active":"2026-02-25T09:25:47.975004+0000","last_became_peered":"2026-02-25T09:25:47.975004+0000","last_unstale":"2026-02-25T09:25:49.958604+0000","last_undegraded":"2026-02-25T09:25:49.958604+0000","last_fullsized":"2026-02-25T09:25:49.958604+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957262+0000","last_change":"2026-02-25T09:25:49.959961+0000","last_active":"2026-02-25T09:25:49.957262+0000","last_peered":"2026-02-25T09:25:49.957262+0000","last_clean":"2026-02-25T09:25:49.957262+0000","last_became_active":"2026-02-25T09:25:47.980413+0000","last_became_peered":"2026-02-25T09:25:47.980413+0000","last_unstale":"2026-02-25T09:25:49.957262+0000","last_undegraded":"2026-02-25T09:25:49.957262+0000","last_fullsized":"2026-02-25T09:25:49.957262+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.960723+0000","last_change":"2026-02-25T09:25:49.960723+0000","last_active":"2026-02-25T09:25:49.960723+0000","last_peered":"2026-02-25T09:25:49.960723+0000","last_clean":"2026-02-25T09:25:49.960723+0000","last_became_active":"2026-02-25T09:25:47.977450+0000","last_became_peered":"2026-02-25T09:25:47.977450+0000","last_unstale":"2026-02-25T09:25:49.960723+0000","last_undegraded":"2026-02-25T09:25:49.960723+0000","last_fullsized":"2026-02-25T09:25:49.960723+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957459+0000","last_change":"2026-02-25T09:25:49.961284+0000","last_active":"2026-02-25T09:25:49.957459+0000","last_peered":"2026-02-25T09:25:49.957459+0000","last_clean":"2026-02-25T09:25:49.957459+0000","last_became_active":"2026-02-25T09:25:47.977345+0000","last_became_peered":"2026-02-25T09:25:47.977345+0000","last_unstale":"2026-02-25T09:25:49.957459+0000","last_undegraded":"2026-02-25T09:25:49.957459+0000","last_fullsized":"2026-02-25T09:25:49.957459+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957840+0000","last_change":"2026-02-25T09:25:49.962130+0000","last_active":"2026-02-25T09:25:49.957840+0000","last_peered":"2026-02-25T09:25:49.957840+0000","last_clean":"2026-02-25T09:25:49.957840+0000","last_became_active":"2026-02-25T09:25:47.971244+0000","last_became_peered":"2026-02-25T09:25:47.971244+0000","last_unstale":"2026-02-25T09:25:49.957840+0000","last_undegraded":"2026-02-25T09:25:49.957840+0000","last_fullsized":"2026-02-25T09:25:49.957840+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253456384,"internally_reserved":0,"allocated":2510848,"data_stored":2510848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1060,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165596,"statfs":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":268,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166388,"statfs":{"total":53418655744,"available":53418381312,"internally_reserved":0,"allocated":274432,"data_stored":274432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":332,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166324,"statfs":{"total":53418655744,"available":53418315776,"internally_reserved":0,"allocated":339968,"data_stored":339968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1060,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165596,"statfs":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":53418655744,"available":53418315776,"internally_reserved":0,"allocated":339968,"data_stored":339968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T09:25:51.495 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T09:25:51.573 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:51.569+0000 7f4e041ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:51.575 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:51.574+0000 7f4e041ed640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:51.685 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:51.685 INFO:teuthology.orchestra.run.trial094.stderr:dumped all 2026-02-25T09:25:51.694 INFO:teuthology.orchestra.run.trial094.stdout:{"pg_ready":true,"pg_map":{"version":11,"stamp":"2026-02-25T09:25:50.959510+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":2720,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208663904,"statfs":{"total":213674622976,"available":213671837696,"internally_reserved":0,"allocated":2785280,"data_stored":2785280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"0.000000"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957514+0000","last_change":"2026-02-25T09:25:49.962053+0000","last_active":"2026-02-25T09:25:49.957514+0000","last_peered":"2026-02-25T09:25:49.957514+0000","last_clean":"2026-02-25T09:25:49.957514+0000","last_became_active":"2026-02-25T09:25:47.983198+0000","last_became_peered":"2026-02-25T09:25:47.983198+0000","last_unstale":"2026-02-25T09:25:49.957514+0000","last_undegraded":"2026-02-25T09:25:49.957514+0000","last_fullsized":"2026-02-25T09:25:49.957514+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.960028+0000","last_change":"2026-02-25T09:25:49.962207+0000","last_active":"2026-02-25T09:25:49.960028+0000","last_peered":"2026-02-25T09:25:49.960028+0000","last_clean":"2026-02-25T09:25:49.960028+0000","last_became_active":"2026-02-25T09:25:47.976048+0000","last_became_peered":"2026-02-25T09:25:47.976048+0000","last_unstale":"2026-02-25T09:25:49.960028+0000","last_undegraded":"2026-02-25T09:25:49.960028+0000","last_fullsized":"2026-02-25T09:25:49.960028+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.959605+0000","last_change":"2026-02-25T09:25:49.962163+0000","last_active":"2026-02-25T09:25:49.959605+0000","last_peered":"2026-02-25T09:25:49.959605+0000","last_clean":"2026-02-25T09:25:49.959605+0000","last_became_active":"2026-02-25T09:25:47.988722+0000","last_became_peered":"2026-02-25T09:25:47.988722+0000","last_unstale":"2026-02-25T09:25:49.959605+0000","last_undegraded":"2026-02-25T09:25:49.959605+0000","last_fullsized":"2026-02-25T09:25:49.959605+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.958958+0000","last_change":"2026-02-25T09:25:49.961011+0000","last_active":"2026-02-25T09:25:49.958958+0000","last_peered":"2026-02-25T09:25:49.958958+0000","last_clean":"2026-02-25T09:25:49.958958+0000","last_became_active":"2026-02-25T09:25:47.979450+0000","last_became_peered":"2026-02-25T09:25:47.979450+0000","last_unstale":"2026-02-25T09:25:49.958958+0000","last_undegraded":"2026-02-25T09:25:49.958958+0000","last_fullsized":"2026-02-25T09:25:49.958958+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":81,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.958604+0000","last_change":"2026-02-25T09:25:47.990942+0000","last_active":"2026-02-25T09:25:49.958604+0000","last_peered":"2026-02-25T09:25:49.958604+0000","last_clean":"2026-02-25T09:25:49.958604+0000","last_became_active":"2026-02-25T09:25:47.975004+0000","last_became_peered":"2026-02-25T09:25:47.975004+0000","last_unstale":"2026-02-25T09:25:49.958604+0000","last_undegraded":"2026-02-25T09:25:49.958604+0000","last_fullsized":"2026-02-25T09:25:49.958604+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957262+0000","last_change":"2026-02-25T09:25:49.959961+0000","last_active":"2026-02-25T09:25:49.957262+0000","last_peered":"2026-02-25T09:25:49.957262+0000","last_clean":"2026-02-25T09:25:49.957262+0000","last_became_active":"2026-02-25T09:25:47.980413+0000","last_became_peered":"2026-02-25T09:25:47.980413+0000","last_unstale":"2026-02-25T09:25:49.957262+0000","last_undegraded":"2026-02-25T09:25:49.957262+0000","last_fullsized":"2026-02-25T09:25:49.957262+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.960723+0000","last_change":"2026-02-25T09:25:49.960723+0000","last_active":"2026-02-25T09:25:49.960723+0000","last_peered":"2026-02-25T09:25:49.960723+0000","last_clean":"2026-02-25T09:25:49.960723+0000","last_became_active":"2026-02-25T09:25:47.977450+0000","last_became_peered":"2026-02-25T09:25:47.977450+0000","last_unstale":"2026-02-25T09:25:49.960723+0000","last_undegraded":"2026-02-25T09:25:49.960723+0000","last_fullsized":"2026-02-25T09:25:49.960723+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957459+0000","last_change":"2026-02-25T09:25:49.961284+0000","last_active":"2026-02-25T09:25:49.957459+0000","last_peered":"2026-02-25T09:25:49.957459+0000","last_clean":"2026-02-25T09:25:49.957459+0000","last_became_active":"2026-02-25T09:25:47.977345+0000","last_became_peered":"2026-02-25T09:25:47.977345+0000","last_unstale":"2026-02-25T09:25:49.957459+0000","last_undegraded":"2026-02-25T09:25:49.957459+0000","last_fullsized":"2026-02-25T09:25:49.957459+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957840+0000","last_change":"2026-02-25T09:25:49.962130+0000","last_active":"2026-02-25T09:25:49.957840+0000","last_peered":"2026-02-25T09:25:49.957840+0000","last_clean":"2026-02-25T09:25:49.957840+0000","last_became_active":"2026-02-25T09:25:47.971244+0000","last_became_peered":"2026-02-25T09:25:47.971244+0000","last_unstale":"2026-02-25T09:25:49.957840+0000","last_undegraded":"2026-02-25T09:25:49.957840+0000","last_fullsized":"2026-02-25T09:25:49.957840+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253456384,"internally_reserved":0,"allocated":2510848,"data_stored":2510848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672966,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1060,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165596,"statfs":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":268,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166388,"statfs":{"total":53418655744,"available":53418381312,"internally_reserved":0,"allocated":274432,"data_stored":274432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":332,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166324,"statfs":{"total":53418655744,"available":53418315776,"internally_reserved":0,"allocated":339968,"data_stored":339968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640261,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1060,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165596,"statfs":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":53418655744,"available":53418315776,"internally_reserved":0,"allocated":339968,"data_stored":339968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T09:25:51.695 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-25T09:25:51.695 INFO:tasks.ceph:Waiting until ceph cluster ceph is healthy... 2026-02-25T09:25:51.695 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy 2026-02-25T09:25:51.695 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph health --format=json 2026-02-25T09:25:51.770 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:51.768+0000 7fc3149b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:51.773 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:25:51.772+0000 7fc3149b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:25:51.914 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:25:51.914 INFO:teuthology.orchestra.run.trial094.stdout:{"status":"HEALTH_OK","checks":{},"mutes":[]} 2026-02-25T09:25:51.923 INFO:tasks.ceph.ceph_manager.ceph:wait_until_healthy done 2026-02-25T09:25:51.924 INFO:teuthology.run_tasks:Running task workunit... 2026-02-25T09:25:51.944 INFO:tasks.workunit:Pulling workunits from ref c32469671c6b0222f786dd36c34eb702f780a11d 2026-02-25T09:25:51.945 INFO:tasks.workunit:Making a separate scratch dir for every client... 2026-02-25T09:25:51.945 DEBUG:teuthology.orchestra.run.trial094:> stat -- /home/ubuntu/cephtest/mnt.0 2026-02-25T09:25:51.962 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T09:25:51.963 INFO:teuthology.orchestra.run.trial094.stderr:stat: cannot statx '/home/ubuntu/cephtest/mnt.0': No such file or directory 2026-02-25T09:25:51.963 DEBUG:teuthology.orchestra.run.trial094:> mkdir -- /home/ubuntu/cephtest/mnt.0 2026-02-25T09:25:52.020 INFO:tasks.workunit:Created dir /home/ubuntu/cephtest/mnt.0 2026-02-25T09:25:52.021 DEBUG:teuthology.orchestra.run.trial094:> cd -- /home/ubuntu/cephtest/mnt.0 && mkdir -- client.0 2026-02-25T09:25:52.077 INFO:tasks.workunit:timeout=3h 2026-02-25T09:25:52.077 INFO:tasks.workunit:cleanup=True 2026-02-25T09:25:52.077 DEBUG:teuthology.orchestra.run.trial094:> rm -rf /home/ubuntu/cephtest/clone.client.0 && git clone https://github.com/ceph/ceph.git /home/ubuntu/cephtest/clone.client.0 && cd /home/ubuntu/cephtest/clone.client.0 && git checkout c32469671c6b0222f786dd36c34eb702f780a11d 2026-02-25T09:25:52.139 INFO:tasks.workunit.client.0.trial094.stderr:Cloning into '/home/ubuntu/cephtest/clone.client.0'... 2026-02-25T09:26:14.970 INFO:tasks.workunit.client.0.trial094.stderr:Note: switching to 'c32469671c6b0222f786dd36c34eb702f780a11d'. 2026-02-25T09:26:14.971 INFO:tasks.workunit.client.0.trial094.stderr: 2026-02-25T09:26:14.971 INFO:tasks.workunit.client.0.trial094.stderr:You are in 'detached HEAD' state. You can look around, make experimental 2026-02-25T09:26:14.971 INFO:tasks.workunit.client.0.trial094.stderr:changes and commit them, and you can discard any commits you make in this 2026-02-25T09:26:14.971 INFO:tasks.workunit.client.0.trial094.stderr:state without impacting any branches by switching back to a branch. 2026-02-25T09:26:14.971 INFO:tasks.workunit.client.0.trial094.stderr: 2026-02-25T09:26:14.971 INFO:tasks.workunit.client.0.trial094.stderr:If you want to create a new branch to retain commits you create, you may 2026-02-25T09:26:14.971 INFO:tasks.workunit.client.0.trial094.stderr:do so (now or later) by using -c with the switch command. Example: 2026-02-25T09:26:14.971 INFO:tasks.workunit.client.0.trial094.stderr: 2026-02-25T09:26:14.972 INFO:tasks.workunit.client.0.trial094.stderr: git switch -c 2026-02-25T09:26:14.972 INFO:tasks.workunit.client.0.trial094.stderr: 2026-02-25T09:26:14.972 INFO:tasks.workunit.client.0.trial094.stderr:Or undo this operation with: 2026-02-25T09:26:14.972 INFO:tasks.workunit.client.0.trial094.stderr: 2026-02-25T09:26:14.972 INFO:tasks.workunit.client.0.trial094.stderr: git switch - 2026-02-25T09:26:14.972 INFO:tasks.workunit.client.0.trial094.stderr: 2026-02-25T09:26:14.972 INFO:tasks.workunit.client.0.trial094.stderr:Turn off this advice by setting config variable advice.detachedHead to false 2026-02-25T09:26:14.972 INFO:tasks.workunit.client.0.trial094.stderr: 2026-02-25T09:26:14.972 INFO:tasks.workunit.client.0.trial094.stderr:HEAD is now at c32469671c6 Merge pull request #66090 from rhcs-dashboard/73674-create-rule-certmgr 2026-02-25T09:26:14.974 DEBUG:teuthology.orchestra.run.trial094:> cd -- /home/ubuntu/cephtest/clone.client.0/qa/workunits && if test -e Makefile ; then make ; fi && find -executable -type f -printf '%P\0' >/home/ubuntu/cephtest/workunits.list.client.0 2026-02-25T09:26:15.030 INFO:tasks.workunit.client.0.trial094.stdout:for d in direct_io fs ; do ( cd $d ; make all ) ; done 2026-02-25T09:26:15.032 INFO:tasks.workunit.client.0.trial094.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-25T09:26:15.033 INFO:tasks.workunit.client.0.trial094.stdout:cc -Wall -Wextra -D_GNU_SOURCE direct_io_test.c -o direct_io_test 2026-02-25T09:26:15.127 INFO:tasks.workunit.client.0.trial094.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_sync_io.c -o test_sync_io 2026-02-25T09:26:15.157 INFO:tasks.workunit.client.0.trial094.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_short_dio_read.c -o test_short_dio_read 2026-02-25T09:26:15.181 INFO:tasks.workunit.client.0.trial094.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/direct_io' 2026-02-25T09:26:15.182 INFO:tasks.workunit.client.0.trial094.stdout:make[1]: Entering directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-25T09:26:15.182 INFO:tasks.workunit.client.0.trial094.stdout:cc -Wall -Wextra -D_GNU_SOURCE test_o_trunc.c -o test_o_trunc 2026-02-25T09:26:15.206 INFO:tasks.workunit.client.0.trial094.stdout:make[1]: Leaving directory '/home/ubuntu/cephtest/clone.client.0/qa/workunits/fs' 2026-02-25T09:26:15.210 DEBUG:teuthology.orchestra.run.trial094:> set -ex 2026-02-25T09:26:15.210 DEBUG:teuthology.orchestra.run.trial094:> dd if=/home/ubuntu/cephtest/workunits.list.client.0 of=/dev/stdout 2026-02-25T09:26:15.268 INFO:tasks.workunit:Running workunits matching cls/test_cls_cmpomap.sh on client.0... 2026-02-25T09:26:15.271 INFO:tasks.workunit:Running workunit cls/test_cls_cmpomap.sh... 2026-02-25T09:26:15.271 DEBUG:teuthology.orchestra.run.trial094:workunit test cls/test_cls_cmpomap.sh> mkdir -p -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && cd -- /home/ubuntu/cephtest/mnt.0/client.0/tmp && CEPH_CLI_TEST_DUP_COMMAND=1 CEPH_REF=c32469671c6b0222f786dd36c34eb702f780a11d TESTDIR="/home/ubuntu/cephtest" CEPH_ARGS="--cluster ceph" CEPH_ID="0" PATH=$PATH:/usr/sbin CEPH_BASE=/home/ubuntu/cephtest/clone.client.0 CEPH_ROOT=/home/ubuntu/cephtest/clone.client.0 CEPH_MNT=/home/ubuntu/cephtest/mnt.0 CRIMSON_COMPAT=1 adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 3h /home/ubuntu/cephtest/clone.client.0/qa/workunits/cls/test_cls_cmpomap.sh 2026-02-25T09:26:15.342 INFO:tasks.workunit.client.0.trial094.stdout:Running main() from gmock_main.cc 2026-02-25T09:26:15.343 INFO:tasks.workunit.client.0.trial094.stdout:[==========] Running 31 tests from 1 test suite. 2026-02-25T09:26:15.343 INFO:tasks.workunit.client.0.trial094.stdout:[----------] Global test environment set-up. 2026-02-25T09:26:16.987 INFO:tasks.workunit.client.0.trial094.stdout:[----------] 31 tests from CmpOmap 2026-02-25T09:26:16.987 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str 2026-02-25T09:26:17.156 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str (169 ms) 2026-02-25T09:26:17.156 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_str_default 2026-02-25T09:26:17.179 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_vals_noexist_str_default (23 ms) 2026-02-25T09:26:17.179 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64 2026-02-25T09:26:17.203 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64 (24 ms) 2026-02-25T09:26:17.204 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_vals_noexist_u64_default 2026-02-25T09:26:17.226 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_vals_noexist_u64_default (22 ms) 2026-02-25T09:26:17.226 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_vals_str 2026-02-25T09:26:17.290 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_vals_str (63 ms) 2026-02-25T09:26:17.290 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_vals_u64 2026-02-25T09:26:17.352 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_vals_u64 (62 ms) 2026-02-25T09:26:17.352 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_input 2026-02-25T09:26:17.375 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_input (23 ms) 2026-02-25T09:26:17.375 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_default 2026-02-25T09:26:17.395 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_default (19 ms) 2026-02-25T09:26:17.395 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_vals_u64_empty_default 2026-02-25T09:26:17.417 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_vals_u64_empty_default (21 ms) 2026-02-25T09:26:17.417 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_vals_u64_invalid_value 2026-02-25T09:26:17.451 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_vals_u64_invalid_value (34 ms) 2026-02-25T09:26:17.451 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_vals_at_max_keys 2026-02-25T09:26:17.452 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_vals_at_max_keys (1 ms) 2026-02-25T09:26:17.452 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_vals_over_max_keys 2026-02-25T09:26:17.452 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_vals_over_max_keys (0 ms) 2026-02-25T09:26:17.453 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str 2026-02-25T09:26:17.467 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str (14 ms) 2026-02-25T09:26:17.467 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_str_default 2026-02-25T09:26:17.507 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_str_default (39 ms) 2026-02-25T09:26:17.507 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64 2026-02-25T09:26:17.521 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64 (14 ms) 2026-02-25T09:26:17.521 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_set_vals_noexist_u64_default 2026-02-25T09:26:17.561 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_set_vals_noexist_u64_default (40 ms) 2026-02-25T09:26:17.562 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_set_vals_str 2026-02-25T09:26:17.611 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_set_vals_str (49 ms) 2026-02-25T09:26:17.612 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64 2026-02-25T09:26:17.657 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_set_vals_u64 (45 ms) 2026-02-25T09:26:17.657 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_einval 2026-02-25T09:26:17.675 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_einval (18 ms) 2026-02-25T09:26:17.675 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_set_vals_u64_eio 2026-02-25T09:26:17.691 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_set_vals_u64_eio (15 ms) 2026-02-25T09:26:17.691 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_set_vals_at_max_keys 2026-02-25T09:26:17.692 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_set_vals_at_max_keys (1 ms) 2026-02-25T09:26:17.692 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_set_vals_over_max_keys 2026-02-25T09:26:17.692 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_set_vals_over_max_keys (0 ms) 2026-02-25T09:26:17.692 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_str 2026-02-25T09:26:17.705 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_str (12 ms) 2026-02-25T09:26:17.705 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_rm_keys_noexist_u64 2026-02-25T09:26:17.719 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_rm_keys_noexist_u64 (14 ms) 2026-02-25T09:26:17.719 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_rm_keys_str 2026-02-25T09:26:17.769 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_rm_keys_str (49 ms) 2026-02-25T09:26:17.769 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64 2026-02-25T09:26:17.822 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64 (53 ms) 2026-02-25T09:26:17.823 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_einval 2026-02-25T09:26:17.840 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_einval (17 ms) 2026-02-25T09:26:17.840 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_eio 2026-02-25T09:26:17.855 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_eio (15 ms) 2026-02-25T09:26:17.855 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_rm_keys_at_max_keys 2026-02-25T09:26:17.856 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_rm_keys_at_max_keys (1 ms) 2026-02-25T09:26:17.856 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_rm_keys_over_max_keys 2026-02-25T09:26:17.856 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_rm_keys_over_max_keys (0 ms) 2026-02-25T09:26:17.856 INFO:tasks.workunit.client.0.trial094.stdout:[ RUN ] CmpOmap.cmp_rm_keys_u64_empty 2026-02-25T09:26:17.908 INFO:tasks.workunit.client.0.trial094.stdout:[ OK ] CmpOmap.cmp_rm_keys_u64_empty (51 ms) 2026-02-25T09:26:17.908 INFO:tasks.workunit.client.0.trial094.stdout:[----------] 31 tests from CmpOmap (921 ms total) 2026-02-25T09:26:17.908 INFO:tasks.workunit.client.0.trial094.stdout: 2026-02-25T09:26:17.909 INFO:tasks.workunit.client.0.trial094.stdout:[----------] Global test environment tear-down 2026-02-25T09:26:17.991 INFO:tasks.workunit.client.0.trial094.stdout:[==========] 31 tests from 1 test suite ran. (2648 ms total) 2026-02-25T09:26:17.992 INFO:tasks.workunit.client.0.trial094.stdout:[ PASSED ] 31 tests. 2026-02-25T09:26:17.994 INFO:teuthology.orchestra.run:Running command with timeout 3600 2026-02-25T09:26:17.994 DEBUG:teuthology.orchestra.run.trial094:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0/tmp 2026-02-25T09:26:18.059 INFO:tasks.workunit:Stopping ['cls/test_cls_cmpomap.sh'] on client.0... 2026-02-25T09:26:18.059 DEBUG:teuthology.orchestra.run.trial094:> sudo rm -rf -- /home/ubuntu/cephtest/workunits.list.client.0 /home/ubuntu/cephtest/clone.client.0 2026-02-25T09:26:18.418 DEBUG:teuthology.parallel:result is None 2026-02-25T09:26:18.418 DEBUG:teuthology.orchestra.run.trial094:> sudo rm -rf -- /home/ubuntu/cephtest/mnt.0/client.0 2026-02-25T09:26:18.442 INFO:tasks.workunit:Deleted dir /home/ubuntu/cephtest/mnt.0/client.0 2026-02-25T09:26:18.442 DEBUG:teuthology.orchestra.run.trial094:> rmdir -- /home/ubuntu/cephtest/mnt.0 2026-02-25T09:26:18.500 INFO:tasks.workunit:Deleted artificial mount point /home/ubuntu/cephtest/mnt.0/client.0 2026-02-25T09:26:18.501 DEBUG:teuthology.run_tasks:Unwinding manager ceph 2026-02-25T09:26:18.576 INFO:tasks.ceph.ceph_manager.ceph:waiting for clean 2026-02-25T09:26:18.577 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T09:26:18.653 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:18.648+0000 7f7236fb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:18.655 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:18.654+0000 7f7236fb8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:18.766 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:26:18.767 INFO:teuthology.orchestra.run.trial094.stderr:dumped all 2026-02-25T09:26:18.775 INFO:teuthology.orchestra.run.trial094.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2026-02-25T09:26:18.676490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":2720,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208663904,"statfs":{"total":213674622976,"available":213671837696,"internally_reserved":0,"allocated":2785280,"data_stored":2785280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001200"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957514+0000","last_change":"2026-02-25T09:25:49.962053+0000","last_active":"2026-02-25T09:25:49.957514+0000","last_peered":"2026-02-25T09:25:49.957514+0000","last_clean":"2026-02-25T09:25:49.957514+0000","last_became_active":"2026-02-25T09:25:47.983198+0000","last_became_peered":"2026-02-25T09:25:47.983198+0000","last_unstale":"2026-02-25T09:25:49.957514+0000","last_undegraded":"2026-02-25T09:25:49.957514+0000","last_fullsized":"2026-02-25T09:25:49.957514+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.960028+0000","last_change":"2026-02-25T09:25:49.962207+0000","last_active":"2026-02-25T09:25:49.960028+0000","last_peered":"2026-02-25T09:25:49.960028+0000","last_clean":"2026-02-25T09:25:49.960028+0000","last_became_active":"2026-02-25T09:25:47.976048+0000","last_became_peered":"2026-02-25T09:25:47.976048+0000","last_unstale":"2026-02-25T09:25:49.960028+0000","last_undegraded":"2026-02-25T09:25:49.960028+0000","last_fullsized":"2026-02-25T09:25:49.960028+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.959605+0000","last_change":"2026-02-25T09:25:49.962163+0000","last_active":"2026-02-25T09:25:49.959605+0000","last_peered":"2026-02-25T09:25:49.959605+0000","last_clean":"2026-02-25T09:25:49.959605+0000","last_became_active":"2026-02-25T09:25:47.988722+0000","last_became_peered":"2026-02-25T09:25:47.988722+0000","last_unstale":"2026-02-25T09:25:49.959605+0000","last_undegraded":"2026-02-25T09:25:49.959605+0000","last_fullsized":"2026-02-25T09:25:49.959605+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.958958+0000","last_change":"2026-02-25T09:25:49.961011+0000","last_active":"2026-02-25T09:25:49.958958+0000","last_peered":"2026-02-25T09:25:49.958958+0000","last_clean":"2026-02-25T09:25:49.958958+0000","last_became_active":"2026-02-25T09:25:47.979450+0000","last_became_peered":"2026-02-25T09:25:47.979450+0000","last_unstale":"2026-02-25T09:25:49.958958+0000","last_undegraded":"2026-02-25T09:25:49.958958+0000","last_fullsized":"2026-02-25T09:25:49.958958+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":81,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.958604+0000","last_change":"2026-02-25T09:25:47.990942+0000","last_active":"2026-02-25T09:25:49.958604+0000","last_peered":"2026-02-25T09:25:49.958604+0000","last_clean":"2026-02-25T09:25:49.958604+0000","last_became_active":"2026-02-25T09:25:47.975004+0000","last_became_peered":"2026-02-25T09:25:47.975004+0000","last_unstale":"2026-02-25T09:25:49.958604+0000","last_undegraded":"2026-02-25T09:25:49.958604+0000","last_fullsized":"2026-02-25T09:25:49.958604+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957262+0000","last_change":"2026-02-25T09:25:49.959961+0000","last_active":"2026-02-25T09:25:49.957262+0000","last_peered":"2026-02-25T09:25:49.957262+0000","last_clean":"2026-02-25T09:25:49.957262+0000","last_became_active":"2026-02-25T09:25:47.980413+0000","last_became_peered":"2026-02-25T09:25:47.980413+0000","last_unstale":"2026-02-25T09:25:49.957262+0000","last_undegraded":"2026-02-25T09:25:49.957262+0000","last_fullsized":"2026-02-25T09:25:49.957262+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.960723+0000","last_change":"2026-02-25T09:25:49.960723+0000","last_active":"2026-02-25T09:25:49.960723+0000","last_peered":"2026-02-25T09:25:49.960723+0000","last_clean":"2026-02-25T09:25:49.960723+0000","last_became_active":"2026-02-25T09:25:47.977450+0000","last_became_peered":"2026-02-25T09:25:47.977450+0000","last_unstale":"2026-02-25T09:25:49.960723+0000","last_undegraded":"2026-02-25T09:25:49.960723+0000","last_fullsized":"2026-02-25T09:25:49.960723+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957459+0000","last_change":"2026-02-25T09:25:49.961284+0000","last_active":"2026-02-25T09:25:49.957459+0000","last_peered":"2026-02-25T09:25:49.957459+0000","last_clean":"2026-02-25T09:25:49.957459+0000","last_became_active":"2026-02-25T09:25:47.977345+0000","last_became_peered":"2026-02-25T09:25:47.977345+0000","last_unstale":"2026-02-25T09:25:49.957459+0000","last_undegraded":"2026-02-25T09:25:49.957459+0000","last_fullsized":"2026-02-25T09:25:49.957459+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957840+0000","last_change":"2026-02-25T09:25:49.962130+0000","last_active":"2026-02-25T09:25:49.957840+0000","last_peered":"2026-02-25T09:25:49.957840+0000","last_clean":"2026-02-25T09:25:49.957840+0000","last_became_active":"2026-02-25T09:25:47.971244+0000","last_became_peered":"2026-02-25T09:25:47.971244+0000","last_unstale":"2026-02-25T09:25:49.957840+0000","last_undegraded":"2026-02-25T09:25:49.957840+0000","last_fullsized":"2026-02-25T09:25:49.957840+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253456384,"internally_reserved":0,"allocated":2510848,"data_stored":2510848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1060,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165596,"statfs":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":268,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166388,"statfs":{"total":53418655744,"available":53418381312,"internally_reserved":0,"allocated":274432,"data_stored":274432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640286,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":332,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166324,"statfs":{"total":53418655744,"available":53418315776,"internally_reserved":0,"allocated":339968,"data_stored":339968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640286,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1060,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165596,"statfs":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":53418655744,"available":53418315776,"internally_reserved":0,"allocated":339968,"data_stored":339968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T09:26:18.777 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T09:26:18.852 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:18.848+0000 7f046250b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:18.854 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:18.852+0000 7f046250b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:18.967 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:26:18.968 INFO:teuthology.orchestra.run.trial094.stderr:dumped all 2026-02-25T09:26:18.977 INFO:teuthology.orchestra.run.trial094.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2026-02-25T09:26:18.676490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":2720,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208663904,"statfs":{"total":213674622976,"available":213671837696,"internally_reserved":0,"allocated":2785280,"data_stored":2785280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001200"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957514+0000","last_change":"2026-02-25T09:25:49.962053+0000","last_active":"2026-02-25T09:25:49.957514+0000","last_peered":"2026-02-25T09:25:49.957514+0000","last_clean":"2026-02-25T09:25:49.957514+0000","last_became_active":"2026-02-25T09:25:47.983198+0000","last_became_peered":"2026-02-25T09:25:47.983198+0000","last_unstale":"2026-02-25T09:25:49.957514+0000","last_undegraded":"2026-02-25T09:25:49.957514+0000","last_fullsized":"2026-02-25T09:25:49.957514+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.960028+0000","last_change":"2026-02-25T09:25:49.962207+0000","last_active":"2026-02-25T09:25:49.960028+0000","last_peered":"2026-02-25T09:25:49.960028+0000","last_clean":"2026-02-25T09:25:49.960028+0000","last_became_active":"2026-02-25T09:25:47.976048+0000","last_became_peered":"2026-02-25T09:25:47.976048+0000","last_unstale":"2026-02-25T09:25:49.960028+0000","last_undegraded":"2026-02-25T09:25:49.960028+0000","last_fullsized":"2026-02-25T09:25:49.960028+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.959605+0000","last_change":"2026-02-25T09:25:49.962163+0000","last_active":"2026-02-25T09:25:49.959605+0000","last_peered":"2026-02-25T09:25:49.959605+0000","last_clean":"2026-02-25T09:25:49.959605+0000","last_became_active":"2026-02-25T09:25:47.988722+0000","last_became_peered":"2026-02-25T09:25:47.988722+0000","last_unstale":"2026-02-25T09:25:49.959605+0000","last_undegraded":"2026-02-25T09:25:49.959605+0000","last_fullsized":"2026-02-25T09:25:49.959605+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.958958+0000","last_change":"2026-02-25T09:25:49.961011+0000","last_active":"2026-02-25T09:25:49.958958+0000","last_peered":"2026-02-25T09:25:49.958958+0000","last_clean":"2026-02-25T09:25:49.958958+0000","last_became_active":"2026-02-25T09:25:47.979450+0000","last_became_peered":"2026-02-25T09:25:47.979450+0000","last_unstale":"2026-02-25T09:25:49.958958+0000","last_undegraded":"2026-02-25T09:25:49.958958+0000","last_fullsized":"2026-02-25T09:25:49.958958+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":81,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.958604+0000","last_change":"2026-02-25T09:25:47.990942+0000","last_active":"2026-02-25T09:25:49.958604+0000","last_peered":"2026-02-25T09:25:49.958604+0000","last_clean":"2026-02-25T09:25:49.958604+0000","last_became_active":"2026-02-25T09:25:47.975004+0000","last_became_peered":"2026-02-25T09:25:47.975004+0000","last_unstale":"2026-02-25T09:25:49.958604+0000","last_undegraded":"2026-02-25T09:25:49.958604+0000","last_fullsized":"2026-02-25T09:25:49.958604+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957262+0000","last_change":"2026-02-25T09:25:49.959961+0000","last_active":"2026-02-25T09:25:49.957262+0000","last_peered":"2026-02-25T09:25:49.957262+0000","last_clean":"2026-02-25T09:25:49.957262+0000","last_became_active":"2026-02-25T09:25:47.980413+0000","last_became_peered":"2026-02-25T09:25:47.980413+0000","last_unstale":"2026-02-25T09:25:49.957262+0000","last_undegraded":"2026-02-25T09:25:49.957262+0000","last_fullsized":"2026-02-25T09:25:49.957262+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.960723+0000","last_change":"2026-02-25T09:25:49.960723+0000","last_active":"2026-02-25T09:25:49.960723+0000","last_peered":"2026-02-25T09:25:49.960723+0000","last_clean":"2026-02-25T09:25:49.960723+0000","last_became_active":"2026-02-25T09:25:47.977450+0000","last_became_peered":"2026-02-25T09:25:47.977450+0000","last_unstale":"2026-02-25T09:25:49.960723+0000","last_undegraded":"2026-02-25T09:25:49.960723+0000","last_fullsized":"2026-02-25T09:25:49.960723+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957459+0000","last_change":"2026-02-25T09:25:49.961284+0000","last_active":"2026-02-25T09:25:49.957459+0000","last_peered":"2026-02-25T09:25:49.957459+0000","last_clean":"2026-02-25T09:25:49.957459+0000","last_became_active":"2026-02-25T09:25:47.977345+0000","last_became_peered":"2026-02-25T09:25:47.977345+0000","last_unstale":"2026-02-25T09:25:49.957459+0000","last_undegraded":"2026-02-25T09:25:49.957459+0000","last_fullsized":"2026-02-25T09:25:49.957459+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957840+0000","last_change":"2026-02-25T09:25:49.962130+0000","last_active":"2026-02-25T09:25:49.957840+0000","last_peered":"2026-02-25T09:25:49.957840+0000","last_clean":"2026-02-25T09:25:49.957840+0000","last_became_active":"2026-02-25T09:25:47.971244+0000","last_became_peered":"2026-02-25T09:25:47.971244+0000","last_unstale":"2026-02-25T09:25:49.957840+0000","last_undegraded":"2026-02-25T09:25:49.957840+0000","last_fullsized":"2026-02-25T09:25:49.957840+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253456384,"internally_reserved":0,"allocated":2510848,"data_stored":2510848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1060,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165596,"statfs":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":268,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166388,"statfs":{"total":53418655744,"available":53418381312,"internally_reserved":0,"allocated":274432,"data_stored":274432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640286,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":332,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166324,"statfs":{"total":53418655744,"available":53418315776,"internally_reserved":0,"allocated":339968,"data_stored":339968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640286,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1060,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165596,"statfs":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":53418655744,"available":53418315776,"internally_reserved":0,"allocated":339968,"data_stored":339968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T09:26:18.979 INFO:tasks.ceph.ceph_manager.ceph:clean! 2026-02-25T09:26:18.979 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T09:26:19.055 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:19.051+0000 7f7bd5251640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:19.057 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:19.056+0000 7f7bd5251640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:19.170 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:26:19.170 INFO:teuthology.orchestra.run.trial094.stderr:dumped all 2026-02-25T09:26:19.179 INFO:teuthology.orchestra.run.trial094.stdout:{"pg_ready":true,"pg_map":{"version":28,"stamp":"2026-02-25T09:26:18.676490+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":2720,"kb_used_data":2720,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208663904,"statfs":{"total":213674622976,"available":213671837696,"internally_reserved":0,"allocated":2785280,"data_stored":2785280,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001200"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957514+0000","last_change":"2026-02-25T09:25:49.962053+0000","last_active":"2026-02-25T09:25:49.957514+0000","last_peered":"2026-02-25T09:25:49.957514+0000","last_clean":"2026-02-25T09:25:49.957514+0000","last_became_active":"2026-02-25T09:25:47.983198+0000","last_became_peered":"2026-02-25T09:25:47.983198+0000","last_unstale":"2026-02-25T09:25:49.957514+0000","last_undegraded":"2026-02-25T09:25:49.957514+0000","last_fullsized":"2026-02-25T09:25:49.957514+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.960028+0000","last_change":"2026-02-25T09:25:49.962207+0000","last_active":"2026-02-25T09:25:49.960028+0000","last_peered":"2026-02-25T09:25:49.960028+0000","last_clean":"2026-02-25T09:25:49.960028+0000","last_became_active":"2026-02-25T09:25:47.976048+0000","last_became_peered":"2026-02-25T09:25:47.976048+0000","last_unstale":"2026-02-25T09:25:49.960028+0000","last_undegraded":"2026-02-25T09:25:49.960028+0000","last_fullsized":"2026-02-25T09:25:49.960028+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.959605+0000","last_change":"2026-02-25T09:25:49.962163+0000","last_active":"2026-02-25T09:25:49.959605+0000","last_peered":"2026-02-25T09:25:49.959605+0000","last_clean":"2026-02-25T09:25:49.959605+0000","last_became_active":"2026-02-25T09:25:47.988722+0000","last_became_peered":"2026-02-25T09:25:47.988722+0000","last_unstale":"2026-02-25T09:25:49.959605+0000","last_undegraded":"2026-02-25T09:25:49.959605+0000","last_fullsized":"2026-02-25T09:25:49.959605+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.958958+0000","last_change":"2026-02-25T09:25:49.961011+0000","last_active":"2026-02-25T09:25:49.958958+0000","last_peered":"2026-02-25T09:25:49.958958+0000","last_clean":"2026-02-25T09:25:49.958958+0000","last_became_active":"2026-02-25T09:25:47.979450+0000","last_became_peered":"2026-02-25T09:25:47.979450+0000","last_unstale":"2026-02-25T09:25:49.958958+0000","last_undegraded":"2026-02-25T09:25:49.958958+0000","last_fullsized":"2026-02-25T09:25:49.958958+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":81,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.958604+0000","last_change":"2026-02-25T09:25:47.990942+0000","last_active":"2026-02-25T09:25:49.958604+0000","last_peered":"2026-02-25T09:25:49.958604+0000","last_clean":"2026-02-25T09:25:49.958604+0000","last_became_active":"2026-02-25T09:25:47.975004+0000","last_became_peered":"2026-02-25T09:25:47.975004+0000","last_unstale":"2026-02-25T09:25:49.958604+0000","last_undegraded":"2026-02-25T09:25:49.958604+0000","last_fullsized":"2026-02-25T09:25:49.958604+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":16,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957262+0000","last_change":"2026-02-25T09:25:49.959961+0000","last_active":"2026-02-25T09:25:49.957262+0000","last_peered":"2026-02-25T09:25:49.957262+0000","last_clean":"2026-02-25T09:25:49.957262+0000","last_became_active":"2026-02-25T09:25:47.980413+0000","last_became_peered":"2026-02-25T09:25:47.980413+0000","last_unstale":"2026-02-25T09:25:49.957262+0000","last_undegraded":"2026-02-25T09:25:49.957262+0000","last_fullsized":"2026-02-25T09:25:49.957262+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.960723+0000","last_change":"2026-02-25T09:25:49.960723+0000","last_active":"2026-02-25T09:25:49.960723+0000","last_peered":"2026-02-25T09:25:49.960723+0000","last_clean":"2026-02-25T09:25:49.960723+0000","last_became_active":"2026-02-25T09:25:47.977450+0000","last_became_peered":"2026-02-25T09:25:47.977450+0000","last_unstale":"2026-02-25T09:25:49.960723+0000","last_undegraded":"2026-02-25T09:25:49.960723+0000","last_fullsized":"2026-02-25T09:25:49.960723+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957459+0000","last_change":"2026-02-25T09:25:49.961284+0000","last_active":"2026-02-25T09:25:49.957459+0000","last_peered":"2026-02-25T09:25:49.957459+0000","last_clean":"2026-02-25T09:25:49.957459+0000","last_became_active":"2026-02-25T09:25:47.977345+0000","last_became_peered":"2026-02-25T09:25:47.977345+0000","last_unstale":"2026-02-25T09:25:49.957459+0000","last_undegraded":"2026-02-25T09:25:49.957459+0000","last_fullsized":"2026-02-25T09:25:49.957459+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957840+0000","last_change":"2026-02-25T09:25:49.962130+0000","last_active":"2026-02-25T09:25:49.957840+0000","last_peered":"2026-02-25T09:25:49.957840+0000","last_clean":"2026-02-25T09:25:49.957840+0000","last_became_active":"2026-02-25T09:25:47.971244+0000","last_became_peered":"2026-02-25T09:25:47.971244+0000","last_unstale":"2026-02-25T09:25:49.957840+0000","last_undegraded":"2026-02-25T09:25:49.957840+0000","last_fullsized":"2026-02-25T09:25:49.957840+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253456384,"internally_reserved":0,"allocated":2510848,"data_stored":2510848,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672989,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1060,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165596,"statfs":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":268,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166388,"statfs":{"total":53418655744,"available":53418381312,"internally_reserved":0,"allocated":274432,"data_stored":274432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640286,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":332,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166324,"statfs":{"total":53418655744,"available":53418315776,"internally_reserved":0,"allocated":339968,"data_stored":339968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640286,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1060,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165596,"statfs":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":53418655744,"available":53418315776,"internally_reserved":0,"allocated":339968,"data_stored":339968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T09:26:19.180 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd dump --format=json 2026-02-25T09:26:19.256 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:19.252+0000 7fbdaa8b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:19.258 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:19.257+0000 7fbdaa8b7640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:19.372 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:26:19.373 INFO:teuthology.orchestra.run.trial094.stdout:{"epoch":18,"fsid":"c0375da7-c136-43b7-8240-9e8e7e46c4e5","created":"2026-02-25T09:25:36.641907+0000","modified":"2026-02-25T09:26:17.984629+0000","last_up_change":"2026-02-25T09:25:45.931020+0000","last_in_change":"2026-02-25T09:25:38.510236+0000","flags":"sortbitwise,recovery_deletes,purged_snapdirs,pglog_hardlimit","flags_num":5799936,"flags_set":["pglog_hardlimit","purged_snapdirs","recovery_deletes","sortbitwise"],"crush_version":5,"full_ratio":0.94999998807907104,"backfillfull_ratio":0.89999997615814209,"nearfull_ratio":0.85000002384185791,"cluster_snapshot":"","pool_max":3,"max_osd":4,"require_min_compat_client":"luminous","min_compat_client":"jewel","require_osd_release":"tentacle","allow_crimson":true,"pools":[{"pool":1,"pool_name":"rbd","create_time":"2026-02-25T09:25:46.231663+0000","flags":270337,"flags_names":"hashpspool,selfmanaged_snaps,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":8,"pg_placement_num":8,"pg_placement_num_target":8,"pg_num_target":8,"pg_num_pending":8,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"15","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":2,"snap_epoch":15,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{},"application_metadata":{"rbd":{}},"read_balance":{"score_type":"Fair distribution","score_acting":2,"score_stable":2,"optimal_score":1,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}},{"pool":2,"pool_name":".mgr","create_time":"2026-02-25T09:25:46.689416+0000","flags":262145,"flags_names":"hashpspool,crimson","type":1,"size":2,"min_size":1,"crush_rule":0,"peering_crush_bucket_count":0,"peering_crush_bucket_target":0,"peering_crush_bucket_barrier":0,"peering_crush_bucket_mandatory_member":2147483647,"is_stretch_pool":false,"object_hash":2,"pg_autoscale_mode":"off","pg_num":1,"pg_placement_num":1,"pg_placement_num_target":1,"pg_num_target":1,"pg_num_pending":1,"last_pg_merge_meta":{"source_pgid":"0.0","ready_epoch":0,"last_epoch_started":0,"last_epoch_clean":0,"source_version":"0'0","target_version":"0'0"},"last_change":"14","last_force_op_resend":"0","last_force_op_resend_prenautilus":"0","last_force_op_resend_preluminous":"0","auid":0,"snap_mode":"selfmanaged","snap_seq":0,"snap_epoch":0,"pool_snaps":[],"removed_snaps":"[]","quota_max_bytes":0,"quota_max_objects":0,"tiers":[],"tier_of":-1,"read_tier":-1,"write_tier":-1,"cache_mode":"none","target_max_bytes":0,"target_max_objects":0,"cache_target_dirty_ratio_micro":400000,"cache_target_dirty_high_ratio_micro":600000,"cache_target_full_ratio_micro":800000,"cache_min_flush_age":0,"cache_min_evict_age":0,"erasure_code_profile":"","hit_set_params":{"type":"none"},"hit_set_period":0,"hit_set_count":0,"use_gmt_hitset":true,"min_read_recency_for_promote":0,"min_write_recency_for_promote":0,"hit_set_grade_decay_rate":0,"hit_set_search_last_n":0,"grade_table":[],"stripe_width":0,"expected_num_objects":0,"fast_read":false,"nonprimary_shards":"{}","options":{"pg_num_max":32,"pg_num_min":1},"application_metadata":{"mgr":{}},"read_balance":{"score_type":"Fair distribution","score_acting":4,"score_stable":4,"optimal_score":0.5,"raw_score_acting":2,"raw_score_stable":2,"primary_affinity_weighted":1,"average_primary_affinity":1,"average_primary_affinity_weighted":1}}],"osds":[{"osd":0,"uuid":"84b58e5b-579c-49b7-9672-6379199b2e0c","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6804","nonce":889111984}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6803","nonce":889111984}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6805","nonce":889111984}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6807","nonce":889111984}]},"public_addr":"10.20.193.94:6804/889111984","cluster_addr":"10.20.193.94:6803/889111984","heartbeat_back_addr":"10.20.193.94:6805/889111984","heartbeat_front_addr":"10.20.193.94:6807/889111984","state":["exists","up"]},{"osd":1,"uuid":"47831adf-49cb-4d5d-af8a-f010094da3b4","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6802","nonce":3409185002}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6801","nonce":3409185002}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6808","nonce":3409185002}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.94:6806","nonce":3409185002}]},"public_addr":"10.20.193.94:6802/3409185002","cluster_addr":"10.20.193.94:6801/3409185002","heartbeat_back_addr":"10.20.193.94:6808/3409185002","heartbeat_front_addr":"10.20.193.94:6806/3409185002","state":["exists","up"]},{"osd":2,"uuid":"764573ae-9128-448c-a36d-b40e3e9ca850","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":11,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6805","nonce":2663299575}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6806","nonce":2663299575}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6815","nonce":2663299575}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6816","nonce":2663299575}]},"public_addr":"10.20.193.151:6805/2663299575","cluster_addr":"10.20.193.151:6806/2663299575","heartbeat_back_addr":"10.20.193.151:6815/2663299575","heartbeat_front_addr":"10.20.193.151:6816/2663299575","state":["exists","up"]},{"osd":3,"uuid":"272a2704-6263-47ab-87f5-4c0dc76ddfac","up":1,"in":1,"weight":1,"primary_affinity":1,"last_clean_begin":0,"last_clean_end":0,"up_from":10,"up_thru":16,"down_at":0,"lost_at":0,"public_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6803","nonce":4248319012}]},"cluster_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6802","nonce":4248319012}]},"heartbeat_back_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6808","nonce":4248319012}]},"heartbeat_front_addrs":{"addrvec":[{"type":"v2","addr":"10.20.193.151:6809","nonce":4248319012}]},"public_addr":"10.20.193.151:6803/4248319012","cluster_addr":"10.20.193.151:6802/4248319012","heartbeat_back_addr":"10.20.193.151:6808/4248319012","heartbeat_front_addr":"10.20.193.151:6809/4248319012","state":["exists","up"]}],"osd_xinfo":[{"osd":0,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":1,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":2,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0},{"osd":3,"down_stamp":"0.000000","laggy_probability":0,"laggy_interval":0,"features":4544132024016699391,"old_weight":0,"last_purged_snaps_scrub":"0.000000","dead_epoch":0}],"pg_upmap":[],"pg_upmap_items":[],"pg_upmap_primaries":[],"pg_temp":[],"primary_temp":[],"blocklist":{},"range_blocklist":{},"erasure_code_profiles":{"default":{"crush-failure-domain":"osd","k":"2","m":"1","plugin":"isa","technique":"reed_sol_van"}},"removed_snaps_queue":[{"pool":1,"snaps":[{"begin":2,"length":1}]}],"new_removed_snaps":[],"new_purged_snaps":[],"crush_node_flags":{},"device_class_flags":{},"stretch_mode":{"stretch_mode_enabled":false,"stretch_bucket_count":0,"degraded_stretch_mode":0,"recovering_stretch_mode":0,"stretch_mode_bucket":0}} 2026-02-25T09:26:20.382 INFO:tasks.ceph:Scrubbing osd.0 2026-02-25T09:26:20.382 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.0 config set osd_debug_deep_scrub_sleep 0 2026-02-25T09:26:20.463 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:20.456+0000 7f35a0fd3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:20.465 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:20.464+0000 7f35a0fd3640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:20.475 INFO:teuthology.orchestra.run.trial094.stdout:{ 2026-02-25T09:26:20.475 INFO:teuthology.orchestra.run.trial094.stdout: "success": "" 2026-02-25T09:26:20.475 INFO:teuthology.orchestra.run.trial094.stdout:} 2026-02-25T09:26:20.485 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 0 2026-02-25T09:26:20.563 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:20.558+0000 7f0ac05b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:20.565 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:20.563+0000 7f0ac05b8640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:20.676 INFO:teuthology.orchestra.run.trial094.stderr:instructed osd(s) 0 to deep-scrub 2026-02-25T09:26:20.687 INFO:tasks.ceph:Scrubbing osd.1 2026-02-25T09:26:20.687 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.1 config set osd_debug_deep_scrub_sleep 0 2026-02-25T09:26:20.765 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:20.759+0000 7f0f9e783640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:20.767 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:20.765+0000 7f0f9e783640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:20.777 INFO:teuthology.orchestra.run.trial094.stdout:{ 2026-02-25T09:26:20.777 INFO:teuthology.orchestra.run.trial094.stdout: "success": "" 2026-02-25T09:26:20.777 INFO:teuthology.orchestra.run.trial094.stdout:} 2026-02-25T09:26:20.785 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 1 2026-02-25T09:26:20.862 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:20.856+0000 7f33e785a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:20.864 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:20.862+0000 7f33e785a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:20.975 INFO:teuthology.orchestra.run.trial094.stderr:instructed osd(s) 1 to deep-scrub 2026-02-25T09:26:20.985 INFO:tasks.ceph:Scrubbing osd.2 2026-02-25T09:26:20.986 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.2 config set osd_debug_deep_scrub_sleep 0 2026-02-25T09:26:21.064 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:21.057+0000 7fc2ee30b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:21.067 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:21.065+0000 7fc2ee30b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:21.078 INFO:teuthology.orchestra.run.trial094.stdout:{ 2026-02-25T09:26:21.078 INFO:teuthology.orchestra.run.trial094.stdout: "success": "" 2026-02-25T09:26:21.078 INFO:teuthology.orchestra.run.trial094.stdout:} 2026-02-25T09:26:21.086 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 2 2026-02-25T09:26:21.162 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:21.158+0000 7f470721a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:21.164 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:21.162+0000 7f470721a640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:21.276 INFO:teuthology.orchestra.run.trial094.stderr:instructed osd(s) 2 to deep-scrub 2026-02-25T09:26:21.286 INFO:tasks.ceph:Scrubbing osd.3 2026-02-25T09:26:21.286 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph tell osd.3 config set osd_debug_deep_scrub_sleep 0 2026-02-25T09:26:21.362 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:21.356+0000 7fc1d8e8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:21.364 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:21.363+0000 7fc1d8e8b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:21.374 INFO:teuthology.orchestra.run.trial094.stdout:{ 2026-02-25T09:26:21.374 INFO:teuthology.orchestra.run.trial094.stdout: "success": "" 2026-02-25T09:26:21.374 INFO:teuthology.orchestra.run.trial094.stdout:} 2026-02-25T09:26:21.382 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph osd deep-scrub 3 2026-02-25T09:26:21.459 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:21.454+0000 7f580770b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:21.462 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:21.460+0000 7f580770b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:21.576 INFO:teuthology.orchestra.run.trial094.stderr:instructed osd(s) 3 to deep-scrub 2026-02-25T09:26:21.585 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T09:26:21.662 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:21.658+0000 7fc2bcf0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:21.664 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:21.663+0000 7fc2bcf0b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:21.778 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:26:21.778 INFO:teuthology.orchestra.run.trial094.stderr:dumped all 2026-02-25T09:26:21.787 INFO:teuthology.orchestra.run.trial094.stdout:{"pg_ready":true,"pg_map":{"version":29,"stamp":"2026-02-25T09:26:20.676871+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":54,"ondisk_log_size":54,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":2772,"kb_used_data":2772,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208663852,"statfs":{"total":213674622976,"available":213671784448,"internally_reserved":0,"allocated":2838528,"data_stored":2838528,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"6.001163"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957514+0000","last_change":"2026-02-25T09:25:49.962053+0000","last_active":"2026-02-25T09:25:49.957514+0000","last_peered":"2026-02-25T09:25:49.957514+0000","last_clean":"2026-02-25T09:25:49.957514+0000","last_became_active":"2026-02-25T09:25:47.983198+0000","last_became_peered":"2026-02-25T09:25:47.983198+0000","last_unstale":"2026-02-25T09:25:49.957514+0000","last_undegraded":"2026-02-25T09:25:49.957514+0000","last_fullsized":"2026-02-25T09:25:49.957514+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.960028+0000","last_change":"2026-02-25T09:25:49.962207+0000","last_active":"2026-02-25T09:25:49.960028+0000","last_peered":"2026-02-25T09:25:49.960028+0000","last_clean":"2026-02-25T09:25:49.960028+0000","last_became_active":"2026-02-25T09:25:47.976048+0000","last_became_peered":"2026-02-25T09:25:47.976048+0000","last_unstale":"2026-02-25T09:25:49.960028+0000","last_undegraded":"2026-02-25T09:25:49.960028+0000","last_fullsized":"2026-02-25T09:25:49.960028+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.959605+0000","last_change":"2026-02-25T09:25:49.962163+0000","last_active":"2026-02-25T09:25:49.959605+0000","last_peered":"2026-02-25T09:25:49.959605+0000","last_clean":"2026-02-25T09:25:49.959605+0000","last_became_active":"2026-02-25T09:25:47.988722+0000","last_became_peered":"2026-02-25T09:25:47.988722+0000","last_unstale":"2026-02-25T09:25:49.959605+0000","last_undegraded":"2026-02-25T09:25:49.959605+0000","last_fullsized":"2026-02-25T09:25:49.959605+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":24,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T09:26:18.005259+0000","last_change":"2026-02-25T09:25:49.961011+0000","last_active":"2026-02-25T09:26:18.005259+0000","last_peered":"2026-02-25T09:26:18.005259+0000","last_clean":"2026-02-25T09:26:18.005259+0000","last_became_active":"2026-02-25T09:25:47.979450+0000","last_became_peered":"2026-02-25T09:25:47.979450+0000","last_unstale":"2026-02-25T09:26:18.005259+0000","last_undegraded":"2026-02-25T09:26:18.005259+0000","last_fullsized":"2026-02-25T09:26:18.005259+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":81,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.958604+0000","last_change":"2026-02-25T09:25:47.990942+0000","last_active":"2026-02-25T09:25:49.958604+0000","last_peered":"2026-02-25T09:25:49.958604+0000","last_clean":"2026-02-25T09:25:49.958604+0000","last_became_active":"2026-02-25T09:25:47.975004+0000","last_became_peered":"2026-02-25T09:25:47.975004+0000","last_unstale":"2026-02-25T09:25:49.958604+0000","last_undegraded":"2026-02-25T09:25:49.958604+0000","last_fullsized":"2026-02-25T09:25:49.958604+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":25,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T09:26:18.004475+0000","last_change":"2026-02-25T09:25:49.959961+0000","last_active":"2026-02-25T09:26:18.004475+0000","last_peered":"2026-02-25T09:26:18.004475+0000","last_clean":"2026-02-25T09:26:18.004475+0000","last_became_active":"2026-02-25T09:25:47.980413+0000","last_became_peered":"2026-02-25T09:25:47.980413+0000","last_unstale":"2026-02-25T09:26:18.004475+0000","last_undegraded":"2026-02-25T09:26:18.004475+0000","last_fullsized":"2026-02-25T09:26:18.004475+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"14'1","reported_seq":17,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.960723+0000","last_change":"2026-02-25T09:25:49.960723+0000","last_active":"2026-02-25T09:25:49.960723+0000","last_peered":"2026-02-25T09:25:49.960723+0000","last_clean":"2026-02-25T09:25:49.960723+0000","last_became_active":"2026-02-25T09:25:47.977450+0000","last_became_peered":"2026-02-25T09:25:47.977450+0000","last_unstale":"2026-02-25T09:25:49.960723+0000","last_undegraded":"2026-02-25T09:25:49.960723+0000","last_fullsized":"2026-02-25T09:25:49.960723+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957459+0000","last_change":"2026-02-25T09:25:49.961284+0000","last_active":"2026-02-25T09:25:49.957459+0000","last_peered":"2026-02-25T09:25:49.957459+0000","last_clean":"2026-02-25T09:25:49.957459+0000","last_became_active":"2026-02-25T09:25:47.977345+0000","last_became_peered":"2026-02-25T09:25:47.977345+0000","last_unstale":"2026-02-25T09:25:49.957459+0000","last_undegraded":"2026-02-25T09:25:49.957459+0000","last_fullsized":"2026-02-25T09:25:49.957459+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":15,"reported_epoch":15,"state":"active+clean","last_fresh":"2026-02-25T09:25:49.957840+0000","last_change":"2026-02-25T09:25:49.962130+0000","last_active":"2026-02-25T09:25:49.957840+0000","last_peered":"2026-02-25T09:25:49.957840+0000","last_clean":"2026-02-25T09:25:49.957840+0000","last_became_active":"2026-02-25T09:25:47.971244+0000","last_became_peered":"2026-02-25T09:25:47.971244+0000","last_unstale":"2026-02-25T09:25:49.957840+0000","last_undegraded":"2026-02-25T09:25:49.957840+0000","last_fullsized":"2026-02-25T09:25:49.957840+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:25:46.934098+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253403136,"internally_reserved":0,"allocated":2564096,"data_stored":2564096,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":2,"ondisk_log_size":2,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949672994,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1112,"kb_used_data":1112,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165544,"statfs":{"total":53418655744,"available":53417517056,"internally_reserved":0,"allocated":1138688,"data_stored":1138688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640285,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":268,"kb_used_data":268,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166388,"statfs":{"total":53418655744,"available":53418381312,"internally_reserved":0,"allocated":274432,"data_stored":274432,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640286,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":332,"kb_used_data":332,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166324,"statfs":{"total":53418655744,"available":53418315776,"internally_reserved":0,"allocated":339968,"data_stored":339968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640286,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1060,"kb_used_data":1060,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165596,"statfs":{"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":53418655744,"available":53418315776,"internally_reserved":0,"allocated":339968,"data_stored":339968,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53417517056,"internally_reserved":0,"allocated":1138688,"data_stored":1138688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417570304,"internally_reserved":0,"allocated":1085440,"data_stored":1085440,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T09:26:21.791 INFO:tasks.ceph:pgid 1.7 last_scrub_stamp 2026-02-25T09:25:46.934098+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=25, tm_sec=46, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=26, tm_sec=19, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T09:26:21.791 INFO:tasks.ceph:pgid 1.6 last_scrub_stamp 2026-02-25T09:25:46.934098+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=25, tm_sec=46, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=26, tm_sec=19, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T09:26:21.791 INFO:tasks.ceph:pgid 1.5 last_scrub_stamp 2026-02-25T09:25:46.934098+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=25, tm_sec=46, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=26, tm_sec=19, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T09:26:21.792 INFO:tasks.ceph:pgid 1.4 last_scrub_stamp 2026-02-25T09:25:46.934098+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=25, tm_sec=46, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=26, tm_sec=19, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T09:26:21.792 INFO:tasks.ceph:pgid 2.0 last_scrub_stamp 2026-02-25T09:25:46.934098+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=25, tm_sec=46, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=26, tm_sec=19, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T09:26:21.792 INFO:tasks.ceph:pgid 1.3 last_scrub_stamp 2026-02-25T09:25:46.934098+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=25, tm_sec=46, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=26, tm_sec=19, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T09:26:21.792 INFO:tasks.ceph:pgid 1.2 last_scrub_stamp 2026-02-25T09:25:46.934098+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=25, tm_sec=46, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=26, tm_sec=19, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T09:26:21.792 INFO:tasks.ceph:pgid 1.1 last_scrub_stamp 2026-02-25T09:25:46.934098+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=25, tm_sec=46, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=26, tm_sec=19, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T09:26:21.792 INFO:tasks.ceph:pgid 1.0 last_scrub_stamp 2026-02-25T09:25:46.934098+0000 time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=25, tm_sec=46, tm_wday=2, tm_yday=56, tm_isdst=-1) <= time.struct_time(tm_year=2026, tm_mon=2, tm_mday=25, tm_hour=9, tm_min=26, tm_sec=19, tm_wday=2, tm_yday=56, tm_isdst=0) 2026-02-25T09:26:21.792 INFO:tasks.ceph:Still waiting for all pgs to be scrubbed. 2026-02-25T09:26:41.794 DEBUG:teuthology.orchestra.run.trial094:> sudo adjust-ulimits ceph-coverage /home/ubuntu/cephtest/archive/coverage timeout 120 ceph --cluster ceph pg dump --format=json 2026-02-25T09:26:41.874 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:41.869+0000 7f1dd370b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:41.877 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:41.875+0000 7f1dd370b640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:41.988 INFO:teuthology.orchestra.run.trial094.stdout: 2026-02-25T09:26:41.988 INFO:teuthology.orchestra.run.trial094.stderr:dumped all 2026-02-25T09:26:41.997 INFO:teuthology.orchestra.run.trial094.stdout:{"pg_ready":true,"pg_map":{"version":39,"stamp":"2026-02-25T09:26:40.680160+0000","last_osdmap_epoch":0,"last_pg_scan":0,"pg_stats_sum":{"stat_sum":{"num_bytes":590387,"num_objects":4,"num_object_clones":0,"num_object_copies":8,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":81,"num_write_kb":1108,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":55,"ondisk_log_size":55,"up":18,"acting":18,"num_store_stats":0},"osd_stats_sum":{"up_from":0,"seq":0,"num_pgs":0,"num_osds":4,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":208666624,"kb_used":2928,"kb_used_data":2928,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":208663696,"statfs":{"total":213674622976,"available":213671624704,"internally_reserved":0,"allocated":2998272,"data_stored":2998272,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[],"network_ping_times":[]},"pg_stats_delta":{"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":0,"available":0,"internally_reserved":0,"allocated":0,"data_stored":0,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":0,"ondisk_log_size":0,"up":0,"acting":0,"num_store_stats":0,"stamp_delta":"12.002035"},"pg_stats":[{"pgid":"1.7","version":"0'0","reported_seq":26,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T09:26:20.976703+0000","last_change":"2026-02-25T09:26:20.987178+0000","last_active":"2026-02-25T09:26:20.976703+0000","last_peered":"2026-02-25T09:26:20.976703+0000","last_clean":"2026-02-25T09:26:20.976703+0000","last_became_active":"2026-02-25T09:25:47.983198+0000","last_became_peered":"2026-02-25T09:25:47.983198+0000","last_unstale":"2026-02-25T09:26:20.976703+0000","last_undegraded":"2026-02-25T09:26:20.976703+0000","last_fullsized":"2026-02-25T09:26:20.976703+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:26:20.987126+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:26:20.987126+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.6","version":"0'0","reported_seq":26,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T09:26:20.976789+0000","last_change":"2026-02-25T09:26:20.984889+0000","last_active":"2026-02-25T09:26:20.976789+0000","last_peered":"2026-02-25T09:26:20.976789+0000","last_clean":"2026-02-25T09:26:20.976789+0000","last_became_active":"2026-02-25T09:25:47.976048+0000","last_became_peered":"2026-02-25T09:25:47.976048+0000","last_unstale":"2026-02-25T09:26:20.976789+0000","last_undegraded":"2026-02-25T09:26:20.976789+0000","last_fullsized":"2026-02-25T09:26:20.976789+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:26:20.984816+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:26:20.984816+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,3],"acting":[1,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.5","version":"0'0","reported_seq":26,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T09:26:20.975821+0000","last_change":"2026-02-25T09:26:20.984162+0000","last_active":"2026-02-25T09:26:20.975821+0000","last_peered":"2026-02-25T09:26:20.975821+0000","last_clean":"2026-02-25T09:26:20.975821+0000","last_became_active":"2026-02-25T09:25:47.988722+0000","last_became_peered":"2026-02-25T09:25:47.988722+0000","last_unstale":"2026-02-25T09:26:20.975821+0000","last_undegraded":"2026-02-25T09:26:20.975821+0000","last_fullsized":"2026-02-25T09:26:20.975821+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:26:20.984057+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:26:20.984057+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,0],"acting":[1,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]},{"pgid":"1.4","version":"0'0","reported_seq":26,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T09:26:21.576651+0000","last_change":"2026-02-25T09:26:21.584649+0000","last_active":"2026-02-25T09:26:21.576651+0000","last_peered":"2026-02-25T09:26:21.576651+0000","last_clean":"2026-02-25T09:26:21.576651+0000","last_became_active":"2026-02-25T09:25:47.979450+0000","last_became_peered":"2026-02-25T09:25:47.979450+0000","last_unstale":"2026-02-25T09:26:21.576651+0000","last_undegraded":"2026-02-25T09:26:21.576651+0000","last_fullsized":"2026-02-25T09:26:21.576651+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:26:21.584578+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:26:21.584578+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,0],"acting":[3,0],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"2.0","version":"13'52","reported_seq":92,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T09:26:20.677770+0000","last_change":"2026-02-25T09:26:20.696029+0000","last_active":"2026-02-25T09:26:20.677770+0000","last_peered":"2026-02-25T09:26:20.677770+0000","last_clean":"2026-02-25T09:26:20.677770+0000","last_became_active":"2026-02-25T09:25:47.975004+0000","last_became_peered":"2026-02-25T09:25:47.975004+0000","last_unstale":"2026-02-25T09:26:20.677770+0000","last_undegraded":"2026-02-25T09:26:20.677770+0000","last_fullsized":"2026-02-25T09:26:20.677770+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"13'52","last_scrub_stamp":"2026-02-25T09:26:20.695927+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:26:20.695927+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":52,"log_dups_size":0,"ondisk_log_size":52,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.3","version":"13'1","reported_seq":27,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T09:26:21.576490+0000","last_change":"2026-02-25T09:26:21.586545+0000","last_active":"2026-02-25T09:26:21.576490+0000","last_peered":"2026-02-25T09:26:21.576490+0000","last_clean":"2026-02-25T09:26:21.576490+0000","last_became_active":"2026-02-25T09:25:47.980413+0000","last_became_peered":"2026-02-25T09:25:47.980413+0000","last_unstale":"2026-02-25T09:26:21.576490+0000","last_undegraded":"2026-02-25T09:26:21.576490+0000","last_fullsized":"2026-02-25T09:26:21.576490+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"13'1","last_scrub_stamp":"2026-02-25T09:26:21.586495+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:26:21.586495+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":1,"log_dups_size":0,"ondisk_log_size":1,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[3,1],"acting":[3,1],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":3,"acting_primary":3,"purged_snaps":[]},{"pgid":"1.2","version":"15'2","reported_seq":28,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T09:26:20.677127+0000","last_change":"2026-02-25T09:26:20.688495+0000","last_active":"2026-02-25T09:26:20.677127+0000","last_peered":"2026-02-25T09:26:20.677127+0000","last_clean":"2026-02-25T09:26:20.677127+0000","last_became_active":"2026-02-25T09:25:47.977450+0000","last_became_peered":"2026-02-25T09:25:47.977450+0000","last_unstale":"2026-02-25T09:26:20.677127+0000","last_undegraded":"2026-02-25T09:26:20.677127+0000","last_fullsized":"2026-02-25T09:26:20.677127+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"15'2","last_scrub_stamp":"2026-02-25T09:26:20.688418+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:26:20.688418+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":2,"log_dups_size":0,"ondisk_log_size":2,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":19,"num_objects":1,"num_object_clones":0,"num_object_copies":2,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.1","version":"0'0","reported_seq":26,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T09:26:20.676991+0000","last_change":"2026-02-25T09:26:20.685859+0000","last_active":"2026-02-25T09:26:20.676991+0000","last_peered":"2026-02-25T09:26:20.676991+0000","last_clean":"2026-02-25T09:26:20.676991+0000","last_became_active":"2026-02-25T09:25:47.977345+0000","last_became_peered":"2026-02-25T09:25:47.977345+0000","last_unstale":"2026-02-25T09:26:20.676991+0000","last_undegraded":"2026-02-25T09:26:20.676991+0000","last_fullsized":"2026-02-25T09:26:20.676991+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:26:20.685772+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:26:20.685772+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[0,3],"acting":[0,3],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":0,"acting_primary":0,"purged_snaps":[]},{"pgid":"1.0","version":"0'0","reported_seq":26,"reported_epoch":18,"state":"active+clean","last_fresh":"2026-02-25T09:26:20.975353+0000","last_change":"2026-02-25T09:26:20.986935+0000","last_active":"2026-02-25T09:26:20.975353+0000","last_peered":"2026-02-25T09:26:20.975353+0000","last_clean":"2026-02-25T09:26:20.975353+0000","last_became_active":"2026-02-25T09:25:47.971244+0000","last_became_peered":"2026-02-25T09:25:47.971244+0000","last_unstale":"2026-02-25T09:26:20.975353+0000","last_undegraded":"2026-02-25T09:26:20.975353+0000","last_fullsized":"2026-02-25T09:26:20.975353+0000","mapping_epoch":12,"log_start":"0'0","ondisk_log_start":"0'0","created":12,"last_epoch_clean":13,"parent":"0.0","parent_split_bits":0,"last_scrub":"0'0","last_scrub_stamp":"2026-02-25T09:26:20.986882+0000","last_deep_scrub":"0'0","last_deep_scrub_stamp":"2026-02-25T09:26:20.986882+0000","last_clean_scrub_stamp":"2026-02-25T09:25:46.934098+0000","objects_scrubbed":0,"log_size":0,"log_dups_size":0,"ondisk_log_size":0,"stats_invalid":false,"dirty_stats_invalid":false,"omap_stats_invalid":false,"hitset_stats_invalid":false,"hitset_bytes_stats_invalid":false,"pin_stats_invalid":false,"manifest_stats_invalid":false,"snaptrimq_len":0,"last_scrub_duration":0,"scrub_schedule":"--","scrub_duration":0,"objects_trimmed":0,"snaptrim_duration":0,"stat_sum":{"num_bytes":0,"num_objects":0,"num_object_clones":0,"num_object_copies":0,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":0,"num_write_kb":0,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"up":[1,2],"acting":[1,2],"avail_no_missing":[],"object_location_counts":[],"blocked_by":[],"up_primary":1,"acting_primary":1,"purged_snaps":[]}],"pool_stats":[{"poolid":2,"num_pg":1,"stat_sum":{"num_bytes":590368,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":53,"num_read_kb":46,"num_write":79,"num_write_kb":1106,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":53418655744,"available":53417517056,"internally_reserved":0,"allocated":1138688,"data_stored":1138688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":52,"ondisk_log_size":52,"up":2,"acting":2,"num_store_stats":1},{"poolid":1,"num_pg":8,"stat_sum":{"num_bytes":19,"num_objects":2,"num_object_clones":0,"num_object_copies":4,"num_objects_missing_on_primary":0,"num_objects_missing":0,"num_objects_degraded":0,"num_objects_misplaced":0,"num_objects_unfound":0,"num_objects_dirty":0,"num_whiteouts":0,"num_read":0,"num_read_kb":0,"num_write":2,"num_write_kb":2,"num_scrub_errors":0,"num_shallow_scrub_errors":0,"num_deep_scrub_errors":0,"num_objects_recovered":0,"num_bytes_recovered":0,"num_keys_recovered":0,"num_objects_omap":0,"num_objects_hit_set_archive":0,"num_bytes_hit_set_archive":0,"num_flush":0,"num_flush_kb":0,"num_evict":0,"num_evict_kb":0,"num_promote":0,"num_flush_mode_high":0,"num_flush_mode_low":0,"num_evict_mode_some":0,"num_evict_mode_full":0,"num_objects_pinned":0,"num_legacy_snapsets":0,"num_large_omap_objects":0,"num_objects_manifest":0,"num_omap_bytes":0,"num_omap_keys":0,"num_objects_repaired":0},"store_stats":{"total":160255967232,"available":160253296640,"internally_reserved":0,"allocated":2670592,"data_stored":2670592,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"log_size":3,"ondisk_log_size":3,"up":16,"acting":16,"num_store_stats":3}],"osd_stats":[{"osd":3,"up_from":10,"seq":42949673014,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1112,"kb_used_data":1112,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165544,"statfs":{"total":53418655744,"available":53417517056,"internally_reserved":0,"allocated":1138688,"data_stored":1138688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,2],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":2,"up_from":11,"seq":47244640305,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":320,"kb_used_data":320,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166336,"statfs":{"total":53418655744,"available":53418328064,"internally_reserved":0,"allocated":327680,"data_stored":327680,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,1,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":1,"up_from":11,"seq":47244640305,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":384,"kb_used_data":384,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52166272,"statfs":{"total":53418655744,"available":53418262528,"internally_reserved":0,"allocated":393216,"data_stored":393216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[0,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]},{"osd":0,"up_from":11,"seq":47244640306,"num_pgs":0,"num_osds":1,"num_per_pool_osds":0,"num_per_pool_omap_osds":0,"kb":52166656,"kb_used":1112,"kb_used_data":1112,"kb_used_omap":0,"kb_used_meta":0,"kb_avail":52165544,"statfs":{"total":53418655744,"available":53417517056,"internally_reserved":0,"allocated":1138688,"data_stored":1138688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},"hb_peers":[1,2,3],"snap_trim_queue_len":0,"num_snap_trimming":0,"num_shards_repaired":0,"op_queue_age_hist":{"histogram":[],"upper_bound":1},"perf_stat":{"commit_latency_ms":0,"apply_latency_ms":0,"commit_latency_ns":0,"apply_latency_ns":0},"alerts":[]}],"pool_statfs":[{"poolid":1,"osd":0,"total":53418655744,"available":53417517056,"internally_reserved":0,"allocated":1138688,"data_stored":1138688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":1,"total":53418655744,"available":53418262528,"internally_reserved":0,"allocated":393216,"data_stored":393216,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":1,"osd":3,"total":53418655744,"available":53417517056,"internally_reserved":0,"allocated":1138688,"data_stored":1138688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0},{"poolid":2,"osd":0,"total":53418655744,"available":53417517056,"internally_reserved":0,"allocated":1138688,"data_stored":1138688,"data_compressed":0,"data_compressed_allocated":0,"data_compressed_original":0,"omap_allocated":0,"internal_metadata":0}]}} 2026-02-25T09:26:41.998 DEBUG:teuthology.orchestra.run.trial094:> sudo ceph --cluster ceph config set global mon_health_to_clog false 2026-02-25T09:26:42.072 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:42.066+0000 7ff54f974640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:42.074 INFO:teuthology.orchestra.run.trial094.stderr:2026-02-25T09:26:42.072+0000 7ff54f974640 -1 WARNING: the following dangerous and experimental features are enabled: crimson 2026-02-25T09:26:42.201 INFO:teuthology.misc:Shutting down mds daemons... 2026-02-25T09:26:42.201 INFO:teuthology.misc:Shutting down osd daemons... 2026-02-25T09:26:42.201 DEBUG:tasks.ceph.osd.0:waiting for process to exit 2026-02-25T09:26:42.201 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T09:26:42.264 INFO:tasks.ceph.osd.0:Stopped 2026-02-25T09:26:42.264 DEBUG:tasks.ceph.osd.1:waiting for process to exit 2026-02-25T09:26:42.264 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T09:26:42.337 INFO:tasks.ceph.osd.1:Stopped 2026-02-25T09:26:42.337 DEBUG:tasks.ceph.osd.2:waiting for process to exit 2026-02-25T09:26:42.338 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T09:26:42.399 INFO:tasks.ceph.osd.2:Stopped 2026-02-25T09:26:42.399 DEBUG:tasks.ceph.osd.3:waiting for process to exit 2026-02-25T09:26:42.399 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T09:26:42.472 INFO:tasks.ceph.osd.3:Stopped 2026-02-25T09:26:42.473 INFO:teuthology.misc:Shutting down mgr daemons... 2026-02-25T09:26:42.473 DEBUG:tasks.ceph.mgr.x:waiting for process to exit 2026-02-25T09:26:42.473 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T09:26:42.504 INFO:tasks.ceph.mgr.x:Stopped 2026-02-25T09:26:42.505 INFO:teuthology.misc:Shutting down mon daemons... 2026-02-25T09:26:42.505 DEBUG:tasks.ceph.mon.a:waiting for process to exit 2026-02-25T09:26:42.505 INFO:teuthology.orchestra.run:waiting for 300 2026-02-25T09:26:42.517 INFO:tasks.ceph.mon.a:Stopped 2026-02-25T09:26:42.517 INFO:tasks.ceph:Checking cluster log for badness... 2026-02-25T09:26:42.517 DEBUG:teuthology.orchestra.run.trial094:> sudo grep -E '\[ERR\]|\[WRN\]|\[SEC\]' /var/log/ceph/ceph.log | grep -E -v '\(MDS_ALL_DOWN\)' | grep -E -v '\(MDS_UP_LESS_THAN_MAX\)' | grep -E -v '\(POOL_APP_NOT_ENABLED\)' | head -n 1 2026-02-25T09:26:42.584 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-0 on ubuntu@trial094.front.sepia.ceph.com 2026-02-25T09:26:42.584 DEBUG:teuthology.orchestra.run.trial094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-0 2026-02-25T09:26:42.651 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-1 on ubuntu@trial094.front.sepia.ceph.com 2026-02-25T09:26:42.651 DEBUG:teuthology.orchestra.run.trial094:> sync && sudo umount -f /var/lib/ceph/osd/ceph-1 2026-02-25T09:26:42.717 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-2 on ubuntu@trial151.front.sepia.ceph.com 2026-02-25T09:26:42.717 DEBUG:teuthology.orchestra.run.trial151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-2 2026-02-25T09:26:42.747 INFO:tasks.ceph:Unmounting /var/lib/ceph/osd/ceph-3 on ubuntu@trial151.front.sepia.ceph.com 2026-02-25T09:26:42.747 DEBUG:teuthology.orchestra.run.trial151:> sync && sudo umount -f /var/lib/ceph/osd/ceph-3 2026-02-25T09:26:42.821 INFO:tasks.ceph:Archiving mon data... 2026-02-25T09:26:42.822 DEBUG:teuthology.misc:Transferring archived files from trial094:/var/lib/ceph/mon/ceph-a to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:58:25-crimson-rados-main-distro-debug-trial/69215/data/mon.a.tgz 2026-02-25T09:26:42.822 DEBUG:teuthology.orchestra.run.trial094:> mktemp 2026-02-25T09:26:42.839 INFO:teuthology.orchestra.run.trial094.stdout:/tmp/tmp.WAhU5EVyiy 2026-02-25T09:26:42.839 DEBUG:teuthology.orchestra.run.trial094:> sudo tar cz -f - -C /var/lib/ceph/mon/ceph-a -- . > /tmp/tmp.WAhU5EVyiy 2026-02-25T09:26:42.923 DEBUG:teuthology.orchestra.run.trial094:> sudo chmod 0666 /tmp/tmp.WAhU5EVyiy 2026-02-25T09:26:43.043 DEBUG:teuthology.orchestra.remote:trial094:/tmp/tmp.WAhU5EVyiy is 171KB 2026-02-25T09:26:43.091 DEBUG:teuthology.orchestra.run.trial094:> rm -fr /tmp/tmp.WAhU5EVyiy 2026-02-25T09:26:43.107 INFO:tasks.ceph:Cleaning ceph cluster... 2026-02-25T09:26:43.107 DEBUG:teuthology.orchestra.run.trial094:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-25T09:26:43.148 DEBUG:teuthology.orchestra.run.trial151:> sudo rm -rf -- /etc/ceph/ceph.conf /etc/ceph/ceph.keyring /home/ubuntu/cephtest/ceph.data /home/ubuntu/cephtest/ceph.monmap /home/ubuntu/cephtest/../*.pid 2026-02-25T09:26:43.225 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-25T09:26:43.241 INFO:teuthology.util.scanner:summary_data or yaml_file is empty! 2026-02-25T09:26:43.241 INFO:tasks.ceph:Archiving crash dumps... 2026-02-25T09:26:43.243 DEBUG:teuthology.misc:Transferring archived files from trial094:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:58:25-crimson-rados-main-distro-debug-trial/69215/remote/trial094/crash 2026-02-25T09:26:43.245 DEBUG:teuthology.orchestra.run.trial094:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-25T09:26:43.290 DEBUG:teuthology.misc:Transferring archived files from trial151:/var/lib/ceph/crash to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:58:25-crimson-rados-main-distro-debug-trial/69215/remote/trial151/crash 2026-02-25T09:26:43.291 DEBUG:teuthology.orchestra.run.trial151:> sudo tar c -f - -C /var/lib/ceph/crash -- . 2026-02-25T09:26:43.316 INFO:tasks.ceph:Compressing logs... 2026-02-25T09:26:43.316 DEBUG:teuthology.orchestra.run.trial094:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T09:26:43.334 DEBUG:teuthology.orchestra.run.trial151:> time sudo find /var/log/ceph -name '*.log' -print0 | sudo xargs --max-args=1 --max-procs=0 --verbose -0 --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T09:26:43.355 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41728.log 2026-02-25T09:26:43.368 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-mon.a.log 2026-02-25T09:26:43.368 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.41728.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41728.log.gz 2026-02-25T09:26:43.368 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40625.log 2026-02-25T09:26:43.368 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-mon.a.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41532.log 2026-02-25T09:26:43.368 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.40625.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40625.log.gz 2026-02-25T09:26:43.369 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40135.log 2026-02-25T09:26:43.369 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.41532.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.0.log 2026-02-25T09:26:43.369 INFO:teuthology.orchestra.run.trial094.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41532.log.gz 2026-02-25T09:26:43.369 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41581.log 2026-02-25T09:26:43.369 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.40135.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40135.log.gz 2026-02-25T09:26:43.369 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-osd.0.log: gzip -5 --verbose -- /var/log/ceph/ceph.tmp-client.admin.38363.log 2026-02-25T09:26:43.369 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.41581.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41581.log.gz 2026-02-25T09:26:43.369 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41238.log 2026-02-25T09:26:43.369 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph.tmp-client.admin.38363.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39866.log 2026-02-25T09:26:43.369 INFO:teuthology.orchestra.run.trial094.stderr: 67.7% -- replaced with /var/log/ceph/ceph.tmp-client.admin.38363.log.gz 2026-02-25T09:26:43.369 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.41238.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41434.log 2026-02-25T09:26:43.370 INFO:teuthology.orchestra.run.trial094.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41238.log.gz 2026-02-25T09:26:43.370 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.39866.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41385.log 2026-02-25T09:26:43.370 INFO:teuthology.orchestra.run.trial094.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.39866.log.gz 2026-02-25T09:26:43.370 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.41434.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41287.log 2026-02-25T09:26:43.370 INFO:teuthology.orchestra.run.trial094.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41434.log.gz 2026-02-25T09:26:43.370 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.41385.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40482.log 2026-02-25T09:26:43.370 INFO:teuthology.orchestra.run.trial094.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41385.log.gz 2026-02-25T09:26:43.370 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.41287.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39425.log 2026-02-25T09:26:43.370 INFO:teuthology.orchestra.run.trial094.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.41287.log.gz 2026-02-25T09:26:43.370 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.40482.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39558.log 2026-02-25T09:26:43.370 INFO:teuthology.orchestra.run.trial094.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40482.log.gz 2026-02-25T09:26:43.370 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.39425.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39937.log 2026-02-25T09:26:43.371 INFO:teuthology.orchestra.run.trial094.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39425.log.gz 2026-02-25T09:26:43.371 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.39558.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40313.log 2026-02-25T09:26:43.371 INFO:teuthology.orchestra.run.trial094.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39558.log.gz 2026-02-25T09:26:43.371 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.39937.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41140.log 2026-02-25T09:26:43.371 INFO:teuthology.orchestra.run.trial094.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.39937.log.gz 2026-02-25T09:26:43.371 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.40313.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.40313.log.gz 2026-02-25T09:26:43.371 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40037.log 2026-02-25T09:26:43.371 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.41140.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41336.log 2026-02-25T09:26:43.371 INFO:teuthology.orchestra.run.trial094.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41140.log.gz 2026-02-25T09:26:43.371 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.40037.log: gzip -5 --verbose -- /var/log/ceph/ceph.audit.log 2026-02-25T09:26:43.371 INFO:teuthology.orchestra.run.trial094.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40037.log.gz 2026-02-25T09:26:43.371 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.41336.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39511.log 2026-02-25T09:26:43.372 INFO:teuthology.orchestra.run.trial094.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41336.log.gz 2026-02-25T09:26:43.372 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph.audit.log: gzip -5 --verbose -- /var/log/ceph/ceph-osd.1.log 2026-02-25T09:26:43.372 INFO:teuthology.orchestra.run.trial094.stderr: 87.1% -- replaced with /var/log/ceph/ceph.audit.log.gz 2026-02-25T09:26:43.372 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.39511.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39766.log 2026-02-25T09:26:43.372 INFO:teuthology.orchestra.run.trial094.stderr: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39511.log.gz 2026-02-25T09:26:43.372 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-osd.1.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39964.log 2026-02-25T09:26:43.372 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.39766.log: 49.2% -- replaced with /var/log/ceph/ceph-client.admin.39766.log.gz 2026-02-25T09:26:43.372 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40674.log 2026-02-25T09:26:43.372 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.39964.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.39792.log 2026-02-25T09:26:43.372 INFO:teuthology.orchestra.run.trial094.stderr: 66.9% -- replaced with /var/log/ceph/ceph-client.admin.39964.log.gz 2026-02-25T09:26:43.372 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.40674.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40674.log.gz 2026-02-25T09:26:43.372 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40723.log 2026-02-25T09:26:43.373 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40086.log 2026-02-25T09:26:43.373 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.39792.log: 50.4% -- replaced with /var/log/ceph/ceph-client.admin.39792.log.gz 2026-02-25T09:26:43.373 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.40723.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40723.log.gz 2026-02-25T09:26:43.373 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40314.log 2026-02-25T09:26:43.373 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.40086.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41778.log 2026-02-25T09:26:43.373 INFO:teuthology.orchestra.run.trial094.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40086.log.gz 2026-02-25T09:26:43.373 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.40314.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40314.log.gz 2026-02-25T09:26:43.373 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41825.log 2026-02-25T09:26:43.373 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.41778.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40184.log 2026-02-25T09:26:43.373 INFO:teuthology.orchestra.run.trial094.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.41778.log.gz 2026-02-25T09:26:43.373 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.41825.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40997.log 2026-02-25T09:26:43.373 INFO:teuthology.orchestra.run.trial094.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41825.log.gz 2026-02-25T09:26:43.373 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.40184.log: gzip -5 --verbose -- /var/log/ceph/ceph.log 2026-02-25T09:26:43.374 INFO:teuthology.orchestra.run.trial094.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40184.log.gz 2026-02-25T09:26:43.374 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.40997.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41630.log 2026-02-25T09:26:43.374 INFO:teuthology.orchestra.run.trial094.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40997.log.gz 2026-02-25T09:26:43.374 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40481.log 2026-02-25T09:26:43.374 INFO:teuthology.orchestra.run.trial094.stderr: 82.8% -- replaced with /var/log/ceph/ceph.log.gz 2026-02-25T09:26:43.374 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.41630.log: 49.6%gzip -- replaced with /var/log/ceph/ceph-client.admin.41630.log.gz 2026-02-25T09:26:43.374 INFO:teuthology.orchestra.run.trial094.stderr: -5 --verbose -- /var/log/ceph/ceph-client.admin.40310.log 2026-02-25T09:26:43.374 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.40481.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40510.log 2026-02-25T09:26:43.374 INFO:teuthology.orchestra.run.trial094.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40481.log.gz 2026-02-25T09:26:43.374 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.40310.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40310.log.gz 2026-02-25T09:26:43.374 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40312.log 2026-02-25T09:26:43.374 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.40510.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41679.log 2026-02-25T09:26:43.375 INFO:teuthology.orchestra.run.trial094.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40510.log.gz 2026-02-25T09:26:43.375 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.40312.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41189.log 2026-02-25T09:26:43.375 INFO:teuthology.orchestra.run.trial094.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.40312.log.gz 2026-02-25T09:26:43.375 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.41679.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.41483.log 50.0% 2026-02-25T09:26:43.375 INFO:teuthology.orchestra.run.trial094.stderr: -- replaced with /var/log/ceph/ceph-client.admin.41679.log.gz 2026-02-25T09:26:43.375 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.41189.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.40483.log 2026-02-25T09:26:43.375 INFO:teuthology.orchestra.run.trial094.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41189.log.gz 2026-02-25T09:26:43.375 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.41483.log: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.41483.log.gz 2026-02-25T09:26:43.375 INFO:teuthology.orchestra.run.trial094.stderr:/var/log/ceph/ceph-client.admin.40483.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.40483.log.gz 2026-02-25T09:26:43.379 INFO:teuthology.orchestra.run.trial094.stderr: 91.3% -- replaced with /var/log/ceph/ceph-mon.a.log.gz 2026-02-25T09:26:43.387 INFO:teuthology.orchestra.run.trial151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.2.log 2026-02-25T09:26:43.387 INFO:teuthology.orchestra.run.trial151.stderr:gzip -5 --verbose -- /var/log/ceph/ceph-osd.3.log 2026-02-25T09:26:43.387 INFO:teuthology.orchestra.run.trial151.stderr:/var/log/ceph/ceph-osd.2.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38592.log 2026-02-25T09:26:43.388 INFO:teuthology.orchestra.run.trial151.stderr:/var/log/ceph/ceph-osd.3.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38639.log 2026-02-25T09:26:43.388 INFO:teuthology.orchestra.run.trial151.stderr:/var/log/ceph/ceph-client.admin.38592.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38691.log 2026-02-25T09:26:43.388 INFO:teuthology.orchestra.run.trial151.stderr: 50.0% -- replaced with /var/log/ceph/ceph-client.admin.38592.log.gz 2026-02-25T09:26:43.388 INFO:teuthology.orchestra.run.trial151.stderr:/var/log/ceph/ceph-client.admin.38639.log: gzip -5 --verbose -- /var/log/ceph/ceph-mgr.x.log 2026-02-25T09:26:43.388 INFO:teuthology.orchestra.run.trial151.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38639.log.gz 2026-02-25T09:26:43.388 INFO:teuthology.orchestra.run.trial151.stderr:/var/log/ceph/ceph-client.admin.38691.log: gzip -5 --verbose -- /var/log/ceph/ceph-client.admin.38744.log 2026-02-25T09:26:43.388 INFO:teuthology.orchestra.run.trial151.stderr: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38691.log.gz 2026-02-25T09:26:43.388 INFO:teuthology.orchestra.run.trial151.stderr:/var/log/ceph/ceph-mgr.x.log: /var/log/ceph/ceph-client.admin.38744.log: 49.6% -- replaced with /var/log/ceph/ceph-client.admin.38744.log.gz 2026-02-25T09:26:43.395 INFO:teuthology.orchestra.run.trial151.stderr: 91.1% -- replaced with /var/log/ceph/ceph-mgr.x.log.gz 2026-02-25T09:26:43.663 INFO:teuthology.orchestra.run.trial151.stderr: 95.0% -- replaced with /var/log/ceph/ceph-osd.2.log.gz 2026-02-25T09:26:43.667 INFO:teuthology.orchestra.run.trial094.stderr: 94.8% -- replaced with /var/log/ceph/ceph-osd.1.log.gz 2026-02-25T09:26:43.695 INFO:teuthology.orchestra.run.trial094.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.0.log.gz 2026-02-25T09:26:43.697 INFO:teuthology.orchestra.run.trial094.stderr: 2026-02-25T09:26:43.697 INFO:teuthology.orchestra.run.trial094.stderr:real 0m0.350s 2026-02-25T09:26:43.697 INFO:teuthology.orchestra.run.trial094.stderr:user 0m0.647s 2026-02-25T09:26:43.697 INFO:teuthology.orchestra.run.trial094.stderr:sys 0m0.067s 2026-02-25T09:26:43.711 INFO:teuthology.orchestra.run.trial151.stderr: 94.6% -- replaced with /var/log/ceph/ceph-osd.3.log.gz 2026-02-25T09:26:43.712 INFO:teuthology.orchestra.run.trial151.stderr: 2026-02-25T09:26:43.712 INFO:teuthology.orchestra.run.trial151.stderr:real 0m0.336s 2026-02-25T09:26:43.712 INFO:teuthology.orchestra.run.trial151.stderr:user 0m0.580s 2026-02-25T09:26:43.713 INFO:teuthology.orchestra.run.trial151.stderr:sys 0m0.052s 2026-02-25T09:26:43.714 INFO:tasks.ceph:Archiving logs... 2026-02-25T09:26:43.714 DEBUG:teuthology.misc:Transferring archived files from trial094:/var/log/ceph to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:58:25-crimson-rados-main-distro-debug-trial/69215/remote/trial094/log 2026-02-25T09:26:43.715 DEBUG:teuthology.orchestra.run.trial094:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-25T09:26:43.952 DEBUG:teuthology.misc:Transferring archived files from trial151:/var/log/ceph to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:58:25-crimson-rados-main-distro-debug-trial/69215/remote/trial151/log 2026-02-25T09:26:43.953 DEBUG:teuthology.orchestra.run.trial151:> sudo tar c -f - -C /var/log/ceph -- . 2026-02-25T09:26:44.095 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-25T09:26:44.220 DEBUG:teuthology.run_tasks:Unwinding manager install 2026-02-25T09:26:44.231 INFO:teuthology.task.install.util:Removing shipped files: /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer... 2026-02-25T09:26:44.232 DEBUG:teuthology.orchestra.run.trial094:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-25T09:26:44.233 DEBUG:teuthology.orchestra.run.trial151:> sudo rm -f -- /home/ubuntu/cephtest/valgrind.supp /usr/bin/daemon-helper /usr/bin/adjust-ulimits /usr/bin/stdin-killer 2026-02-25T09:26:44.259 DEBUG:teuthology.run_tasks:Unwinding manager clock 2026-02-25T09:26:44.270 INFO:teuthology.task.clock:Checking final clock skew... 2026-02-25T09:26:44.270 DEBUG:teuthology.orchestra.run.trial094:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T09:26:44.299 DEBUG:teuthology.orchestra.run.trial151:> PATH=/usr/bin:/usr/sbin ntpq -p || PATH=/usr/bin:/usr/sbin chronyc sources || true 2026-02-25T09:26:44.311 INFO:teuthology.orchestra.run.trial094.stderr:bash: line 1: ntpq: command not found 2026-02-25T09:26:44.313 INFO:teuthology.orchestra.run.trial151.stderr:bash: line 1: ntpq: command not found 2026-02-25T09:26:44.332 INFO:teuthology.orchestra.run.trial094.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T09:26:44.332 INFO:teuthology.orchestra.run.trial094.stdout:=============================================================================== 2026-02-25T09:26:44.332 INFO:teuthology.orchestra.run.trial094.stdout:^+ 198.71.50.75 3 6 377 41 +378us[ +354us] +/- 36ms 2026-02-25T09:26:44.332 INFO:teuthology.orchestra.run.trial094.stdout:^+ 149.28.200.179.vultruser> 2 6 377 41 -3193us[-3217us] +/- 44ms 2026-02-25T09:26:44.333 INFO:teuthology.orchestra.run.trial094.stdout:^+ time.trtnw.net 2 6 377 40 -152us[ -176us] +/- 39ms 2026-02-25T09:26:44.333 INFO:teuthology.orchestra.run.trial094.stdout:^* 50.218.103.254 2 6 377 38 +1489us[+1479us] +/- 26ms 2026-02-25T09:26:44.337 INFO:teuthology.orchestra.run.trial151.stdout:MS Name/IP address Stratum Poll Reach LastRx Last sample 2026-02-25T09:26:44.337 INFO:teuthology.orchestra.run.trial151.stdout:=============================================================================== 2026-02-25T09:26:44.337 INFO:teuthology.orchestra.run.trial151.stdout:^- 198.71.50.75 3 6 377 39 +374us[ +374us] +/- 36ms 2026-02-25T09:26:44.337 INFO:teuthology.orchestra.run.trial151.stdout:^- ns2.fortrockdc.com 2 6 377 104 +2056us[+2071us] +/- 40ms 2026-02-25T09:26:44.337 INFO:teuthology.orchestra.run.trial151.stdout:^- time.trtnw.net 2 6 377 38 -137us[ -137us] +/- 39ms 2026-02-25T09:26:44.337 INFO:teuthology.orchestra.run.trial151.stdout:^* nyc2.us.ntp.li 2 6 377 40 +62us[ +80us] +/- 2996us 2026-02-25T09:26:44.338 DEBUG:teuthology.run_tasks:Unwinding manager ansible.cephlab 2026-02-25T09:26:44.351 INFO:teuthology.task.ansible:Skipping ansible cleanup... 2026-02-25T09:26:44.351 DEBUG:teuthology.run_tasks:Unwinding manager selinux 2026-02-25T09:26:44.364 DEBUG:teuthology.orchestra.run.trial094:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-25T09:26:44.428 DEBUG:teuthology.orchestra.run.trial151:> mkdir /home/ubuntu/cephtest/archive/audit && sudo cp /var/log/audit/audit.log /home/ubuntu/cephtest/archive/audit && sudo chown $USER /home/ubuntu/cephtest/archive/audit/audit.log && gzip /home/ubuntu/cephtest/archive/audit/audit.log 2026-02-25T09:26:44.482 DEBUG:teuthology.orchestra.run.trial094:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T09:26:44.509 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T09:26:44.510 DEBUG:teuthology.orchestra.run.trial151:> sudo grep -a 'avc: .*denied' /var/log/audit/audit.log | grep -av -e 'comm="dmidecode"' -e chronyd.service -e 'name="cephtest"' -e scontext=system_u:system_r:nrpe_t:s0 -e scontext=system_u:system_r:pcp_pmlogger_t -e scontext=system_u:system_r:pcp_pmcd_t:s0 -e 'comm="rhsmd"' -e scontext=system_u:system_r:syslogd_t:s0 -e tcontext=system_u:system_r:nrpe_t:s0 -e 'comm="updatedb"' -e 'comm="smartd"' -e 'comm="rhsmcertd-worke"' -e 'comm="setroubleshootd"' -e 'comm="rpm"' -e tcontext=system_u:object_r:container_runtime_exec_t:s0 -e 'comm="ksmtuned"' -e 'comm="sssd"' -e 'comm="sss_cache"' -e context=system_u:system_r:NetworkManager_dispatcher_t:s0 -e context=system_u:system_r:getty_t:s0 2026-02-25T09:26:44.536 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T09:26:44.537 DEBUG:teuthology.run_tasks:Unwinding manager pcp 2026-02-25T09:26:44.553 DEBUG:teuthology.run_tasks:Unwinding manager internal.timer 2026-02-25T09:26:44.567 INFO:teuthology.task.internal:Duration was 1063.097408 seconds 2026-02-25T09:26:44.568 DEBUG:teuthology.run_tasks:Unwinding manager internal.syslog 2026-02-25T09:26:44.583 INFO:teuthology.task.internal.syslog:Shutting down syslog monitoring... 2026-02-25T09:26:44.583 DEBUG:teuthology.orchestra.run.trial094:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-25T09:26:44.586 DEBUG:teuthology.orchestra.run.trial151:> sudo rm -f -- /etc/rsyslog.d/80-cephtest.conf && sudo service rsyslog restart 2026-02-25T09:26:44.619 INFO:teuthology.orchestra.run.trial094.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T09:26:44.621 INFO:teuthology.orchestra.run.trial151.stderr:Redirecting to /bin/systemctl restart rsyslog.service 2026-02-25T09:26:45.019 INFO:teuthology.task.internal.syslog:Checking logs for errors... 2026-02-25T09:26:45.020 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial094.front.sepia.ceph.com 2026-02-25T09:26:45.020 DEBUG:teuthology.orchestra.run.trial094:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-25T09:26:45.077 DEBUG:teuthology.task.internal.syslog:Checking ubuntu@trial151.front.sepia.ceph.com 2026-02-25T09:26:45.078 DEBUG:teuthology.orchestra.run.trial151:> grep -E --binary-files=text '\bBUG\b|\bINFO\b|\bDEADLOCK\b' /home/ubuntu/cephtest/archive/syslog/kern.log | grep -v 'task .* blocked for more than .* seconds' | grep -v 'lockdep is turned off' | grep -v 'trying to register non-static key' | grep -v 'DEBUG: fsize' | grep -v CRON | grep -v 'BUG: bad unlock balance detected' | grep -v 'inconsistent lock state' | grep -v '*** DEADLOCK ***' | grep -v 'INFO: possible irq lock inversion dependency detected' | grep -v 'INFO: NMI handler (perf_event_nmi_handler) took too long to run' | grep -v 'INFO: recovery required on readonly' | grep -v 'ceph-create-keys: INFO' | grep -v INFO:ceph-create-keys | grep -v 'Loaded datasource DataSourceOpenStack' | grep -v 'container-storage-setup: INFO: Volume group backing root filesystem could not be determined' | grep -E -v '\bsalt-master\b|\bsalt-minion\b|\bsalt-api\b' | grep -v ceph-crash | grep -E -v '\btcmu-runner\b.*\bINFO\b' | head -n 1 2026-02-25T09:26:45.098 INFO:teuthology.task.internal.syslog:Gathering journactl... 2026-02-25T09:26:45.098 DEBUG:teuthology.orchestra.run.trial094:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T09:26:45.120 DEBUG:teuthology.orchestra.run.trial151:> sudo journalctl > /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T09:26:45.198 INFO:teuthology.task.internal.syslog:Compressing syslogs... 2026-02-25T09:26:45.198 DEBUG:teuthology.orchestra.run.trial094:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T09:26:45.200 DEBUG:teuthology.orchestra.run.trial151:> find /home/ubuntu/cephtest/archive/syslog -name '*.log' -print0 | sudo xargs -0 --max-args=1 --max-procs=0 --verbose --no-run-if-empty -- gzip -5 --verbose -- 2026-02-25T09:26:45.221 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T09:26:45.222 INFO:teuthology.orchestra.run.trial094.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T09:26:45.222 INFO:teuthology.orchestra.run.trial094.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T09:26:45.222 INFO:teuthology.orchestra.run.trial094.stderr: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-25T09:26:45.222 INFO:teuthology.orchestra.run.trial094.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-25T09:26:45.223 INFO:teuthology.orchestra.run.trial151.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/misc.log 2026-02-25T09:26:45.224 INFO:teuthology.orchestra.run.trial151.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/kern.log 2026-02-25T09:26:45.224 INFO:teuthology.orchestra.run.trial151.stderr:/home/ubuntu/cephtest/archive/syslog/misc.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/misc.log.gz 2026-02-25T09:26:45.224 INFO:teuthology.orchestra.run.trial151.stderr:gzip -5 --verbose -- /home/ubuntu/cephtest/archive/syslog/journalctl.log 2026-02-25T09:26:45.225 INFO:teuthology.orchestra.run.trial151.stderr:/home/ubuntu/cephtest/archive/syslog/kern.log: 0.0% -- replaced with /home/ubuntu/cephtest/archive/syslog/kern.log.gz 2026-02-25T09:26:45.232 INFO:teuthology.orchestra.run.trial094.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 82.1% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-25T09:26:45.234 INFO:teuthology.orchestra.run.trial151.stderr:/home/ubuntu/cephtest/archive/syslog/journalctl.log: 81.8% -- replaced with /home/ubuntu/cephtest/archive/syslog/journalctl.log.gz 2026-02-25T09:26:45.235 DEBUG:teuthology.run_tasks:Unwinding manager internal.sudo 2026-02-25T09:26:45.252 INFO:teuthology.task.internal:Restoring /etc/sudoers... 2026-02-25T09:26:45.253 DEBUG:teuthology.orchestra.run.trial094:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-25T09:26:45.299 DEBUG:teuthology.orchestra.run.trial151:> sudo mv -f /etc/sudoers.orig.teuthology /etc/sudoers 2026-02-25T09:26:45.325 DEBUG:teuthology.run_tasks:Unwinding manager internal.coredump 2026-02-25T09:26:45.343 DEBUG:teuthology.orchestra.run.trial094:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-25T09:26:45.345 DEBUG:teuthology.orchestra.run.trial151:> sudo sysctl -w kernel.core_pattern=core && sudo bash -c 'for f in `find /home/ubuntu/cephtest/archive/coredump -type f`; do file $f | grep -q systemd-sysusers && rm $f || true ; done' && rmdir --ignore-fail-on-non-empty -- /home/ubuntu/cephtest/archive/coredump 2026-02-25T09:26:45.366 INFO:teuthology.orchestra.run.trial094.stdout:kernel.core_pattern = core 2026-02-25T09:26:45.389 INFO:teuthology.orchestra.run.trial151.stdout:kernel.core_pattern = core 2026-02-25T09:26:45.406 DEBUG:teuthology.orchestra.run.trial094:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-25T09:26:45.434 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T09:26:45.435 DEBUG:teuthology.orchestra.run.trial151:> test -e /home/ubuntu/cephtest/archive/coredump 2026-02-25T09:26:45.462 DEBUG:teuthology.orchestra.run:got remote process result: 1 2026-02-25T09:26:45.462 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive 2026-02-25T09:26:45.479 INFO:teuthology.task.internal:Transferring archived files... 2026-02-25T09:26:45.480 DEBUG:teuthology.misc:Transferring archived files from trial094:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:58:25-crimson-rados-main-distro-debug-trial/69215/remote/trial094 2026-02-25T09:26:45.480 DEBUG:teuthology.orchestra.run.trial094:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-25T09:26:45.516 DEBUG:teuthology.misc:Transferring archived files from trial151:/home/ubuntu/cephtest/archive to /home/teuthworker/mnt/teuthology/matan-2026-02-25_07:58:25-crimson-rados-main-distro-debug-trial/69215/remote/trial151 2026-02-25T09:26:45.516 DEBUG:teuthology.orchestra.run.trial151:> sudo tar c -f - -C /home/ubuntu/cephtest/archive -- . 2026-02-25T09:26:45.553 INFO:teuthology.task.internal:Removing archive directory... 2026-02-25T09:26:45.553 DEBUG:teuthology.orchestra.run.trial094:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-25T09:26:45.556 DEBUG:teuthology.orchestra.run.trial151:> rm -rf -- /home/ubuntu/cephtest/archive 2026-02-25T09:26:45.601 DEBUG:teuthology.run_tasks:Unwinding manager internal.archive_upload 2026-02-25T09:26:45.618 INFO:teuthology.task.internal:Not uploading archives. 2026-02-25T09:26:45.619 DEBUG:teuthology.run_tasks:Unwinding manager internal.base 2026-02-25T09:26:45.635 INFO:teuthology.task.internal:Tidying up after the test... 2026-02-25T09:26:45.635 DEBUG:teuthology.orchestra.run.trial094:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-25T09:26:45.638 DEBUG:teuthology.orchestra.run.trial151:> find /home/ubuntu/cephtest -ls ; rmdir -- /home/ubuntu/cephtest 2026-02-25T09:26:45.653 INFO:teuthology.orchestra.run.trial094.stdout: 920834 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 25 09:26 /home/ubuntu/cephtest 2026-02-25T09:26:45.657 INFO:teuthology.orchestra.run.trial151.stdout: 920834 4 drwxr-xr-x 2 ubuntu ubuntu 4096 Feb 25 09:26 /home/ubuntu/cephtest 2026-02-25T09:26:45.658 DEBUG:teuthology.run_tasks:Unwinding manager kernel 2026-02-25T09:26:45.675 DEBUG:teuthology.run_tasks:Unwinding manager console_log 2026-02-25T09:26:45.715 INFO:teuthology.run:Summary data: description: crimson-rados/basic/{0-crimson_install clusters/crimson-fixed-2 crimson-supported-all-distro/centos_latest crimson_qa_overrides deploy/ceph objectstore/seastore/segmented$/{crimson_seastore_segmented_2q} tasks/cls} duration: 1063.0974082946777 flavor: debug owner: scheduled_matan@soko04.front.sepia.ceph.com success: true 2026-02-25T09:26:45.716 DEBUG:teuthology.report:Pushing job info to https://paddles-paddles.apps.pok.os.sepia.ceph.com 2026-02-25T09:26:45.828 INFO:teuthology.run:pass